So the model reads the page and tells the harness what to do?

We covered the core of a script⁵⁰ last time, so a short recap. It is a plain text file of commands, run by an interpreter⁵¹ instead of being compiled. It started at Bell Labs in the 1970s: the Unix shell read commands you typed, and it could just as easily read them from a file.

• The [[shebang]]. #!/bin/bash or #!/usr/bin/env node tells the kernel²⁹ which interpreter gets the file.
• Commands in sequence. What you would type: cd, npm install, next build.
• Variables. NAME="x", then $NAME.
• Control flow. if, for, while, which turn a recording into a program.
• The execute bit. chmod +x, and a place on PATH²⁴ if you want to call it by name.

Now Grok Bot. The page says Bots have their own computer, sign in to your apps and use them as you would, run in parallel all day and night with your laptop shut, and can watch you do a workflow once and save it as a routine. Take the marketing away and it is built from pieces you know:

• The "own computer" is a Linux box in a data centre. The same folders, PATH and shell, set up by a script that installs Node, Chrome and Python.
• The model is the interpreter and the task is the script, sort of. A classic script is fixed. An agent replaces the list with a loop: look at the screen or the terminal, ask the model for the next command, run it. The model writes the next line on the fly and the harness⁰³ runs it with ordinary tools.
• "Sign in once" is stored state. Cookies and tokens on that machine, a Chrome profile that never gets wiped.
• Routines are scripts again. It records the actions as a stored plan and replays it on a schedule, with something like cron⁸⁶ on their server.
• Parallel Bots are separate processes or containers. Each has its own files and its own PATH, so they don't step on each other.

I can't tell you xAI's actual internals. The page is marketing, not a technical write-up. But this is how computer-use agents are generally built, including Claude Code, which is a Node script with a shebang, sitting on your PATH, running a loop that turns model output into shell commands.

To try it: write a five-line bash script, put it in /usr/local/bin, schedule it with cron, then swap one fixed line for a call to an API that decides what that line should be. That is a tiny Grok Bot.

Two separate mechanisms, which people blur together: how the bot takes in the page and acts on it, and how a recording becomes a routine.

The [[dom]]. A web page isn't a picture but a tree of elements. Chrome has a debugging channel, the Chrome DevTools Protocol⁸³. A library like playwright⁸¹ uses it to ask for every button and input on the page, and gets back text. That list goes to the model, which replies with something like {"action": "click", "target": "login-btn"}. The harness turns that into page.click('#login-btn').

Screenshots. For things with no readable DOM, a native app or a canvas, the harness sends a picture to a vision model, which answers in coordinates: {"action": "click", "x": 412, "y": 233}. The harness fires the click through the OS accessibility layer, xdotool on Linux and the Accessibility API on a Mac, then takes another screenshot. Slower and less reliable, but it works on anything a person could click.

The terminal is simplest: the model emits a command, the harness runs it in a shell²², and what it prints goes back to the model as text.

A recording happens at two levels. Low down, every click, keystroke and navigation is logged with its target and a screenshot, the data test recorders have captured for years. Raw, it is literally a script: navigate, click button[data-test="filter-open"], type input[name="search"] "refund". Replayed word for word, it is a macro, and a macro breaks when the button moves.

Higher up, the model takes that log and writes down what you were doing: open Zendesk, filter tickets tagged refund from the last seven days, export them, append them to a sheet, post a summary to Slack. That is what gets stored, with the recording as a hint. On replay the model treats the steps as goals and picks its actions live. The script says exactly what; the routine says what for, and the model fills in how. A scheduler triggers it, and when the model isn't confident it pauses and messages you.

To see the raw layer, run npx playwright codegen https://example.com. A browser opens, you click around, and a script writes itself in a side window.

Playwright isn't a browser. It is a library, an npm package, that remote-controls one. It launches an ordinary Chromium, Firefox or WebKit and talks to it over a socket using the Chrome DevTools Protocol. Five lines of Node and a script is driving Chrome. It was built for automated testing (Microsoft, 2020; Puppeteer from Google before it; Selenium since 2004), and agents repurposed it. When I say harness, picture a script like that, with a model supplying the click targets.

Browserbase doesn't invent a new browser either. It runs regular headless⁸⁵ Chromium in the cloud and hands you a connection: chromium.connectOverCDP(...) instead of chromium.launch(), and your script drives a browser in their data centre. What they add is the hard operational part: many browsers at once, sessions that stay logged in, proxies and stealth, recordings. They also ship Stagehand, so you can write page.act("click the login button"). Think of it as AWS for browsers.

Comet and Atlas are the same again, packaged for people: Chromium with the harness built into a sidebar. They already have your logins and a way to approve actions. So to your question: there is nothing current browsers can't do. Chrome could always be scripted. The new part is that the script's author is now a model, at run time, instead of a developer, in advance.

Yes, really. It sounds too simple because it is. The harness gets an accessibility snapshot⁸⁴, a compact text form of the page, builds a prompt round it, and the model replies with one action. The harness carries it out, takes a fresh snapshot and sends it back. When the model says done, the harness stops.

- heading "Sign in to Zendesk"
- textbox "Email" [ref=e12]
- textbox "Password" [ref=e13]
- button "Sign in" [ref=e14]
- link "Forgot password?" [ref=e15]

That is the entire architecture: a text-in, text-out model, a page flattened into text, and a while loop. It works because models got good enough to look at a list of buttons, pick the right one, and recover when the page doesn't do what they expected. Screenshots are the fallback, and some harnesses send both, but DOM as text is the workhorse.

Your Next.js dev server emits HTML, Chrome parses it into a DOM, Playwright reads it back out as text, and the model reads that text. Run npx playwright codegen http://localhost:3000 and you'll see the accessibility snapshot, looking exactly like the one above.

Exactly. Every action is text the model generated, and text is tokens⁵⁹. {"action":"click","ref":"e14"} is about a dozen. The model only emits a string; the harness parses it and Playwright does the clicking. A 40-step task is 40 short completions, each billed like any chat reply.

The input is where tokens pile up. Each turn sends the goal, the current page, the history of actions so far, and a screenshot if used, which is a fixed chunk of tokens, often more than the DOM text. Agent products are metered on how many times the loop runs times how big the page is.

- heading "Issues" [level=1]
- link "New issue"
- link "Open (36)"
- link "Closed (18,369)"
- button "Filter by author": Author
- button "Filter by labels": Labels

• Accessibility snapshots instead of raw HTML. Raw HTML of a Zendesk page might be 200,000 tokens and the flattened list 2,000. That is a cost decision as much as a clarity one.
• Summarising history. After 30 steps the harness usually replaces the log with a short summary, so the context doesn't grow without bound.
• Shell commands over clicking. If one curl replaces five pages, that is five fewer snapshots to pay for.

A classic script is written once and runs for free for ever. An agent's script is regenerated every run, one token at a time. Scripts are cheap and brittle; agents are expensive and adaptive. Routines are an attempt at both: record the cheap fixed version and spend tokens only when the recording stops matching the page.

Default to the API whenever one exists and you can get access. An API call is one step: curl https://api.zendesk.com/v2/tickets?tags=refund returns clean JSON in one token-cheap round trip. By clicking, the same list is maybe 8 to 15 turns of the loop, each sending a page to the model. So the API is roughly 10 times cheaper and faster, and deterministic: same request, same response. A UI can change on Tuesday and break your Monday routine; a versioned API won't.

It is also safer. An API key can be scoped to reading tickets only. A logged-in browser session can do anything your account can, including deleting things, because the model is sitting at your keyboard.

When clicking is the right call. First, there is no API: internal tools, government portals, legacy dashboards, small products. Second, there is one but you can't get to it: enterprise plan only, admin approval needed, no keys issued.

Use the API:

• Pull last week's support tickets tagged refund. Zendesk, Intercom, Linear and Jira all have REST APIs.
• Post a message to Slack. One webhook call.
• Create or update rows in Airtable, Notion or Google Sheets.
• Send email or read an inbox, with the Gmail API.
• Check a repository, open a pull request, read CI status, with GitHub's API or git and gh.
• Fetch weather, stock prices, exchange rates, shipping status.
• Anything you'll run every day: the cost difference adds up and the stability matters.

A rule of thumb: if a developer could write it as a 20-line Node script that never opens a browser, the agent should do that too.

Use the browser:

• Government portals: a permit, a tax form, a visa status. No API, ever.
• Vendor back offices: the freight company's dashboard, the printer's order portal, the landlord's rent portal.
• Booking: restaurants, hotels and clinic slots without an API.
• LinkedIn-type sites that deliberately have no public API: researching a lead, sending a connection note.
• Comparison shopping across twenty stores with twenty layouts.
• Sites with CAPTCHAs, MFA or login walls, where you'd rather hand the bot your session.
• A tool you've never used, before an integration exists.
• The internal tool your company built in 2017 that only exists as a web page.

Mixed, the realistic case. The weekly report: get ticket data from the API; aggregate it in a shell or Node script, with no model needed per row; paste it into the internal dashboard with the browser, because nothing else reaches it; post the summary to Slack through the API. Ask one question of any task: is there a URL that returns JSON for this? If yes, use that. If no, click.