# WebFlow Authoring Guide

This guide walks through creating, testing, and managing WebFlows — the
browser agent's system for recorded, parameterized browser automation. For
architecture details, see the WebFlow section in
[browserScenarios.md](../../../../docs/architecture/browserScenarios.md).

---

## What is a WebFlow?

A WebFlow is a reusable browser automation script that:

- Was **recorded** from user interactions or **generated** by an AI agent
- Has **named parameters** that can be filled with different values each run
- Has **grammar patterns** so it can be triggered with natural language
- Is **scoped** to a specific site or available globally

Example: A "search-amazon" WebFlow might have a `query` parameter and
grammar pattern "search amazon for $(query:wildcard)", so saying "search
amazon for wireless headphones" replays the recorded search flow with the
new query.

---

## Creating a WebFlow by Recording

### Step 1: Start recording

**Via chat panel:**
Type in the side panel chat:

```
start recording
```

**Via CLI:**

```
@browser actions record "my-flow-name"
```

The extension badge changes to indicate recording is active.

### Step 2: Perform the actions

Interact with the page normally. The content script captures:

- **Clicks** — element ID, CSS selector, coordinates, bounding box
- **Text input** — value, target selector
- **Navigation** — URL changes, page loads
- **Scroll** — scroll position (captured but used for context only)

Each action also captures an HTML snapshot and annotated screenshot for
context.

### Step 3: Stop recording

**Via chat panel:**

```
stop recording
```

**Via CLI:**

```
@browser actions stop recording
```

### Step 4: Generate the WebFlow

After stopping, create the WebFlow:

**Via chat panel:**

```
create webflow "search-product" described as "Search for a product on the current site"
```

This triggers the generation pipeline:

1. **Normalization** — Deduplicates actions (e.g., multiple scrolls
   consolidated), merges consecutive text inputs on the same field
2. **Script generation** — LLM analyzes the recorded steps, HTML
   snapshots, and screenshots to produce a parameterized async function
3. **Validation** — Static analysis checks for forbidden operations
   (eval, fetch, document access, etc.)
4. **Storage** — Flow definition + script saved to disk, grammar and
   schema registered with dispatcher

### Step 5: Test the WebFlow

Try invoking it with natural language:

```
search amazon for bluetooth speaker
```

Or execute directly:

```
@browser run webflow "search-product" with query="bluetooth speaker"
```

---

## WebFlow Definition Structure

A stored WebFlow has this shape:

```typescript
{
    name: "search-product",
    description: "Search for a product on the current site",
    version: 1,
    parameters: {
        query: {
            type: "string",
            required: true,
            description: "Product to search for",
            default: undefined,
            valueOptions: undefined
        }
    },
    script: "async function execute(browser, params) { ... }",
    grammarPatterns: [
        "search for $(query:wildcard)",
        "find $(query:wildcard) on this site"
    ],
    scope: {
        type: "site",
        domains: ["amazon.com"],
        urlPatterns: undefined
    },
    source: "recording"
}
```

### Scope types

| Scope    | Behavior                                             |
| -------- | ---------------------------------------------------- |
| `site`   | Only available when browsing the specified domain(s) |
| `global` | Available on any page                                |

Site-scoped flows are discovered automatically when you visit the
matching domain. Global flows are always available.

### Source types

| Source        | Meaning                              |
| ------------- | ------------------------------------ |
| `recording`   | Generated from user recording        |
| `goal-driven` | Generated by AI reasoning agent      |
| `discovered`  | Generated from page action discovery |
| `manual`      | Hand-written by a developer          |

---

## The WebFlow Browser API

WebFlow scripts execute against a sandboxed API. The script receives two
arguments: `browser` (the API) and `params` (the bound parameters).

### Navigation

```typescript
await browser.navigateTo(url);
await browser.goBack();
await browser.getCurrentUrl(); // Returns current page URL
await browser.waitForNavigation(); // Alias for awaitPageLoad
await browser.awaitPageLoad(); // Wait for page to finish loading
await browser.awaitPageInteraction(); // Wait for user to interact
```

### Interaction

```typescript
await browser.click(selector); // Click element by CSS selector
await browser.clickAndWait(selector); // Click + wait for page load
await browser.followLink(text); // Click link matching text
await browser.enterText(selector, text); // Type into input field
await browser.clearAndType(selector, text); // Clear field, then type
await browser.enterTextOnPage(text); // Type on page (no specific field)
await browser.pressKey("Enter"); // Press key (Enter only currently)
await browser.selectOption(selector, value); // Set dropdown value
```

### Content

```typescript
const text = await browser.getPageText(); // Get visible text
const screenshot = await browser.captureScreenshot(); // Base64 PNG
const answer = await browser.queryContent(question); // Ask LLM about page
const state = await browser.checkPageState(question); // Check page condition
const data = await browser.extractComponent(schema); // Extract typed data
```

### Important constraints

- **Sandboxed**: Scripts cannot access `window`, `document`, `fetch`,
  `eval`, `require`, or Node.js APIs
- **Frozen API**: The `browser` and `params` objects are frozen — scripts
  cannot modify them
- **Timeout**: 180-second global timeout — scripts that run longer are
  terminated
- **Scope enforcement**: `navigateTo()` checks that the target URL
  matches the flow's domain scope
- **`pressKey()` limitation**: Currently only supports "Enter" — other
  keys are silently ignored

---

## Managing WebFlows

### List flows

**Chat panel / NL:**

```
list my webflows
list webflows for this site
list all webflows
```

**CLI:**

```
@browser webflows list
```

### Delete a flow

```
delete webflow "search-product"
```

### Change scope

```
make webflow "search-product" global
make webflow "search-product" site-only
```

---

## Ad-hoc Script Execution

You can run inline WebFlow scripts without recording:

```
@browser execute script "await browser.click('#search-btn'); await browser.enterText('#query', 'test');"
```

This uses the `ExecuteAdHocScript` action with the same sandboxed API.

---

## Goal-Driven Tasks

Instead of recording, you can describe what you want and let the AI
figure out the steps:

```
start goal-driven task: "Find the cheapest flight from Seattle to NYC next Friday"
```

The AI reasoning agent will:

1. Analyze the current page
2. Plan a sequence of browser actions
3. Execute them step by step
4. Generate a WebFlow from the successful trace

---

## Tips for Reliable WebFlows

1. **Use clear, stable selectors** — When the recording captures a click,
   it records the CSS selector. If the site uses dynamic class names (e.g.,
   `class="css-1a2b3c"`), the WebFlow may break on the next visit. Prefer
   sites with stable IDs or data attributes.

2. **Keep flows short** — A 3-step flow is more reliable than a 20-step
   flow. Break complex workflows into smaller, composable flows.

3. **Add waits for dynamic content** — If the page loads content
   asynchronously, use `awaitPageLoad()` or `awaitPageInteraction()` between
   steps.

4. **Test on fresh pages** — After creating a flow, navigate away and back
   to test it from a clean state. Recording artifacts (DOM IDs, element
   order) may differ on a fresh load.

5. **Use `queryContent()` for dynamic pages** — If you need to check whether
   an action succeeded, use the semantic query API rather than checking
   specific element state.

6. **Site-scope by default** — Unless a flow genuinely works across sites,
   keep it site-scoped. This prevents false grammar matches on unrelated
   pages.

---

## Storage Location

WebFlows are stored on disk at:

```
~/.typeagent/browser/         (or configured instance directory)
├── registry/
│   └── webflow-index.json    # Central index of all flows
├── flows/
│   ├── global/               # Global scope flows
│   │   └── {name}.json
│   └── sites/
│       └── {domain}/         # Site-scoped flows
│           └── {name}.json
└── scripts/
    └── {name}.js             # Script source code
```

The index file is loaded on agent startup. Flow metadata and scripts are
loaded on demand.
