agent-browser

magentosh/superpowers/agent-browser

Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video

安装量 · 257查看来源

Installation

npx skills add https://github.com/magentosh/superpowers --skill agent-browser

技能文件

SKILL.md

最近同步 · 2026年8月29日

references/authentication.md›

# Authentication Patterns

Login flows, OAuth, 2FA, and authenticated browsing.

**Related**: [session-management.md](session-management.md) for session details, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Login Flow](#basic-login-flow)
- [OAuth / SSO Flows](#oauth--sso-flows)
- [Two-Factor Authentication](#two-factor-authentication)
- [Session Reuse Patterns](#session-reuse-patterns)
- [Cookie Extraction](#cookie-extraction)
- [Security Best Practices](#security-best-practices)

## Basic Login Flow

Standard username/password login:

```bash
#!/bin/bash

# Start session
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/login"
}' | jq -r '.session_id')

# Get form elements
# Expected: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Sign In"

# Fill credentials
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e2", "text": "'"$PASSWORD"'"
}'

# Submit
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e3"
}'

# Wait for redirect
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 2000
}'

# Verify login succeeded
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
URL=$(echo $RESULT | jq -r '.url')

if [[ "$URL" == *"/login"* ]]; then
  echo "Login failed - still on login page"
  exit 1
fi

echo "Login successful"
# Continue with authenticated actions...
```

## OAuth / SSO Flows

For OAuth redirects (Google, GitHub, etc.):

```bash
#!/bin/bash

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/auth/google"
}' | jq -r '.session_id')

# Wait for redirect to Google
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 3000
}'

# Snapshot to see Google login form
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
echo $RESULT | jq '.elements_text'

# Fill Google email
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'

# Click Next
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e2"
}'

# Wait and snapshot for password field
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 2000
}'
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')

# Fill password
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "'"$GOOGLE_PASSWORD"'"
}'

# Click Sign in
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e2"
}'

# Wait for redirect back to app
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 5000
}'

# Verify we're back on the app
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
URL=$(echo $RESULT | jq -r '.url')
echo "Final URL: $URL"
```

## Two-Factor Authentication

For 2FA, you may need human intervention or TOTP generation:

### With TOTP Code

```bash
# After password, check for 2FA prompt
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
ELEMENTS=$(echo $RESULT | jq -r '.elements_text')

if echo "$ELEMENTS" | grep -qi "verification\|2fa\|authenticator"; then
  # Generate TOTP code (requires oathtool)
  TOTP_CODE=$(oathtool --totp -b "$TOTP_SECRET")

  # Fill 2FA code
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "fill", "ref": "@e1", "text": "'"$TOTP_CODE"'"
  }'

  # Submit
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "click", "ref": "@e2"
  }'
fi
```

### With Manual Intervention

For SMS or hardware token 2FA:

```bash
# Record video so user can see the 2FA prompt
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/login",
  "record_video": true
}' | jq -r '.session_id')

# ... login flow ...

# At 2FA step, prompt user
echo "2FA code sent. Enter the code:"
read -r CODE

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "'"$CODE"'"
}'
```

## Session Reuse Patterns

Since sessions maintain cookies, you can reuse authenticated sessions:

```bash
#!/bin/bash
# login-and-work.sh

# Login once
login() {
  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://app.example.com/login"
  }' | jq -r '.session_id')

  # ... login steps ...

  echo $SESSION
}

# Do work with authenticated session
do_work() {
  local SESSION=$1

  # Navigate to protected page
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "goto", "url": "https://app.example.com/dashboard"
  }'

  # Extract data
  belt app run agent-browser --function snapshot --session $SESSION --input '{}'
}

# Main
SESSION=$(login)
do_work $SESSION

# Don't close if you want to reuse!
# belt app run agent-browser --function close --session $SESSION --input '{}'
```

## Cookie Extraction

Extract cookies for use in other tools:

```bash
# Get cookies via JavaScript
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.cookie"
}')
COOKIES=$(echo $RESULT | jq -r '.result')
echo "Cookies: $COOKIES"

# Get all cookies including httpOnly (more complete)
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "JSON.stringify(performance.getEntriesByType(\"resource\").map(r => r.name))"
}')
```

## Security Best Practices

### 1. Never Hardcode Credentials

```bash
# Good: Use environment variables
'{"action": "fill", "ref": "@e2", "text": "'"$PASSWORD"'"}'

# Bad: Hardcoded
'{"action": "fill", "ref": "@e2", "text": "mypassword123"}'
```

### 2. Use Secure Environment Variables

```bash
# Set securely
export PASSWORD=$(cat /path/to/secure/password)

# Or use a secrets manager
export PASSWORD=$(vault read -field=password secret/app)
```

### 3. Don't Log Sensitive Data

```bash
# Good: Redact sensitive info
echo "Logging in as $USERNAME"

# Bad: Logging passwords
echo "Password: $PASSWORD"  # Never do this!
```

### 4. Close Sessions After Use

```bash
# Always clean up
trap 'belt app run agent-browser --function close --session $SESSION --input "{}" 2>/dev/null' EXIT
```

### 5. Use Video Recording for Debugging Only

Video may capture sensitive information:

```bash
# Only enable when debugging
if [ "$DEBUG" = "true" ]; then
  RECORD_VIDEO="true"
else
  RECORD_VIDEO="false"
fi
```

### 6. Verify Login Success

Always confirm authentication worked:

```bash
# Check URL changed from login page
URL=$(echo $RESULT | jq -r '.url')
if [[ "$URL" == *"/login"* ]] || [[ "$URL" == *"/signin"* ]]; then
  echo "ERROR: Login failed"
  exit 1
fi

# Or check for specific element on authenticated page
ELEMENTS=$(echo $RESULT | jq -r '.elements_text')
if ! echo "$ELEMENTS" | grep -q "Logout\|Dashboard\|Welcome"; then
  echo "ERROR: Not authenticated"
  exit 1
fi
```

references/commands.md›

# Command Reference

Complete reference for all agent-browser functions. For quick start, see [SKILL.md](../SKILL.md).

## Base Command

All commands follow this pattern:

```bash
belt app run agent-browser --function <function> --session <session_id|new> --input '<json>'
```

- `--function`: Function to call (open, snapshot, interact, screenshot, execute, close)
- `--session`: Session ID from previous call, or `new` to start fresh
- `--input`: JSON input for the function

## Functions

### open

Navigate to URL and configure browser. This is the entry point for all sessions.

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "width": 1280,
  "height": 720,
  "user_agent": "Mozilla/5.0...",
  "record_video": false,
  "show_cursor": false,
  "proxy_url": null,
  "proxy_username": null,
  "proxy_password": null
}'
```

**Input Fields:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | required | URL to navigate to |
| `width` | int | 1280 | Viewport width in pixels |
| `height` | int | 720 | Viewport height in pixels |
| `user_agent` | string | null | Custom user agent string |
| `record_video` | bool | false | Record video (returned on close) |
| `show_cursor` | bool | false | Show cursor indicator in screenshots/video |
| `proxy_url` | string | null | Proxy server URL |
| `proxy_username` | string | null | Proxy auth username |
| `proxy_password` | string | null | Proxy auth password |

**Output:**

```json
{
  "session_id": "abc123",
  "url": "https://example.com",
  "title": "Example Domain",
  "elements": [...],
  "elements_text": "@e1 [a] \"More information...\" href=\"...\"\n...",
  "screenshot": "<File>"
}
```

### snapshot

Re-fetch page state with `@e` refs. Call after navigation or DOM changes.

```bash
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'
```

**Output:** Same as `open` (url, title, elements, elements_text, screenshot)

### interact

Perform actions on the page using `@e` refs.

```bash
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click",
  "ref": "@e1"
}'
```

**Input Fields:**

| Field | Type | Description |
|-------|------|-------------|
| `action` | string | Action to perform (see Actions table) |
| `ref` | string | Element ref (e.g., `@e1`) |
| `text` | string | Text for fill/type/press/select |
| `direction` | string | Scroll direction: up, down, left, right |
| `scroll_amount` | int | Scroll pixels (default 400) |
| `wait_ms` | int | Wait duration in milliseconds |
| `url` | string | URL for goto action |
| `target_ref` | string | Target ref for drag action |
| `file_paths` | array | File paths for upload action |

**Actions:**

| Action | Required Fields | Description |
|--------|-----------------|-------------|
| `click` | `ref` | Single click |
| `dblclick` | `ref` | Double click |
| `fill` | `ref`, `text` | Clear input and type text |
| `type` | `text` | Type text without clearing |
| `press` | `text` | Press key (Enter, Tab, Escape, etc.) |
| `select` | `ref`, `text` | Select dropdown option by label |
| `hover` | `ref` | Hover over element |
| `check` | `ref` | Check checkbox |
| `uncheck` | `ref` | Uncheck checkbox |
| `drag` | `ref`, `target_ref` | Drag from ref to target_ref |
| `upload` | `ref`, `file_paths` | Upload files to file input |
| `scroll` | `direction` | Scroll page (optional: `scroll_amount`) |
| `back` | - | Go back in browser history |
| `wait` | `wait_ms` | Wait for specified milliseconds |
| `goto` | `url` | Navigate to different URL |

**Output:**

```json
{
  "success": true,
  "action": "click",
  "message": null,
  "screenshot": "<File>",
  "snapshot": {
    "url": "...",
    "title": "...",
    "elements": [...],
    "elements_text": "..."
  }
}
```

### screenshot

Take a screenshot of the current page.

```bash
belt app run agent-browser --function screenshot --session $SESSION_ID --input '{
  "full_page": true
}'
```

**Input Fields:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `full_page` | bool | false | Capture full scrollable page |

**Output:**

```json
{
  "screenshot": "<File>",
  "width": 1280,
  "height": 720
}
```

### execute

Run JavaScript code on the page.

```bash
belt app run agent-browser --function execute --session $SESSION_ID --input '{
  "code": "document.title"
}'
```

**Input Fields:**

| Field | Type | Description |
|-------|------|-------------|
| `code` | string | JavaScript code to execute |

**Output:**

```json
{
  "result": "Example Domain",
  "error": null,
  "screenshot": "<File>"
}
```

**Examples:**

```bash
# Get page title
'{"code": "document.title"}'

# Count elements
'{"code": "document.querySelectorAll(\"a\").length"}'

# Extract text
'{"code": "document.querySelector(\"h1\").textContent"}'

# Get all links
'{"code": "Array.from(document.querySelectorAll(\"a\")).map(a => a.href)"}'

# Scroll to bottom
'{"code": "window.scrollTo(0, document.body.scrollHeight)"}'

# Get computed style
'{"code": "getComputedStyle(document.body).backgroundColor"}'
```

### close

Close the browser session. Returns video if recording was enabled.

```bash
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

**Output:**

```json
{
  "success": true,
  "video": "<File or null>"
}
```

## Key Combinations

For the `press` action, use these key names:

| Key | Name |
|-----|------|
| Enter | `Enter` |
| Tab | `Tab` |
| Escape | `Escape` |
| Backspace | `Backspace` |
| Delete | `Delete` |
| Arrow keys | `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight` |
| Modifiers | `Control`, `Shift`, `Alt`, `Meta` |

**Key combinations:**

```bash
# Ctrl+A (select all)
'{"action": "press", "text": "Control+a"}'

# Ctrl+C (copy)
'{"action": "press", "text": "Control+c"}'

# Shift+Tab (focus previous)
'{"action": "press", "text": "Shift+Tab"}'
```

## Error Handling

When an action fails, `success` is `false` and `message` contains the error:

```json
{
  "success": false,
  "action": "click",
  "message": "Unknown ref: @e99. Run 'snapshot' to get current elements.",
  "screenshot": "<File>",
  "snapshot": {...}
}
```

Common errors:
- `Unknown ref: @eN` - Ref doesn't exist, re-snapshot needed
- `'text' required for fill action` - Missing required field
- `'target_ref' required for drag action` - Missing drag target
- `Timeout 5000ms exceeded` - Element not found or not clickable

references/proxy-support.md›

# Proxy Support

Proxy configuration for geo-testing, privacy, and corporate environments.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Proxy Configuration](#basic-proxy-configuration)
- [Authenticated Proxy](#authenticated-proxy)
- [Common Use Cases](#common-use-cases)
- [Proxy Types](#proxy-types)
- [Verifying Proxy Connection](#verifying-proxy-connection)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)

## Basic Proxy Configuration

Set proxy when opening a session:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080"
}' | jq -r '.session_id')
```

All traffic for this session routes through the proxy.

## Authenticated Proxy

For proxies requiring username/password:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "myuser",
  "proxy_password": "mypassword"
}' | jq -r '.session_id')
```

## Common Use Cases

### Geo-Location Testing

Test how your site appears from different regions:

```bash
#!/bin/bash
# Test from multiple regions

PROXIES=(
  "us|http://us-proxy.example.com:8080"
  "eu|http://eu-proxy.example.com:8080"
  "asia|http://asia-proxy.example.com:8080"
)

for entry in "${PROXIES[@]}"; do
  REGION="${entry%%|*}"
  PROXY="${entry##*|}"

  echo "Testing from: $REGION"

  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://mysite.com",
    "proxy_url": "'"$PROXY"'"
  }' | jq -r '.session_id')

  # Take screenshot
  belt app run agent-browser --function screenshot --session $SESSION --input '{
    "full_page": true
  }' > "${REGION}-screenshot.json"

  # Get page content
  RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
  echo $RESULT | jq '.elements_text' > "${REGION}-elements.txt"

  belt app run agent-browser --function close --session $SESSION --input '{}'
done

echo "Geo-testing complete"
```

### Rate Limit Avoidance

Rotate proxies for web scraping:

```bash
#!/bin/bash
# Rotate through proxy list

PROXIES=(
  "http://proxy1.example.com:8080"
  "http://proxy2.example.com:8080"
  "http://proxy3.example.com:8080"
)

URLS=(
  "https://site.com/page1"
  "https://site.com/page2"
  "https://site.com/page3"
)

for i in "${!URLS[@]}"; do
  # Rotate proxy
  PROXY_INDEX=$((i % ${#PROXIES[@]}))
  PROXY="${PROXIES[$PROXY_INDEX]}"
  URL="${URLS[$i]}"

  echo "Fetching $URL via proxy $((PROXY_INDEX + 1))"

  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "'"$URL"'",
    "proxy_url": "'"$PROXY"'"
  }' | jq -r '.session_id')

  # Extract data
  RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
    "code": "document.body.innerText"
  }')
  echo $RESULT | jq -r '.result' > "page-$i.txt"

  belt app run agent-browser --function close --session $SESSION --input '{}'

  # Polite delay
  sleep 1
done
```

### Corporate Network Access

Access sites through corporate proxy:

```bash
# Use corporate proxy for external sites
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://external-vendor.com",
  "proxy_url": "http://corpproxy.company.com:8080",
  "proxy_username": "'"$CORP_USER"'",
  "proxy_password": "'"$CORP_PASS"'"
}' | jq -r '.session_id')
```

### Privacy and Anonymity

Route through privacy-focused proxy:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://whatismyip.com",
  "proxy_url": "socks5://privacy-proxy.example.com:1080"
}' | jq -r '.session_id')
```

## Proxy Types

### HTTP/HTTPS Proxy

```json
{"proxy_url": "http://proxy.example.com:8080"}
{"proxy_url": "https://proxy.example.com:8080"}
```

### SOCKS5 Proxy

```json
{"proxy_url": "socks5://proxy.example.com:1080"}
```

### With Authentication

```json
{
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}
```

## Verifying Proxy Connection

Check that traffic routes through proxy:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://httpbin.org/ip",
  "proxy_url": "http://proxy.example.com:8080"
}' | jq -r '.session_id')

# Get the IP shown
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.body.innerText"
}')
echo "IP via proxy: $(echo $RESULT | jq -r '.result')"

belt app run agent-browser --function close --session $SESSION --input '{}'
```

The IP should be the proxy's IP, not your real IP.

## Troubleshooting

### Connection Failed

```
Error: Failed to open URL: net::ERR_PROXY_CONNECTION_FAILED
```

**Solutions:**
1. Verify proxy URL is correct
2. Check proxy is running and accessible
3. Confirm port is correct
4. Test proxy with curl: `curl -x http://proxy:8080 https://example.com`

### Authentication Failed

```
Error: 407 Proxy Authentication Required
```

**Solutions:**
1. Verify username/password are correct
2. Check if proxy requires different auth method
3. Ensure credentials don't contain special characters that need escaping

### SSL Errors

Some proxies perform SSL inspection. If you see certificate errors:

```bash
# The browser should handle most SSL proxies automatically
# If issues persist, verify proxy SSL certificate is valid
```

### Slow Performance

**Solutions:**
1. Choose proxy closer to target site
2. Use faster proxy provider
3. Reduce number of requests per session

## Best Practices

### 1. Use Environment Variables

```bash
# Good: Credentials in env vars
'{"proxy_url": "'"$PROXY_URL"'", "proxy_username": "'"$PROXY_USER"'"}'

# Bad: Hardcoded
'{"proxy_url": "http://user:[email protected]:8080"}'
```

### 2. Test Proxy Before Automation

```bash
# Verify proxy works
curl -x "$PROXY_URL" https://httpbin.org/ip
```

### 3. Handle Proxy Failures

```bash
# Retry with different proxy on failure
for PROXY in "${PROXIES[@]}"; do
  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "'"$URL"'",
    "proxy_url": "'"$PROXY"'"
  }' 2>&1)

  if echo "$SESSION" | jq -e '.session_id' > /dev/null 2>&1; then
    SESSION_ID=$(echo $SESSION | jq -r '.session_id')
    break
  fi
  echo "Proxy $PROXY failed, trying next..."
done
```

### 4. Respect Rate Limits

Even with proxies, be a good citizen:

```bash
# Add delays between requests
'{"action": "wait", "wait_ms": 1000}'
```

### 5. Log Proxy Usage

For debugging, log which proxy was used:

```bash
echo "$(date): Using proxy $PROXY for $URL" >> proxy.log
```

references/session-management.md›

# Session Management

Browser sessions for state persistence and parallel browsing.

**Related**: [authentication.md](authentication.md) for login patterns, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [How Sessions Work](#how-sessions-work)
- [Starting a Session](#starting-a-session)
- [Using Session IDs](#using-session-ids)
- [Session State](#session-state)
- [Parallel Sessions](#parallel-sessions)
- [Session Cleanup](#session-cleanup)
- [Best Practices](#best-practices)

## How Sessions Work

Each session maintains an isolated browser context with:
- Cookies
- LocalStorage / SessionStorage
- Browser history
- Page state
- Video recording (if enabled)

Sessions persist across function calls, allowing multi-step workflows.

## Starting a Session

Use `--session new` to create a fresh session:

```bash
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
echo "Session: $SESSION_ID"
```

## Using Session IDs

All subsequent calls use the session ID:

```bash
# Navigate
belt app run agent-browser --function open --session $SESSION_ID --input '{
  "url": "https://example.com/page2"
}'

# Interact
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e1"
}'

# Screenshot
belt app run agent-browser --function screenshot --session $SESSION_ID --input '{}'

# Close
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

## Session State

### What Persists

Within a session, these persist across calls:
- Cookies (login state, preferences)
- LocalStorage and SessionStorage
- IndexedDB data
- Browser history (for back/forward)
- Current page and DOM state
- Video recording buffer

### What Doesn't Persist

- Sessions don't persist across server restarts
- No automatic session recovery
- Video is only available until close is called

## Parallel Sessions

Run multiple independent sessions simultaneously:

```bash
#!/bin/bash
# Scrape multiple sites in parallel

# Start sessions
RESULT1=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://site1.com"
}')
SESSION1=$(echo $RESULT1 | jq -r '.session_id')

RESULT2=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://site2.com"
}')
SESSION2=$(echo $RESULT2 | jq -r '.session_id')

# Work with each session independently
belt app run agent-browser --function screenshot --session $SESSION1 --input '{}' &
belt app run agent-browser --function screenshot --session $SESSION2 --input '{}' &
wait

# Clean up both
belt app run agent-browser --function close --session $SESSION1 --input '{}'
belt app run agent-browser --function close --session $SESSION2 --input '{}'
```

### Use Cases for Parallel Sessions

1. **A/B Testing** - Compare different pages or user experiences
2. **Multi-site scraping** - Gather data from multiple sources
3. **Load testing** - Simulate multiple users
4. **Cross-region testing** - Use different proxies per session

## Session Cleanup

Always close sessions when done:

```bash
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

**Why close matters:**
- Releases server resources
- Returns video recording (if enabled)
- Prevents resource leaks

### Error Handling

```bash
#!/bin/bash
set -e

cleanup() {
  belt app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
}
trap cleanup EXIT

SESSION_ID=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com"
}' | jq -r '.session_id')

# ... your automation ...
# cleanup runs automatically on exit
```

## Best Practices

### 1. Store Session IDs

```bash
# Good: Store for reuse
SESSION_ID=$(... | jq -r '.session_id')
belt ... --session $SESSION_ID ...

# Bad: Parse every time
belt ... --session $(... | jq -r '.session_id') ...
```

### 2. Close Sessions Promptly

Don't leave sessions open longer than needed. Server resources are limited.

### 3. Use Meaningful Variable Names

```bash
# Good: Clear purpose
LOGIN_SESSION=$(...)
SCRAPE_SESSION=$(...)

# Bad: Generic names
S1=$(...)
S2=$(...)
```

### 4. Handle Session Expiry

Sessions may expire after extended inactivity:

```bash
# Check if session is still valid
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}' 2>&1)
if echo "$RESULT" | grep -q "session not found"; then
  echo "Session expired, starting new one"
  SESSION_ID=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://example.com"
  }' | jq -r '.session_id')
fi
```

### 5. One Task Per Session

For clarity, use one session per logical task:

```bash
# Good: Separate sessions for separate tasks
LOGIN_SESSION=$(...)  # Handle login
SCRAPE_SESSION=$(...)  # Handle scraping

# Okay for related tasks: One session for a workflow
SESSION=$(...)
# login -> navigate -> extract -> close
```

references/snapshot-refs.md›

# Snapshot and Refs

Compact element references that reduce context usage for AI agents.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [How Refs Work](#how-refs-work)
- [Snapshot Output Format](#snapshot-output-format)
- [Using Refs](#using-refs)
- [Ref Lifecycle](#ref-lifecycle)
- [Best Practices](#best-practices)
- [Ref Notation Details](#ref-notation-details)
- [Troubleshooting](#troubleshooting)

## How Refs Work

Traditional approach:
```
Full DOM/HTML -> AI parses -> CSS selector -> Action (~3000-5000 tokens)
```

agent-browser approach:
```
Compact snapshot -> @refs assigned -> Direct interaction (~200-400 tokens)
```

The snapshot extracts interactive elements and assigns short `@e` refs, reducing token usage significantly.

## Snapshot Output Format

```bash
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
```

**Response `elements_text`:**

```
@e1 [a] "Home" href="/"
@e2 [a] "Products" href="/products"
@e3 [a] "About" href="/about"
@e4 [button] "Sign In"
@e5 [input type="email"] placeholder="Email"
@e6 [input type="password"] placeholder="Password"
@e7 [button type="submit"] "Log In"
@e8 [input type="checkbox"] name="remember"
```

**Response `elements` (structured):**

```json
[
  {
    "ref": "@e1",
    "desc": "@e1 [a] \"Home\" href=\"/\"",
    "tag": "a",
    "text": "Home",
    "role": null,
    "name": null,
    "href": "/",
    "input_type": null
  },
  ...
]
```

## Using Refs

Once you have refs, interact directly:

```bash
# Click the "Sign In" button
'{"action": "click", "ref": "@e4"}'

# Fill email input
'{"action": "fill", "ref": "@e5", "text": "[email protected]"}'

# Fill password
'{"action": "fill", "ref": "@e6", "text": "password123"}'

# Submit the form
'{"action": "click", "ref": "@e7"}'

# Check the "remember me" checkbox
'{"action": "check", "ref": "@e8"}'
```

## Ref Lifecycle

**IMPORTANT**: Refs are invalidated when the page changes!

```bash
# Get initial snapshot
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# @e1 [button] "Next"

# Click triggers page change
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}'

# MUST re-snapshot to get new refs!
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# @e1 [h1] "Page 2"  <- Different element now!
```

### When to Re-snapshot

Always re-snapshot after:

1. **Navigation** - Clicking links, form submissions, `goto` action
2. **Dynamic content** - AJAX loads, modals opening, tabs switching
3. **Page mutations** - JavaScript modifying the DOM

The `interact` function returns a fresh snapshot in its response, so you can often use that instead of a separate snapshot call.

## Best Practices

### 1. Always Use the Latest Snapshot

```bash
# CORRECT: Use snapshot from previous response
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}')
# Use elements from $RESULT.snapshot for next action

# WRONG: Using stale refs
# After navigation, @e1 may point to a completely different element
```

### 2. Check Success Before Continuing

```bash
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e5"
}')

SUCCESS=$(echo $RESULT | jq -r '.success')
if [ "$SUCCESS" != "true" ]; then
  echo "Click failed: $(echo $RESULT | jq -r '.message')"
  # Re-snapshot and retry
fi
```

### 3. Use elements_text for Quick Decisions

For AI agents, `elements_text` provides a compact text representation:

```
@e1 [input type="email"] placeholder="Email"
@e2 [input type="password"] placeholder="Password"
@e3 [button] "Submit"
```

This is often enough to decide which element to interact with without parsing the full `elements` array.

## Ref Notation Details

```
@e1 [tag type="value"] "text content" name="attr"
|    |   |             |              |
|    |   |             |              +- Additional attributes
|    |   |             +- Visible text
|    |   +- Key attributes shown
|    +- HTML tag name
+- Unique ref ID
```

### Common Patterns

```
@e1 [button] "Submit"                    # Button with text
@e2 [input type="email"]                 # Email input
@e3 [input type="password"]              # Password input
@e4 [a] "Link Text" href="/page"         # Anchor link
@e5 [select]                             # Dropdown
@e6 [textarea] placeholder="Message"     # Text area
@e7 [input type="file"]                  # File upload
@e8 [input type="checkbox"] checked      # Checked checkbox
@e9 [input type="radio"] selected        # Selected radio
@e10 [button type="submit"] "Send"       # Submit button
```

### Elements Captured

The snapshot captures these interactive elements:

- Links (`<a href>`)
- Buttons (`<button>`, `[role="button"]`)
- Inputs (`<input>`, `<textarea>`, `<select>`)
- Clickable elements (`[onclick]`, `[tabindex]`)
- ARIA roles (`[role="link"]`, `[role="checkbox"]`, etc.)

Non-interactive or hidden elements are filtered out.

## Troubleshooting

### "Unknown ref" Error

```json
{
  "success": false,
  "message": "Unknown ref: @e15. Run 'snapshot' to get current elements."
}
```

**Solution**: Re-snapshot. The page changed and refs are stale.

```bash
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# Now use the new refs
```

### Element Not in Snapshot

The element you need might not appear because:

1. **Not visible** - Scroll to reveal it
   ```bash
   '{"action": "scroll", "direction": "down", "scroll_amount": 500}'
   ```

2. **Not interactive** - Use JavaScript to interact
   ```bash
   '{"code": "document.querySelector(\".hidden-btn\").click()"}'
   ```

3. **In iframe** - Currently not supported (use `execute` with JS)

4. **Dynamic** - Wait for it to load
   ```bash
   '{"action": "wait", "wait_ms": 2000}'
   ```

### Too Many Elements

Snapshots are limited to 50 elements. If the page has more:

1. **Scroll** to bring relevant elements into view
2. **Use JavaScript** to target specific elements
3. **Navigate** to a more specific page

### Ref Points to Wrong Element

If a ref seems to interact with the wrong element:

1. Re-snapshot to get fresh refs
2. Check if the page structure changed
3. Verify with screenshot that the right element is targeted

references/video-recording.md›

# Video Recording

Capture browser automation as video for debugging, documentation, or verification.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Recording](#basic-recording)
- [Cursor Indicator](#cursor-indicator)
- [How Recording Works](#how-recording-works)
- [Use Cases](#use-cases)
- [Best Practices](#best-practices)
- [Output Format](#output-format)
- [Limitations](#limitations)

## Basic Recording

Enable video recording when opening a session:

```bash
# Start with recording enabled
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Perform actions
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}'

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e2", "text": "test input"
}'

# Close to get the video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $RESULT | jq -r '.video')
echo "Video file: $VIDEO"
```

## Cursor Indicator

For demos and documentation, show a visible cursor that follows mouse movements:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')
```

The cursor appears as a red dot that:
- Follows mouse movements in real-time
- Shows click feedback (shrinks on mousedown)
- Persists across page navigations
- Appears in both screenshots and video

This is especially useful for:
- Tutorial/documentation videos
- Debugging interaction issues
- Sharing recordings with non-technical stakeholders

## How Recording Works

1. **Start**: Pass `"record_video": true` in the `open` function
2. **Record**: All browser activity is captured throughout the session
3. **Stop**: Video is finalized when `close` is called
4. **Retrieve**: Video file is returned in the `close` response

The video captures:
- Page loads and navigations
- Element interactions (clicks, typing)
- Scrolling and animations
- Dynamic content changes

## Use Cases

### Debugging Failed Automation

```bash
#!/bin/bash
# Record automation for debugging

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com",
  "record_video": true
}' | jq -r '.session_id')

# Run automation
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}')

SUCCESS=$(echo $RESULT | jq -r '.success')
if [ "$SUCCESS" != "true" ]; then
  echo "Action failed!"
  echo "Message: $(echo $RESULT | jq -r '.message')"

  # Get video for debugging
  CLOSE_RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
  echo "Debug video: $(echo $CLOSE_RESULT | jq -r '.video')"
  exit 1
fi

belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Documentation Generation

Record workflows for user documentation:

```bash
#!/bin/bash
# Record how-to video

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/settings",
  "record_video": true,
  "width": 1920,
  "height": 1080
}' | jq -r '.session_id')

# Add pauses for clarity
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 1000
}'

# Step 1: Click settings
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e5"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 500
}'

# Step 2: Change setting
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e10"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 500
}'

# Step 3: Save
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e15"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 1000
}'

# Get the video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo "Documentation video: $(echo $RESULT | jq -r '.video')"
```

### Test Evidence for CI/CD

```bash
#!/bin/bash
# Record E2E test for CI artifacts

TEST_NAME="${1:-e2e-test}"

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "'"$TEST_URL"'",
  "record_video": true
}' | jq -r '.session_id')

# Run test steps
run_test_steps $SESSION
TEST_RESULT=$?

# Always get video
CLOSE_RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $CLOSE_RESULT | jq -r '.video')

# Save to artifacts
if [ -n "$CI_ARTIFACTS_DIR" ]; then
  cp "$VIDEO" "$CI_ARTIFACTS_DIR/${TEST_NAME}.webm"
fi

exit $TEST_RESULT
```

### Monitoring and Auditing

```bash
#!/bin/bash
# Record automated task for audit trail

TASK_ID=$(date +%Y%m%d-%H%M%S)

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://admin.example.com",
  "record_video": true
}' | jq -r '.session_id')

# Perform admin task
# ... automation steps ...

# Save recording
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $RESULT | jq -r '.video')

# Archive for audit
mv "$VIDEO" "/audit/recordings/${TASK_ID}.webm"
echo "Audit recording saved: ${TASK_ID}.webm"
```

## Best Practices

### 1. Add Strategic Pauses

Pauses make videos easier to follow:

```bash
# After significant actions, add a pause
'{"action": "click", "ref": "@e1"}'
'{"action": "wait", "wait_ms": 500}'  # Let viewer see result
```

### 2. Use Larger Viewport for Documentation

```bash
'{"url": "...", "record_video": true, "width": 1920, "height": 1080}'
```

### 3. Handle Errors Gracefully

Always retrieve video even on failure:

```bash
cleanup() {
  if [ -n "$SESSION" ]; then
    belt app run agent-browser --function close --session $SESSION --input '{}' 2>/dev/null
  fi
}
trap cleanup EXIT
```

### 4. Combine with Screenshots

Use screenshots for key frames, video for flow:

```bash
# Record overall flow
'{"record_video": true}'

# Capture key states
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'
```

### 5. Don't Record Sensitive Sessions

Avoid recording when handling credentials:

```bash
if [ "$CONTAINS_SENSITIVE_DATA" = "true" ]; then
  RECORD="false"
else
  RECORD="true"
fi

'{"url": "...", "record_video": '$RECORD'}'
```

## Output Format

- **Format**: WebM (VP8/VP9 codec)
- **Compatibility**: All modern browsers and video players
- **Quality**: Matches viewport size
- **Compression**: Efficient for screen content

## Limitations

1. **Session-level only** - Can't start/stop mid-session
2. **Memory usage** - Long sessions consume more memory
3. **File size** - Complex pages with animations produce larger files
4. **No audio** - Browser audio is not captured
5. **Returned on close** - Video only available after session ends

SKILL.md›

---
name: agent-browser
description: "Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video"
allowed-tools: Bash(belt *)
---

> **Install the belt CLI skill:** `npx skills add belt-sh/cli`

# Agentic Browser

Browser automation for AI agents via [inference.sh](https://inference.sh). Uses Playwright under the hood with a simple `@e` ref system for element interaction.

![Agentic Browser](https://cloud.inference.sh/app/files/u/4mg21r6ta37mpaz6ktzwtt8krr/01kgjw8atdxgkrsr8a2t5peq7b.jpeg)

## Quick Start

> Requires inference.sh CLI (`belt`). [Install instructions](https://raw.githubusercontent.com/inference-sh/skills/refs/heads/main/cli-install.md)

```bash
belt login

# Open a page and get interactive elements
belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new
```


## Core Workflow

Every browser automation follows this pattern:

1. **Open** - Navigate to URL, get `@e` refs for elements
2. **Interact** - Use refs to click, fill, drag, etc.
3. **Re-snapshot** - After navigation/changes, get fresh refs
4. **Close** - End session (returns video if recording)

```bash
# 1. Start session
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# 3. Re-snapshot after navigation
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

## Functions

| Function | Description |
|----------|-------------|
| `open` | Navigate to URL, configure browser (viewport, proxy, video recording) |
| `snapshot` | Re-fetch page state with `@e` refs after DOM changes |
| `interact` | Perform actions using `@e` refs (click, fill, drag, upload, etc.) |
| `screenshot` | Take page screenshot (viewport or full page) |
| `execute` | Run JavaScript code on the page |
| `close` | Close session, returns video if recording was enabled |

## Interact Actions

| Action | Description | Required Fields |
|--------|-------------|-----------------|
| `click` | Click element | `ref` |
| `dblclick` | Double-click element | `ref` |
| `fill` | Clear and type text | `ref`, `text` |
| `type` | Type text (no clear) | `text` |
| `press` | Press key (Enter, Tab, etc.) | `text` |
| `select` | Select dropdown option | `ref`, `text` |
| `hover` | Hover over element | `ref` |
| `check` | Check checkbox | `ref` |
| `uncheck` | Uncheck checkbox | `ref` |
| `drag` | Drag and drop | `ref`, `target_ref` |
| `upload` | Upload file(s) | `ref`, `file_paths` |
| `scroll` | Scroll page | `direction` (up/down/left/right), `scroll_amount` |
| `back` | Go back in history | - |
| `wait` | Wait milliseconds | `wait_ms` |
| `goto` | Navigate to URL | `url` |

## Element Refs

Elements are returned with `@e` refs:

```
@e1 [a] "Home" href="/"
@e2 [input type="text"] placeholder="Search"
@e3 [button] "Submit"
@e4 [select] "Choose option"
@e5 [input type="checkbox"] name="agree"
```

**Important:** Refs are invalidated after navigation. Always re-snapshot after:
- Clicking links/buttons that navigate
- Form submissions
- Dynamic content loading

## Features

### Video Recording

Record browser sessions for debugging or documentation:

```bash
# Start with recording enabled (optionally show cursor indicator)
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
belt app run agent-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}
```

### Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "show_cursor": true,
  "record_video": true
}'
```

The cursor appears as a red dot that follows mouse movements and shows click feedback.

### Proxy Support

Route traffic through a proxy server:

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'
```

### File Upload

Upload files to file inputs:

```bash
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'
```

### Drag and Drop

Drag elements to targets:

```bash
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "drag",
  "ref": "@e1",
  "target_ref": "@e2"
}'
```

### JavaScript Execution

Run custom JavaScript:

```bash
belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}
```

## Deep-Dive Documentation

| Reference | Description |
|-----------|-------------|
| [references/commands.md](references/commands.md) | Full function reference with all options |
| [references/snapshot-refs.md](references/snapshot-refs.md) | Ref lifecycle, invalidation rules, troubleshooting |
| [references/session-management.md](references/session-management.md) | Session persistence, parallel sessions |
| [references/authentication.md](references/authentication.md) | Login flows, OAuth, 2FA handling |
| [references/video-recording.md](references/video-recording.md) | Recording workflows for debugging |
| [references/proxy-support.md](references/proxy-support.md) | Proxy configuration, geo-testing |

## Ready-to-Use Templates

| Template | Description |
|----------|-------------|
| [templates/form-automation.sh](templates/form-automation.sh) | Form filling with validation |
| [templates/authenticated-session.sh](templates/authenticated-session.sh) | Login once, reuse session |
| [templates/capture-workflow.sh](templates/capture-workflow.sh) | Content extraction with screenshots |

## Examples

### Form Submission

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/contact"
}' | jq -r '.session_id')

# Get elements: @e1 [input] "Name", @e2 [input] "Email", @e3 [textarea], @e4 [button] "Send"

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "John Doe"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e2", "text": "[email protected]"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e3", "text": "Hello!"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "click", "ref": "@e4"}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Search and Extract

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://google.com"
}' | jq -r '.session_id')

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "weather today"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "press", "text": "Enter"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "wait", "wait_ms": 2000}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Screenshot with Video

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Take full page screenshot
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'

# Close and get video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo $RESULT | jq '.video'
```

## Sessions

Browser state persists within a session. Always:

1. Start with `--session new` on first call
2. Use returned `session_id` for subsequent calls
3. Close session when done

## Related Skills

```bash
# Web search (for research + browse)
npx skills add inference-sh/skills@web-search

# LLM models (analyze extracted content)
npx skills add inference-sh/skills@llm-models
```

## Documentation

- [inference.sh Sessions](https://inference.sh/docs/extend/sessions) - Session management
- [Multi-function Apps](https://inference.sh/docs/extend/multi-function-apps) - How functions work

templates/authenticated-session.sh›

#!/bin/bash
# Template: Authenticated Session Workflow
# Purpose: Login once, perform actions, clean up
# Usage: ./authenticated-session.sh <login-url>
#
# Environment variables:
#   APP_USERNAME - Login username/email
#   APP_PASSWORD - Login password
#
# Two modes:
#   1. Discovery mode (default): Shows login form structure
#   2. Login mode: Performs actual login after you update refs
#
# Setup steps:
#   1. Run once to see form structure (discovery mode)
#   2. Update refs in LOGIN FLOW section below
#   3. Set APP_USERNAME and APP_PASSWORD
#   4. Comment out the DISCOVERY section

set -euo pipefail

LOGIN_URL="${1:?Usage: $0 <login-url>}"

echo "Authentication workflow: $LOGIN_URL"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    echo "Closing session..."
    infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
  fi
}
trap cleanup EXIT

# ================================================================
# DISCOVERY MODE: Shows login form structure
# Delete this section after setup
# ================================================================
echo "Opening login page..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$LOGIN_URL"'"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

echo ""
echo "Login form structure:"
echo "---"
echo $RESULT | jq -r '.elements_text'
echo "---"
echo ""
echo "Discovery mode complete."
echo ""
echo "Next steps:"
echo "  1. Identify the refs: username=@e?, password=@e?, submit=@e?"
echo "  2. Update the LOGIN FLOW section below with your refs"
echo "  3. Set environment variables:"
echo "     export APP_USERNAME='your-username'"
echo "     export APP_PASSWORD='your-password'"
echo "  4. Comment out this DISCOVERY MODE section"
echo ""
exit 0

# ================================================================
# LOGIN FLOW: Uncomment and customize after discovery
# ================================================================
# : "${APP_USERNAME:?Set APP_USERNAME environment variable}"
# : "${APP_PASSWORD:?Set APP_PASSWORD environment variable}"
#
# echo "Opening login page..."
# RESULT=$(infsh app run agent-browser --function open --session new --input '{
#   "url": "'"$LOGIN_URL"'",
#   "record_video": false
# }')
# SESSION_ID=$(echo $RESULT | jq -r '.session_id')
#
# echo "Filling credentials..."
# # Update @e1, @e2, @e3 to match your form
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e1", "text": "'"$APP_USERNAME"'"
# }'
#
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e2", "text": "'"$APP_PASSWORD"'"
# }'
#
# echo "Submitting..."
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "click", "ref": "@e3"
# }'
#
# # Wait for redirect
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "wait", "wait_ms": 3000
# }'
#
# # Verify login succeeded
# RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
# URL=$(echo $RESULT | jq -r '.url')
#
# if [[ "$URL" == *"/login"* ]] || [[ "$URL" == *"/signin"* ]]; then
#   echo "ERROR: Login failed - still on login page"
#   echo "URL: $URL"
#   infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{}' > login-failed.json
#   exit 1
# fi
#
# echo "Login successful!"
# echo "Current URL: $URL"
# echo ""
#
# # ================================================================
# # AUTHENTICATED ACTIONS: Add your post-login automation here
# # ================================================================
# echo "Performing authenticated actions..."
#
# # Example: Navigate to dashboard
# # infsh app run agent-browser --function interact --session $SESSION_ID --input '{
# #   "action": "goto", "url": "https://app.example.com/dashboard"
# # }'
#
# # Example: Click a menu item
# # infsh app run agent-browser --function interact --session $SESSION_ID --input '{
# #   "action": "click", "ref": "@e5"
# # }'
#
# # Example: Extract data
# # RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
# #   "code": "document.querySelector(\".user-data\").textContent"
# # }')
# # echo "Data: $(echo $RESULT | jq -r '.result')"
#
# # Example: Take screenshot of authenticated page
# # infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
# #   "full_page": true
# # }' > authenticated-page.json
#
# echo ""
# echo "Authenticated session complete"

templates/capture-workflow.sh›

#!/bin/bash
# Template: Content Capture Workflow
# Purpose: Extract content from web pages (text, screenshots, video)
# Usage: ./capture-workflow.sh <url> [output-dir]
#
# Outputs:
#   - page-screenshot.json: Page screenshot data
#   - page-full-screenshot.json: Full page screenshot data
#   - page-elements.txt: Interactive elements with refs
#   - page-text.txt: All text content
#   - page-links.txt: All links on the page
#   - session-video.json: Video recording (if enabled)

set -euo pipefail

TARGET_URL="${1:?Usage: $0 <url> [output-dir]}"
OUTPUT_DIR="${2:-.}"

echo "Content capture: $TARGET_URL"
echo "Output directory: $OUTPUT_DIR"

mkdir -p "$OUTPUT_DIR"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    echo "Closing session..."
    CLOSE_RESULT=$(infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || echo '{}')

    # Save video if available
    VIDEO=$(echo $CLOSE_RESULT | jq -r '.video // empty')
    if [ -n "$VIDEO" ]; then
      echo "$CLOSE_RESULT" > "$OUTPUT_DIR/session-video.json"
      echo "Video saved to: $OUTPUT_DIR/session-video.json"
    fi
  fi
}
trap cleanup EXIT

# ================================================================
# CONFIGURATION
# ================================================================
RECORD_VIDEO=false  # Set to true to record video
FULL_PAGE=true      # Set to true for full page screenshots
EXTRACT_LINKS=true  # Set to true to extract all links
SCROLL_PAGES=0      # Number of scroll actions for infinite scroll pages

# ================================================================
# CAPTURE WORKFLOW
# ================================================================

# Start session
echo "Opening page..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$TARGET_URL"'",
  "record_video": '$RECORD_VIDEO',
  "width": 1920,
  "height": 1080
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

# Get metadata
URL=$(echo $RESULT | jq -r '.url')
TITLE=$(echo $RESULT | jq -r '.title')
echo "Title: $TITLE"
echo "URL: $URL"

# Save elements
echo $RESULT | jq -r '.elements_text' > "$OUTPUT_DIR/page-elements.txt"
echo "Elements saved to: $OUTPUT_DIR/page-elements.txt"

# Handle infinite scroll (if configured)
if [ $SCROLL_PAGES -gt 0 ]; then
  echo "Scrolling through $SCROLL_PAGES pages..."
  for ((i=1; i<=SCROLL_PAGES; i++)); do
    infsh app run agent-browser --function interact --session $SESSION_ID --input '{
      "action": "scroll", "direction": "down", "scroll_amount": 800
    }' > /dev/null
    infsh app run agent-browser --function interact --session $SESSION_ID --input '{
      "action": "wait", "wait_ms": 1000
    }' > /dev/null
    echo "  Scrolled page $i/$SCROLL_PAGES"
  done

  # Re-snapshot after scrolling
  RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
fi

# Take viewport screenshot
echo "Taking viewport screenshot..."
infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{}' > "$OUTPUT_DIR/page-screenshot.json"
echo "Screenshot saved to: $OUTPUT_DIR/page-screenshot.json"

# Take full page screenshot (if configured)
if [ "$FULL_PAGE" = true ]; then
  echo "Taking full page screenshot..."
  infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
    "full_page": true
  }' > "$OUTPUT_DIR/page-full-screenshot.json"
  echo "Full screenshot saved to: $OUTPUT_DIR/page-full-screenshot.json"
fi

# Extract all text content
echo "Extracting text content..."
RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
  "code": "document.body.innerText"
}')
echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/page-text.txt"
echo "Text saved to: $OUTPUT_DIR/page-text.txt"

# Extract all links (if configured)
if [ "$EXTRACT_LINKS" = true ]; then
  echo "Extracting links..."
  RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
    "code": "Array.from(document.querySelectorAll(\"a[href]\")).map(a => a.href + \" | \" + (a.textContent || \"\").trim().slice(0,50)).join(\"\\n\")"
  }')
  echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/page-links.txt"
  echo "Links saved to: $OUTPUT_DIR/page-links.txt"
fi

# ================================================================
# CUSTOM EXTRACTION: Add your specific extraction logic here
# ================================================================

# Example: Extract specific elements by selector
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "Array.from(document.querySelectorAll(\"h2\")).map(h => h.textContent).join(\"\\n\")"
# }')
# echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/headings.txt"

# Example: Extract JSON data from script tag
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "JSON.parse(document.querySelector(\"script[type=application/json]\").textContent)"
# }')
# echo $RESULT | jq '.result' > "$OUTPUT_DIR/json-data.json"

# Example: Extract table data
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "Array.from(document.querySelectorAll(\"table tr\")).map(tr => Array.from(tr.cells).map(td => td.textContent.trim()).join(\",\")).join(\"\\n\")"
# }')
# echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/table-data.csv"

# ================================================================
# SUMMARY
# ================================================================
echo ""
echo "Capture complete!"
echo "Files created:"
ls -la "$OUTPUT_DIR"/*.txt "$OUTPUT_DIR"/*.json 2>/dev/null || true

templates/form-automation.sh›

#!/bin/bash
# Template: Form Automation Workflow
# Purpose: Fill and submit web forms with validation
# Usage: ./form-automation.sh <form-url>
#
# This template demonstrates the snapshot-interact-verify pattern:
# 1. Navigate to form
# 2. Snapshot to get element refs
# 3. Fill fields using refs
# 4. Submit and verify result
#
# Customize: Update the refs (@e1, @e2, etc.) based on your form's snapshot output

set -euo pipefail

FORM_URL="${1:?Usage: $0 <form-url>}"

echo "Form automation: $FORM_URL"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
  fi
}
trap cleanup EXIT

# Step 1: Navigate to form
echo "Opening form..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$FORM_URL"'"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

# Step 2: Display form structure
echo ""
echo "Form elements:"
echo "---"
echo $RESULT | jq -r '.elements_text'
echo "---"
echo ""

# ================================================================
# DISCOVERY MODE: Shows form structure
# After running once, update the FORM FILL section below with your refs
# then delete or comment out this section
# ================================================================
echo "Discovery mode: Form structure shown above"
echo ""
echo "Next steps:"
echo "  1. Note the refs for your form fields (e.g., @e1 for name, @e2 for email)"
echo "  2. Update the FORM FILL section below"
echo "  3. Set environment variables for form data"
echo "  4. Comment out this discovery section"
echo ""
exit 0

# ================================================================
# FORM FILL: Uncomment and customize after discovery
# ================================================================
# echo "Filling form..."
#
# # Text input
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e1", "text": "'"${FORM_NAME:-John Doe}"'"
# }'
#
# # Email input
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e2", "text": "'"${FORM_EMAIL:[email protected]}"'"
# }'
#
# # Dropdown/select
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "select", "ref": "@e3", "text": "Option 1"
# }'
#
# # Checkbox
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "check", "ref": "@e4"
# }'
#
# # Textarea
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e5", "text": "'"${FORM_MESSAGE:-Hello, this is a test message.}"'"
# }'
#
# # Submit button
# echo "Submitting form..."
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "click", "ref": "@e6"
# }'
#
# # Wait for submission
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "wait", "wait_ms": 2000
# }'
#
# # Step 3: Verify result
# echo ""
# echo "Verifying submission..."
# RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
#
# URL=$(echo $RESULT | jq -r '.url')
# TITLE=$(echo $RESULT | jq -r '.title')
# echo "Final URL: $URL"
# echo "Page title: $TITLE"
#
# # Check for success indicators
# ELEMENTS=$(echo $RESULT | jq -r '.elements_text')
# if echo "$ELEMENTS" | grep -qi "thank you\|success\|submitted"; then
#   echo "SUCCESS: Form submitted successfully"
# elif echo "$URL" | grep -qi "error\|fail"; then
#   echo "ERROR: Form submission may have failed"
#   exit 1
# else
#   echo "UNKNOWN: Check the result manually"
# fi
#
# # Optional: Capture evidence
# infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
#   "full_page": true
# }' > form-result-screenshot.json
# echo "Screenshot saved to form-result-screenshot.json"

echo "Done"