Retour aux skills
skills-101/superpowersContrôle réussi

SKILL DETAIL

agent-browser

skills-101/superpowers/agent-browser

Agent Browser is a browser automation tool for AI agents, powered by inference.sh. It uses Playwright under the hood and provides a simple @e ref system for element interaction. The skill supports web scraping, form filling, clicking, typing, drag-and-drop, file upload, and JavaScript execution, making it suitable for web automation, data extraction, testing, agent browsing, and research. To use the skill, you start by opening a URL with the open function to get interactive element refs, then use the interact function to perform actions like click, fill, select, and more. After navigation or dynamic content changes, you need to re-snapshot to refresh the refs. The skill also offers features such as video recording, cursor indicator, proxy support, and file upload, which are useful for debugging and demonstrations.

Installations · 4 087Voir la source

Installation

npx skills add https://github.com/skills-101/superpowers --skill agent-browser

Fichiers du skill

SKILL.md

Dernière synchronisation · 29 août 2026

references/authentication.md›
# Authentication Patterns

Login flows, OAuth, 2FA, and authenticated browsing.

**Related**: [session-management.md](session-management.md) for session details, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Login Flow](#basic-login-flow)
- [OAuth / SSO Flows](#oauth--sso-flows)
- [Two-Factor Authentication](#two-factor-authentication)
- [Session Reuse Patterns](#session-reuse-patterns)
- [Cookie Extraction](#cookie-extraction)
- [Security Best Practices](#security-best-practices)

## Basic Login Flow

Standard username/password login:

```bash
#!/bin/bash

# Start session
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/login"
}' | jq -r '.session_id')

# Get form elements
# Expected: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Sign In"

# Fill credentials
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e2", "text": "'"$PASSWORD"'"
}'

# Submit
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e3"
}'

# Wait for redirect
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 2000
}'

# Verify login succeeded
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
URL=$(echo $RESULT | jq -r '.url')

if [[ "$URL" == *"/login"* ]]; then
  echo "Login failed - still on login page"
  exit 1
fi

echo "Login successful"
# Continue with authenticated actions...
```

## OAuth / SSO Flows

For OAuth redirects (Google, GitHub, etc.):

```bash
#!/bin/bash

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/auth/google"
}' | jq -r '.session_id')

# Wait for redirect to Google
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 3000
}'

# Snapshot to see Google login form
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
echo $RESULT | jq '.elements_text'

# Fill Google email
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'

# Click Next
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e2"
}'

# Wait and snapshot for password field
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 2000
}'
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')

# Fill password
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "'"$GOOGLE_PASSWORD"'"
}'

# Click Sign in
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e2"
}'

# Wait for redirect back to app
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 5000
}'

# Verify we're back on the app
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
URL=$(echo $RESULT | jq -r '.url')
echo "Final URL: $URL"
```

## Two-Factor Authentication

For 2FA, you may need human intervention or TOTP generation:

### With TOTP Code

```bash
# After password, check for 2FA prompt
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
ELEMENTS=$(echo $RESULT | jq -r '.elements_text')

if echo "$ELEMENTS" | grep -qi "verification\|2fa\|authenticator"; then
  # Generate TOTP code (requires oathtool)
  TOTP_CODE=$(oathtool --totp -b "$TOTP_SECRET")

  # Fill 2FA code
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "fill", "ref": "@e1", "text": "'"$TOTP_CODE"'"
  }'

  # Submit
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "click", "ref": "@e2"
  }'
fi
```

### With Manual Intervention

For SMS or hardware token 2FA:

```bash
# Record video so user can see the 2FA prompt
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/login",
  "record_video": true
}' | jq -r '.session_id')

# ... login flow ...

# At 2FA step, prompt user
echo "2FA code sent. Enter the code:"
read -r CODE

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e1", "text": "'"$CODE"'"
}'
```

## Session Reuse Patterns

Since sessions maintain cookies, you can reuse authenticated sessions:

```bash
#!/bin/bash
# login-and-work.sh

# Login once
login() {
  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://app.example.com/login"
  }' | jq -r '.session_id')

  # ... login steps ...

  echo $SESSION
}

# Do work with authenticated session
do_work() {
  local SESSION=$1

  # Navigate to protected page
  belt app run agent-browser --function interact --session $SESSION --input '{
    "action": "goto", "url": "https://app.example.com/dashboard"
  }'

  # Extract data
  belt app run agent-browser --function snapshot --session $SESSION --input '{}'
}

# Main
SESSION=$(login)
do_work $SESSION

# Don't close if you want to reuse!
# belt app run agent-browser --function close --session $SESSION --input '{}'
```

## Cookie Extraction

Extract cookies for use in other tools:

```bash
# Get cookies via JavaScript
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.cookie"
}')
COOKIES=$(echo $RESULT | jq -r '.result')
echo "Cookies: $COOKIES"

# Get all cookies including httpOnly (more complete)
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "JSON.stringify(performance.getEntriesByType(\"resource\").map(r => r.name))"
}')
```

## Security Best Practices

### 1. Never Hardcode Credentials

```bash
# Good: Use environment variables
'{"action": "fill", "ref": "@e2", "text": "'"$PASSWORD"'"}'

# Bad: Hardcoded
'{"action": "fill", "ref": "@e2", "text": "mypassword123"}'
```

### 2. Use Secure Environment Variables

```bash
# Set securely
export PASSWORD=$(cat /path/to/secure/password)

# Or use a secrets manager
export PASSWORD=$(vault read -field=password secret/app)
```

### 3. Don't Log Sensitive Data

```bash
# Good: Redact sensitive info
echo "Logging in as $USERNAME"

# Bad: Logging passwords
echo "Password: $PASSWORD"  # Never do this!
```

### 4. Close Sessions After Use

```bash
# Always clean up
trap 'belt app run agent-browser --function close --session $SESSION --input "{}" 2>/dev/null' EXIT
```

### 5. Use Video Recording for Debugging Only

Video may capture sensitive information:

```bash
# Only enable when debugging
if [ "$DEBUG" = "true" ]; then
  RECORD_VIDEO="true"
else
  RECORD_VIDEO="false"
fi
```

### 6. Verify Login Success

Always confirm authentication worked:

```bash
# Check URL changed from login page
URL=$(echo $RESULT | jq -r '.url')
if [[ "$URL" == *"/login"* ]] || [[ "$URL" == *"/signin"* ]]; then
  echo "ERROR: Login failed"
  exit 1
fi

# Or check for specific element on authenticated page
ELEMENTS=$(echo $RESULT | jq -r '.elements_text')
if ! echo "$ELEMENTS" | grep -q "Logout\|Dashboard\|Welcome"; then
  echo "ERROR: Not authenticated"
  exit 1
fi
```
references/commands.md›
# Command Reference

Complete reference for all agent-browser functions. For quick start, see [SKILL.md](../SKILL.md).

## Base Command

All commands follow this pattern:

```bash
belt app run agent-browser --function <function> --session <session_id|new> --input '<json>'
```

- `--function`: Function to call (open, snapshot, interact, screenshot, execute, close)
- `--session`: Session ID from previous call, or `new` to start fresh
- `--input`: JSON input for the function

## Functions

### open

Navigate to URL and configure browser. This is the entry point for all sessions.

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "width": 1280,
  "height": 720,
  "user_agent": "Mozilla/5.0...",
  "record_video": false,
  "show_cursor": false,
  "proxy_url": null,
  "proxy_username": null,
  "proxy_password": null
}'
```

**Input Fields:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | required | URL to navigate to |
| `width` | int | 1280 | Viewport width in pixels |
| `height` | int | 720 | Viewport height in pixels |
| `user_agent` | string | null | Custom user agent string |
| `record_video` | bool | false | Record video (returned on close) |
| `show_cursor` | bool | false | Show cursor indicator in screenshots/video |
| `proxy_url` | string | null | Proxy server URL |
| `proxy_username` | string | null | Proxy auth username |
| `proxy_password` | string | null | Proxy auth password |

**Output:**

```json
{
  "session_id": "abc123",
  "url": "https://example.com",
  "title": "Example Domain",
  "elements": [...],
  "elements_text": "@e1 [a] \"More information...\" href=\"...\"\n...",
  "screenshot": "<File>"
}
```

### snapshot

Re-fetch page state with `@e` refs. Call after navigation or DOM changes.

```bash
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'
```

**Output:** Same as `open` (url, title, elements, elements_text, screenshot)

### interact

Perform actions on the page using `@e` refs.

```bash
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click",
  "ref": "@e1"
}'
```

**Input Fields:**

| Field | Type | Description |
|-------|------|-------------|
| `action` | string | Action to perform (see Actions table) |
| `ref` | string | Element ref (e.g., `@e1`) |
| `text` | string | Text for fill/type/press/select |
| `direction` | string | Scroll direction: up, down, left, right |
| `scroll_amount` | int | Scroll pixels (default 400) |
| `wait_ms` | int | Wait duration in milliseconds |
| `url` | string | URL for goto action |
| `target_ref` | string | Target ref for drag action |
| `file_paths` | array | File paths for upload action |

**Actions:**

| Action | Required Fields | Description |
|--------|-----------------|-------------|
| `click` | `ref` | Single click |
| `dblclick` | `ref` | Double click |
| `fill` | `ref`, `text` | Clear input and type text |
| `type` | `text` | Type text without clearing |
| `press` | `text` | Press key (Enter, Tab, Escape, etc.) |
| `select` | `ref`, `text` | Select dropdown option by label |
| `hover` | `ref` | Hover over element |
| `check` | `ref` | Check checkbox |
| `uncheck` | `ref` | Uncheck checkbox |
| `drag` | `ref`, `target_ref` | Drag from ref to target_ref |
| `upload` | `ref`, `file_paths` | Upload files to file input |
| `scroll` | `direction` | Scroll page (optional: `scroll_amount`) |
| `back` | - | Go back in browser history |
| `wait` | `wait_ms` | Wait for specified milliseconds |
| `goto` | `url` | Navigate to different URL |

**Output:**

```json
{
  "success": true,
  "action": "click",
  "message": null,
  "screenshot": "<File>",
  "snapshot": {
    "url": "...",
    "title": "...",
    "elements": [...],
    "elements_text": "..."
  }
}
```

### screenshot

Take a screenshot of the current page.

```bash
belt app run agent-browser --function screenshot --session $SESSION_ID --input '{
  "full_page": true
}'
```

**Input Fields:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `full_page` | bool | false | Capture full scrollable page |

**Output:**

```json
{
  "screenshot": "<File>",
  "width": 1280,
  "height": 720
}
```

### execute

Run JavaScript code on the page.

```bash
belt app run agent-browser --function execute --session $SESSION_ID --input '{
  "code": "document.title"
}'
```

**Input Fields:**

| Field | Type | Description |
|-------|------|-------------|
| `code` | string | JavaScript code to execute |

**Output:**

```json
{
  "result": "Example Domain",
  "error": null,
  "screenshot": "<File>"
}
```

**Examples:**

```bash
# Get page title
'{"code": "document.title"}'

# Count elements
'{"code": "document.querySelectorAll(\"a\").length"}'

# Extract text
'{"code": "document.querySelector(\"h1\").textContent"}'

# Get all links
'{"code": "Array.from(document.querySelectorAll(\"a\")).map(a => a.href)"}'

# Scroll to bottom
'{"code": "window.scrollTo(0, document.body.scrollHeight)"}'

# Get computed style
'{"code": "getComputedStyle(document.body).backgroundColor"}'
```

### close

Close the browser session. Returns video if recording was enabled.

```bash
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

**Output:**

```json
{
  "success": true,
  "video": "<File or null>"
}
```

## Key Combinations

For the `press` action, use these key names:

| Key | Name |
|-----|------|
| Enter | `Enter` |
| Tab | `Tab` |
| Escape | `Escape` |
| Backspace | `Backspace` |
| Delete | `Delete` |
| Arrow keys | `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight` |
| Modifiers | `Control`, `Shift`, `Alt`, `Meta` |

**Key combinations:**

```bash
# Ctrl+A (select all)
'{"action": "press", "text": "Control+a"}'

# Ctrl+C (copy)
'{"action": "press", "text": "Control+c"}'

# Shift+Tab (focus previous)
'{"action": "press", "text": "Shift+Tab"}'
```

## Error Handling

When an action fails, `success` is `false` and `message` contains the error:

```json
{
  "success": false,
  "action": "click",
  "message": "Unknown ref: @e99. Run 'snapshot' to get current elements.",
  "screenshot": "<File>",
  "snapshot": {...}
}
```

Common errors:
- `Unknown ref: @eN` - Ref doesn't exist, re-snapshot needed
- `'text' required for fill action` - Missing required field
- `'target_ref' required for drag action` - Missing drag target
- `Timeout 5000ms exceeded` - Element not found or not clickable
references/proxy-support.md›
# Proxy Support

Proxy configuration for geo-testing, privacy, and corporate environments.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Proxy Configuration](#basic-proxy-configuration)
- [Authenticated Proxy](#authenticated-proxy)
- [Common Use Cases](#common-use-cases)
- [Proxy Types](#proxy-types)
- [Verifying Proxy Connection](#verifying-proxy-connection)
- [Troubleshooting](#troubleshooting)
- [Best Practices](#best-practices)

## Basic Proxy Configuration

Set proxy when opening a session:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080"
}' | jq -r '.session_id')
```

All traffic for this session routes through the proxy.

## Authenticated Proxy

For proxies requiring username/password:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "myuser",
  "proxy_password": "mypassword"
}' | jq -r '.session_id')
```

## Common Use Cases

### Geo-Location Testing

Test how your site appears from different regions:

```bash
#!/bin/bash
# Test from multiple regions

PROXIES=(
  "us|http://us-proxy.example.com:8080"
  "eu|http://eu-proxy.example.com:8080"
  "asia|http://asia-proxy.example.com:8080"
)

for entry in "${PROXIES[@]}"; do
  REGION="${entry%%|*}"
  PROXY="${entry##*|}"

  echo "Testing from: $REGION"

  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://mysite.com",
    "proxy_url": "'"$PROXY"'"
  }' | jq -r '.session_id')

  # Take screenshot
  belt app run agent-browser --function screenshot --session $SESSION --input '{
    "full_page": true
  }' > "${REGION}-screenshot.json"

  # Get page content
  RESULT=$(belt app run agent-browser --function snapshot --session $SESSION --input '{}')
  echo $RESULT | jq '.elements_text' > "${REGION}-elements.txt"

  belt app run agent-browser --function close --session $SESSION --input '{}'
done

echo "Geo-testing complete"
```

### Rate Limit Avoidance

Rotate proxies for web scraping:

```bash
#!/bin/bash
# Rotate through proxy list

PROXIES=(
  "http://proxy1.example.com:8080"
  "http://proxy2.example.com:8080"
  "http://proxy3.example.com:8080"
)

URLS=(
  "https://site.com/page1"
  "https://site.com/page2"
  "https://site.com/page3"
)

for i in "${!URLS[@]}"; do
  # Rotate proxy
  PROXY_INDEX=$((i % ${#PROXIES[@]}))
  PROXY="${PROXIES[$PROXY_INDEX]}"
  URL="${URLS[$i]}"

  echo "Fetching $URL via proxy $((PROXY_INDEX + 1))"

  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "'"$URL"'",
    "proxy_url": "'"$PROXY"'"
  }' | jq -r '.session_id')

  # Extract data
  RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
    "code": "document.body.innerText"
  }')
  echo $RESULT | jq -r '.result' > "page-$i.txt"

  belt app run agent-browser --function close --session $SESSION --input '{}'

  # Polite delay
  sleep 1
done
```

### Corporate Network Access

Access sites through corporate proxy:

```bash
# Use corporate proxy for external sites
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://external-vendor.com",
  "proxy_url": "http://corpproxy.company.com:8080",
  "proxy_username": "'"$CORP_USER"'",
  "proxy_password": "'"$CORP_PASS"'"
}' | jq -r '.session_id')
```

### Privacy and Anonymity

Route through privacy-focused proxy:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://whatismyip.com",
  "proxy_url": "socks5://privacy-proxy.example.com:1080"
}' | jq -r '.session_id')
```

## Proxy Types

### HTTP/HTTPS Proxy

```json
{"proxy_url": "http://proxy.example.com:8080"}
{"proxy_url": "https://proxy.example.com:8080"}
```

### SOCKS5 Proxy

```json
{"proxy_url": "socks5://proxy.example.com:1080"}
```

### With Authentication

```json
{
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}
```

## Verifying Proxy Connection

Check that traffic routes through proxy:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://httpbin.org/ip",
  "proxy_url": "http://proxy.example.com:8080"
}' | jq -r '.session_id')

# Get the IP shown
RESULT=$(belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.body.innerText"
}')
echo "IP via proxy: $(echo $RESULT | jq -r '.result')"

belt app run agent-browser --function close --session $SESSION --input '{}'
```

The IP should be the proxy's IP, not your real IP.

## Troubleshooting

### Connection Failed

```
Error: Failed to open URL: net::ERR_PROXY_CONNECTION_FAILED
```

**Solutions:**
1. Verify proxy URL is correct
2. Check proxy is running and accessible
3. Confirm port is correct
4. Test proxy with curl: `curl -x http://proxy:8080 https://example.com`

### Authentication Failed

```
Error: 407 Proxy Authentication Required
```

**Solutions:**
1. Verify username/password are correct
2. Check if proxy requires different auth method
3. Ensure credentials don't contain special characters that need escaping

### SSL Errors

Some proxies perform SSL inspection. If you see certificate errors:

```bash
# The browser should handle most SSL proxies automatically
# If issues persist, verify proxy SSL certificate is valid
```

### Slow Performance

**Solutions:**
1. Choose proxy closer to target site
2. Use faster proxy provider
3. Reduce number of requests per session

## Best Practices

### 1. Use Environment Variables

```bash
# Good: Credentials in env vars
'{"proxy_url": "'"$PROXY_URL"'", "proxy_username": "'"$PROXY_USER"'"}'

# Bad: Hardcoded
'{"proxy_url": "http://user:[email protected]:8080"}'
```

### 2. Test Proxy Before Automation

```bash
# Verify proxy works
curl -x "$PROXY_URL" https://httpbin.org/ip
```

### 3. Handle Proxy Failures

```bash
# Retry with different proxy on failure
for PROXY in "${PROXIES[@]}"; do
  SESSION=$(belt app run agent-browser --function open --session new --input '{
    "url": "'"$URL"'",
    "proxy_url": "'"$PROXY"'"
  }' 2>&1)

  if echo "$SESSION" | jq -e '.session_id' > /dev/null 2>&1; then
    SESSION_ID=$(echo $SESSION | jq -r '.session_id')
    break
  fi
  echo "Proxy $PROXY failed, trying next..."
done
```

### 4. Respect Rate Limits

Even with proxies, be a good citizen:

```bash
# Add delays between requests
'{"action": "wait", "wait_ms": 1000}'
```

### 5. Log Proxy Usage

For debugging, log which proxy was used:

```bash
echo "$(date): Using proxy $PROXY for $URL" >> proxy.log
```
references/session-management.md›
# Session Management

Browser sessions for state persistence and parallel browsing.

**Related**: [authentication.md](authentication.md) for login patterns, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [How Sessions Work](#how-sessions-work)
- [Starting a Session](#starting-a-session)
- [Using Session IDs](#using-session-ids)
- [Session State](#session-state)
- [Parallel Sessions](#parallel-sessions)
- [Session Cleanup](#session-cleanup)
- [Best Practices](#best-practices)

## How Sessions Work

Each session maintains an isolated browser context with:
- Cookies
- LocalStorage / SessionStorage
- Browser history
- Page state
- Video recording (if enabled)

Sessions persist across function calls, allowing multi-step workflows.

## Starting a Session

Use `--session new` to create a fresh session:

```bash
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
echo "Session: $SESSION_ID"
```

## Using Session IDs

All subsequent calls use the session ID:

```bash
# Navigate
belt app run agent-browser --function open --session $SESSION_ID --input '{
  "url": "https://example.com/page2"
}'

# Interact
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e1"
}'

# Screenshot
belt app run agent-browser --function screenshot --session $SESSION_ID --input '{}'

# Close
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

## Session State

### What Persists

Within a session, these persist across calls:
- Cookies (login state, preferences)
- LocalStorage and SessionStorage
- IndexedDB data
- Browser history (for back/forward)
- Current page and DOM state
- Video recording buffer

### What Doesn't Persist

- Sessions don't persist across server restarts
- No automatic session recovery
- Video is only available until close is called

## Parallel Sessions

Run multiple independent sessions simultaneously:

```bash
#!/bin/bash
# Scrape multiple sites in parallel

# Start sessions
RESULT1=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://site1.com"
}')
SESSION1=$(echo $RESULT1 | jq -r '.session_id')

RESULT2=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://site2.com"
}')
SESSION2=$(echo $RESULT2 | jq -r '.session_id')

# Work with each session independently
belt app run agent-browser --function screenshot --session $SESSION1 --input '{}' &
belt app run agent-browser --function screenshot --session $SESSION2 --input '{}' &
wait

# Clean up both
belt app run agent-browser --function close --session $SESSION1 --input '{}'
belt app run agent-browser --function close --session $SESSION2 --input '{}'
```

### Use Cases for Parallel Sessions

1. **A/B Testing** - Compare different pages or user experiences
2. **Multi-site scraping** - Gather data from multiple sources
3. **Load testing** - Simulate multiple users
4. **Cross-region testing** - Use different proxies per session

## Session Cleanup

Always close sessions when done:

```bash
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

**Why close matters:**
- Releases server resources
- Returns video recording (if enabled)
- Prevents resource leaks

### Error Handling

```bash
#!/bin/bash
set -e

cleanup() {
  belt app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
}
trap cleanup EXIT

SESSION_ID=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com"
}' | jq -r '.session_id')

# ... your automation ...
# cleanup runs automatically on exit
```

## Best Practices

### 1. Store Session IDs

```bash
# Good: Store for reuse
SESSION_ID=$(... | jq -r '.session_id')
belt ... --session $SESSION_ID ...

# Bad: Parse every time
belt ... --session $(... | jq -r '.session_id') ...
```

### 2. Close Sessions Promptly

Don't leave sessions open longer than needed. Server resources are limited.

### 3. Use Meaningful Variable Names

```bash
# Good: Clear purpose
LOGIN_SESSION=$(...)
SCRAPE_SESSION=$(...)

# Bad: Generic names
S1=$(...)
S2=$(...)
```

### 4. Handle Session Expiry

Sessions may expire after extended inactivity:

```bash
# Check if session is still valid
RESULT=$(belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}' 2>&1)
if echo "$RESULT" | grep -q "session not found"; then
  echo "Session expired, starting new one"
  SESSION_ID=$(belt app run agent-browser --function open --session new --input '{
    "url": "https://example.com"
  }' | jq -r '.session_id')
fi
```

### 5. One Task Per Session

For clarity, use one session per logical task:

```bash
# Good: Separate sessions for separate tasks
LOGIN_SESSION=$(...)  # Handle login
SCRAPE_SESSION=$(...)  # Handle scraping

# Okay for related tasks: One session for a workflow
SESSION=$(...)
# login -> navigate -> extract -> close
```
references/snapshot-refs.md›
# Snapshot and Refs

Compact element references that reduce context usage for AI agents.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [How Refs Work](#how-refs-work)
- [Snapshot Output Format](#snapshot-output-format)
- [Using Refs](#using-refs)
- [Ref Lifecycle](#ref-lifecycle)
- [Best Practices](#best-practices)
- [Ref Notation Details](#ref-notation-details)
- [Troubleshooting](#troubleshooting)

## How Refs Work

Traditional approach:
```
Full DOM/HTML -> AI parses -> CSS selector -> Action (~3000-5000 tokens)
```

agent-browser approach:
```
Compact snapshot -> @refs assigned -> Direct interaction (~200-400 tokens)
```

The snapshot extracts interactive elements and assigns short `@e` refs, reducing token usage significantly.

## Snapshot Output Format

```bash
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
```

**Response `elements_text`:**

```
@e1 [a] "Home" href="/"
@e2 [a] "Products" href="/products"
@e3 [a] "About" href="/about"
@e4 [button] "Sign In"
@e5 [input type="email"] placeholder="Email"
@e6 [input type="password"] placeholder="Password"
@e7 [button type="submit"] "Log In"
@e8 [input type="checkbox"] name="remember"
```

**Response `elements` (structured):**

```json
[
  {
    "ref": "@e1",
    "desc": "@e1 [a] \"Home\" href=\"/\"",
    "tag": "a",
    "text": "Home",
    "role": null,
    "name": null,
    "href": "/",
    "input_type": null
  },
  ...
]
```

## Using Refs

Once you have refs, interact directly:

```bash
# Click the "Sign In" button
'{"action": "click", "ref": "@e4"}'

# Fill email input
'{"action": "fill", "ref": "@e5", "text": "[email protected]"}'

# Fill password
'{"action": "fill", "ref": "@e6", "text": "password123"}'

# Submit the form
'{"action": "click", "ref": "@e7"}'

# Check the "remember me" checkbox
'{"action": "check", "ref": "@e8"}'
```

## Ref Lifecycle

**IMPORTANT**: Refs are invalidated when the page changes!

```bash
# Get initial snapshot
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# @e1 [button] "Next"

# Click triggers page change
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}'

# MUST re-snapshot to get new refs!
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# @e1 [h1] "Page 2"  <- Different element now!
```

### When to Re-snapshot

Always re-snapshot after:

1. **Navigation** - Clicking links, form submissions, `goto` action
2. **Dynamic content** - AJAX loads, modals opening, tabs switching
3. **Page mutations** - JavaScript modifying the DOM

The `interact` function returns a fresh snapshot in its response, so you can often use that instead of a separate snapshot call.

## Best Practices

### 1. Always Use the Latest Snapshot

```bash
# CORRECT: Use snapshot from previous response
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}')
# Use elements from $RESULT.snapshot for next action

# WRONG: Using stale refs
# After navigation, @e1 may point to a completely different element
```

### 2. Check Success Before Continuing

```bash
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e5"
}')

SUCCESS=$(echo $RESULT | jq -r '.success')
if [ "$SUCCESS" != "true" ]; then
  echo "Click failed: $(echo $RESULT | jq -r '.message')"
  # Re-snapshot and retry
fi
```

### 3. Use elements_text for Quick Decisions

For AI agents, `elements_text` provides a compact text representation:

```
@e1 [input type="email"] placeholder="Email"
@e2 [input type="password"] placeholder="Password"
@e3 [button] "Submit"
```

This is often enough to decide which element to interact with without parsing the full `elements` array.

## Ref Notation Details

```
@e1 [tag type="value"] "text content" name="attr"
|    |   |             |              |
|    |   |             |              +- Additional attributes
|    |   |             +- Visible text
|    |   +- Key attributes shown
|    +- HTML tag name
+- Unique ref ID
```

### Common Patterns

```
@e1 [button] "Submit"                    # Button with text
@e2 [input type="email"]                 # Email input
@e3 [input type="password"]              # Password input
@e4 [a] "Link Text" href="/page"         # Anchor link
@e5 [select]                             # Dropdown
@e6 [textarea] placeholder="Message"     # Text area
@e7 [input type="file"]                  # File upload
@e8 [input type="checkbox"] checked      # Checked checkbox
@e9 [input type="radio"] selected        # Selected radio
@e10 [button type="submit"] "Send"       # Submit button
```

### Elements Captured

The snapshot captures these interactive elements:

- Links (`<a href>`)
- Buttons (`<button>`, `[role="button"]`)
- Inputs (`<input>`, `<textarea>`, `<select>`)
- Clickable elements (`[onclick]`, `[tabindex]`)
- ARIA roles (`[role="link"]`, `[role="checkbox"]`, etc.)

Non-interactive or hidden elements are filtered out.

## Troubleshooting

### "Unknown ref" Error

```json
{
  "success": false,
  "message": "Unknown ref: @e15. Run 'snapshot' to get current elements."
}
```

**Solution**: Re-snapshot. The page changed and refs are stale.

```bash
belt app run agent-browser --function snapshot --session $SESSION --input '{}'
# Now use the new refs
```

### Element Not in Snapshot

The element you need might not appear because:

1. **Not visible** - Scroll to reveal it
   ```bash
   '{"action": "scroll", "direction": "down", "scroll_amount": 500}'
   ```

2. **Not interactive** - Use JavaScript to interact
   ```bash
   '{"code": "document.querySelector(\".hidden-btn\").click()"}'
   ```

3. **In iframe** - Currently not supported (use `execute` with JS)

4. **Dynamic** - Wait for it to load
   ```bash
   '{"action": "wait", "wait_ms": 2000}'
   ```

### Too Many Elements

Snapshots are limited to 50 elements. If the page has more:

1. **Scroll** to bring relevant elements into view
2. **Use JavaScript** to target specific elements
3. **Navigate** to a more specific page

### Ref Points to Wrong Element

If a ref seems to interact with the wrong element:

1. Re-snapshot to get fresh refs
2. Check if the page structure changed
3. Verify with screenshot that the right element is targeted
references/video-recording.md›
# Video Recording

Capture browser automation as video for debugging, documentation, or verification.

**Related**: [commands.md](commands.md) for full function reference, [SKILL.md](../SKILL.md) for quick start.

## Contents

- [Basic Recording](#basic-recording)
- [Cursor Indicator](#cursor-indicator)
- [How Recording Works](#how-recording-works)
- [Use Cases](#use-cases)
- [Best Practices](#best-practices)
- [Output Format](#output-format)
- [Limitations](#limitations)

## Basic Recording

Enable video recording when opening a session:

```bash
# Start with recording enabled
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Perform actions
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}'

belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "fill", "ref": "@e2", "text": "test input"
}'

# Close to get the video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $RESULT | jq -r '.video')
echo "Video file: $VIDEO"
```

## Cursor Indicator

For demos and documentation, show a visible cursor that follows mouse movements:

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')
```

The cursor appears as a red dot that:
- Follows mouse movements in real-time
- Shows click feedback (shrinks on mousedown)
- Persists across page navigations
- Appears in both screenshots and video

This is especially useful for:
- Tutorial/documentation videos
- Debugging interaction issues
- Sharing recordings with non-technical stakeholders

## How Recording Works

1. **Start**: Pass `"record_video": true` in the `open` function
2. **Record**: All browser activity is captured throughout the session
3. **Stop**: Video is finalized when `close` is called
4. **Retrieve**: Video file is returned in the `close` response

The video captures:
- Page loads and navigations
- Element interactions (clicks, typing)
- Scrolling and animations
- Dynamic content changes

## Use Cases

### Debugging Failed Automation

```bash
#!/bin/bash
# Record automation for debugging

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com",
  "record_video": true
}' | jq -r '.session_id')

# Run automation
RESULT=$(belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e1"
}')

SUCCESS=$(echo $RESULT | jq -r '.success')
if [ "$SUCCESS" != "true" ]; then
  echo "Action failed!"
  echo "Message: $(echo $RESULT | jq -r '.message')"

  # Get video for debugging
  CLOSE_RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
  echo "Debug video: $(echo $CLOSE_RESULT | jq -r '.video')"
  exit 1
fi

belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Documentation Generation

Record workflows for user documentation:

```bash
#!/bin/bash
# Record how-to video

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://app.example.com/settings",
  "record_video": true,
  "width": 1920,
  "height": 1080
}' | jq -r '.session_id')

# Add pauses for clarity
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 1000
}'

# Step 1: Click settings
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e5"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 500
}'

# Step 2: Change setting
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e10"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 500
}'

# Step 3: Save
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "click", "ref": "@e15"
}'
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "wait", "wait_ms": 1000
}'

# Get the video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo "Documentation video: $(echo $RESULT | jq -r '.video')"
```

### Test Evidence for CI/CD

```bash
#!/bin/bash
# Record E2E test for CI artifacts

TEST_NAME="${1:-e2e-test}"

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "'"$TEST_URL"'",
  "record_video": true
}' | jq -r '.session_id')

# Run test steps
run_test_steps $SESSION
TEST_RESULT=$?

# Always get video
CLOSE_RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $CLOSE_RESULT | jq -r '.video')

# Save to artifacts
if [ -n "$CI_ARTIFACTS_DIR" ]; then
  cp "$VIDEO" "$CI_ARTIFACTS_DIR/${TEST_NAME}.webm"
fi

exit $TEST_RESULT
```

### Monitoring and Auditing

```bash
#!/bin/bash
# Record automated task for audit trail

TASK_ID=$(date +%Y%m%d-%H%M%S)

SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://admin.example.com",
  "record_video": true
}' | jq -r '.session_id')

# Perform admin task
# ... automation steps ...

# Save recording
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
VIDEO=$(echo $RESULT | jq -r '.video')

# Archive for audit
mv "$VIDEO" "/audit/recordings/${TASK_ID}.webm"
echo "Audit recording saved: ${TASK_ID}.webm"
```

## Best Practices

### 1. Add Strategic Pauses

Pauses make videos easier to follow:

```bash
# After significant actions, add a pause
'{"action": "click", "ref": "@e1"}'
'{"action": "wait", "wait_ms": 500}'  # Let viewer see result
```

### 2. Use Larger Viewport for Documentation

```bash
'{"url": "...", "record_video": true, "width": 1920, "height": 1080}'
```

### 3. Handle Errors Gracefully

Always retrieve video even on failure:

```bash
cleanup() {
  if [ -n "$SESSION" ]; then
    belt app run agent-browser --function close --session $SESSION --input '{}' 2>/dev/null
  fi
}
trap cleanup EXIT
```

### 4. Combine with Screenshots

Use screenshots for key frames, video for flow:

```bash
# Record overall flow
'{"record_video": true}'

# Capture key states
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'
```

### 5. Don't Record Sensitive Sessions

Avoid recording when handling credentials:

```bash
if [ "$CONTAINS_SENSITIVE_DATA" = "true" ]; then
  RECORD="false"
else
  RECORD="true"
fi

'{"url": "...", "record_video": '$RECORD'}'
```

## Output Format

- **Format**: WebM (VP8/VP9 codec)
- **Compatibility**: All modern browsers and video players
- **Quality**: Matches viewport size
- **Compression**: Efficient for screen content

## Limitations

1. **Session-level only** - Can't start/stop mid-session
2. **Memory usage** - Long sessions consume more memory
3. **File size** - Complex pages with animations produce larger files
4. **No audio** - Browser audio is not captured
5. **Returned on close** - Video only available after session ends
SKILL.md›
---
name: agent-browser
description: "Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: web automation, data extraction, testing, agent browsing, research. Triggers: browser, web automation, scrape, navigate, click, fill form, screenshot, browse web, playwright, headless browser, web agent, surf internet, record video"
allowed-tools: Bash(belt *)
---

> **Install the belt CLI skill:** `npx skills add belt-sh/cli`

# Agentic Browser

Browser automation for AI agents via [inference.sh](https://inference.sh). Uses Playwright under the hood with a simple `@e` ref system for element interaction.

![Agentic Browser](https://cloud.inference.sh/app/files/u/4mg21r6ta37mpaz6ktzwtt8krr/01kgjw8atdxgkrsr8a2t5peq7b.jpeg)

## Quick Start

> Requires inference.sh CLI (`belt`). [Install instructions](https://raw.githubusercontent.com/inference-sh/skills/refs/heads/main/cli-install.md)

```bash
belt login

# Open a page and get interactive elements
belt app run agent-browser --function open --input '{"url": "https://example.com"}' --session new
```


## Core Workflow

Every browser automation follows this pattern:

1. **Open** - Navigate to URL, get `@e` refs for elements
2. **Interact** - Use refs to click, fill, drag, etc.
3. **Re-snapshot** - After navigation/changes, get fresh refs
4. **Close** - End session (returns video if recording)

```bash
# 1. Start session
RESULT=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/login"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')
# Elements: @e1 [input] "Email", @e2 [input] "Password", @e3 [button] "Sign In"

# 2. Fill and submit
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e1", "text": "[email protected]"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "fill", "ref": "@e2", "text": "password123"
}'
belt app run agent-browser --function interact --session $SESSION_ID --input '{
  "action": "click", "ref": "@e3"
}'

# 3. Re-snapshot after navigation
belt app run agent-browser --function snapshot --session $SESSION_ID --input '{}'

# 4. Close when done
belt app run agent-browser --function close --session $SESSION_ID --input '{}'
```

## Functions

| Function | Description |
|----------|-------------|
| `open` | Navigate to URL, configure browser (viewport, proxy, video recording) |
| `snapshot` | Re-fetch page state with `@e` refs after DOM changes |
| `interact` | Perform actions using `@e` refs (click, fill, drag, upload, etc.) |
| `screenshot` | Take page screenshot (viewport or full page) |
| `execute` | Run JavaScript code on the page |
| `close` | Close session, returns video if recording was enabled |

## Interact Actions

| Action | Description | Required Fields |
|--------|-------------|-----------------|
| `click` | Click element | `ref` |
| `dblclick` | Double-click element | `ref` |
| `fill` | Clear and type text | `ref`, `text` |
| `type` | Type text (no clear) | `text` |
| `press` | Press key (Enter, Tab, etc.) | `text` |
| `select` | Select dropdown option | `ref`, `text` |
| `hover` | Hover over element | `ref` |
| `check` | Check checkbox | `ref` |
| `uncheck` | Uncheck checkbox | `ref` |
| `drag` | Drag and drop | `ref`, `target_ref` |
| `upload` | Upload file(s) | `ref`, `file_paths` |
| `scroll` | Scroll page | `direction` (up/down/left/right), `scroll_amount` |
| `back` | Go back in history | - |
| `wait` | Wait milliseconds | `wait_ms` |
| `goto` | Navigate to URL | `url` |

## Element Refs

Elements are returned with `@e` refs:

```
@e1 [a] "Home" href="/"
@e2 [input type="text"] placeholder="Search"
@e3 [button] "Submit"
@e4 [select] "Choose option"
@e5 [input type="checkbox"] name="agree"
```

**Important:** Refs are invalidated after navigation. Always re-snapshot after:
- Clicking links/buttons that navigate
- Form submissions
- Dynamic content loading

## Features

### Video Recording

Record browser sessions for debugging or documentation:

```bash
# Start with recording enabled (optionally show cursor indicator)
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true,
  "show_cursor": true
}' | jq -r '.session_id')

# ... perform actions ...

# Close to get the video file
belt app run agent-browser --function close --session $SESSION --input '{}'
# Returns: {"success": true, "video": <File>}
```

### Cursor Indicator

Show a visible cursor in screenshots and video (useful for demos):

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "show_cursor": true,
  "record_video": true
}'
```

The cursor appears as a red dot that follows mouse movements and shows click feedback.

### Proxy Support

Route traffic through a proxy server:

```bash
belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "proxy_url": "http://proxy.example.com:8080",
  "proxy_username": "user",
  "proxy_password": "pass"
}'
```

### File Upload

Upload files to file inputs:

```bash
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "upload",
  "ref": "@e5",
  "file_paths": ["/path/to/file.pdf"]
}'
```

### Drag and Drop

Drag elements to targets:

```bash
belt app run agent-browser --function interact --session $SESSION --input '{
  "action": "drag",
  "ref": "@e1",
  "target_ref": "@e2"
}'
```

### JavaScript Execution

Run custom JavaScript:

```bash
belt app run agent-browser --function execute --session $SESSION --input '{
  "code": "document.querySelectorAll(\"h2\").length"
}'
# Returns: {"result": "5", "screenshot": <File>}
```

## Deep-Dive Documentation

| Reference | Description |
|-----------|-------------|
| [references/commands.md](references/commands.md) | Full function reference with all options |
| [references/snapshot-refs.md](references/snapshot-refs.md) | Ref lifecycle, invalidation rules, troubleshooting |
| [references/session-management.md](references/session-management.md) | Session persistence, parallel sessions |
| [references/authentication.md](references/authentication.md) | Login flows, OAuth, 2FA handling |
| [references/video-recording.md](references/video-recording.md) | Recording workflows for debugging |
| [references/proxy-support.md](references/proxy-support.md) | Proxy configuration, geo-testing |

## Ready-to-Use Templates

| Template | Description |
|----------|-------------|
| [templates/form-automation.sh](templates/form-automation.sh) | Form filling with validation |
| [templates/authenticated-session.sh](templates/authenticated-session.sh) | Login once, reuse session |
| [templates/capture-workflow.sh](templates/capture-workflow.sh) | Content extraction with screenshots |

## Examples

### Form Submission

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com/contact"
}' | jq -r '.session_id')

# Get elements: @e1 [input] "Name", @e2 [input] "Email", @e3 [textarea], @e4 [button] "Send"

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "John Doe"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e2", "text": "[email protected]"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e3", "text": "Hello!"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "click", "ref": "@e4"}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Search and Extract

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://google.com"
}' | jq -r '.session_id')

belt app run agent-browser --function interact --session $SESSION --input '{"action": "fill", "ref": "@e1", "text": "weather today"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "press", "text": "Enter"}'
belt app run agent-browser --function interact --session $SESSION --input '{"action": "wait", "wait_ms": 2000}'

belt app run agent-browser --function snapshot --session $SESSION --input '{}'
belt app run agent-browser --function close --session $SESSION --input '{}'
```

### Screenshot with Video

```bash
SESSION=$(belt app run agent-browser --function open --session new --input '{
  "url": "https://example.com",
  "record_video": true
}' | jq -r '.session_id')

# Take full page screenshot
belt app run agent-browser --function screenshot --session $SESSION --input '{
  "full_page": true
}'

# Close and get video
RESULT=$(belt app run agent-browser --function close --session $SESSION --input '{}')
echo $RESULT | jq '.video'
```

## Sessions

Browser state persists within a session. Always:

1. Start with `--session new` on first call
2. Use returned `session_id` for subsequent calls
3. Close session when done

## Related Skills

```bash
# Web search (for research + browse)
npx skills add inference-sh/skills@web-search

# LLM models (analyze extracted content)
npx skills add inference-sh/skills@llm-models
```

## Documentation

- [inference.sh Sessions](https://inference.sh/docs/extend/sessions) - Session management
- [Multi-function Apps](https://inference.sh/docs/extend/multi-function-apps) - How functions work
templates/authenticated-session.sh›
#!/bin/bash
# Template: Authenticated Session Workflow
# Purpose: Login once, perform actions, clean up
# Usage: ./authenticated-session.sh <login-url>
#
# Environment variables:
#   APP_USERNAME - Login username/email
#   APP_PASSWORD - Login password
#
# Two modes:
#   1. Discovery mode (default): Shows login form structure
#   2. Login mode: Performs actual login after you update refs
#
# Setup steps:
#   1. Run once to see form structure (discovery mode)
#   2. Update refs in LOGIN FLOW section below
#   3. Set APP_USERNAME and APP_PASSWORD
#   4. Comment out the DISCOVERY section

set -euo pipefail

LOGIN_URL="${1:?Usage: $0 <login-url>}"

echo "Authentication workflow: $LOGIN_URL"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    echo "Closing session..."
    infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
  fi
}
trap cleanup EXIT

# ================================================================
# DISCOVERY MODE: Shows login form structure
# Delete this section after setup
# ================================================================
echo "Opening login page..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$LOGIN_URL"'"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

echo ""
echo "Login form structure:"
echo "---"
echo $RESULT | jq -r '.elements_text'
echo "---"
echo ""
echo "Discovery mode complete."
echo ""
echo "Next steps:"
echo "  1. Identify the refs: username=@e?, password=@e?, submit=@e?"
echo "  2. Update the LOGIN FLOW section below with your refs"
echo "  3. Set environment variables:"
echo "     export APP_USERNAME='your-username'"
echo "     export APP_PASSWORD='your-password'"
echo "  4. Comment out this DISCOVERY MODE section"
echo ""
exit 0

# ================================================================
# LOGIN FLOW: Uncomment and customize after discovery
# ================================================================
# : "${APP_USERNAME:?Set APP_USERNAME environment variable}"
# : "${APP_PASSWORD:?Set APP_PASSWORD environment variable}"
#
# echo "Opening login page..."
# RESULT=$(infsh app run agent-browser --function open --session new --input '{
#   "url": "'"$LOGIN_URL"'",
#   "record_video": false
# }')
# SESSION_ID=$(echo $RESULT | jq -r '.session_id')
#
# echo "Filling credentials..."
# # Update @e1, @e2, @e3 to match your form
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e1", "text": "'"$APP_USERNAME"'"
# }'
#
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e2", "text": "'"$APP_PASSWORD"'"
# }'
#
# echo "Submitting..."
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "click", "ref": "@e3"
# }'
#
# # Wait for redirect
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "wait", "wait_ms": 3000
# }'
#
# # Verify login succeeded
# RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
# URL=$(echo $RESULT | jq -r '.url')
#
# if [[ "$URL" == *"/login"* ]] || [[ "$URL" == *"/signin"* ]]; then
#   echo "ERROR: Login failed - still on login page"
#   echo "URL: $URL"
#   infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{}' > login-failed.json
#   exit 1
# fi
#
# echo "Login successful!"
# echo "Current URL: $URL"
# echo ""
#
# # ================================================================
# # AUTHENTICATED ACTIONS: Add your post-login automation here
# # ================================================================
# echo "Performing authenticated actions..."
#
# # Example: Navigate to dashboard
# # infsh app run agent-browser --function interact --session $SESSION_ID --input '{
# #   "action": "goto", "url": "https://app.example.com/dashboard"
# # }'
#
# # Example: Click a menu item
# # infsh app run agent-browser --function interact --session $SESSION_ID --input '{
# #   "action": "click", "ref": "@e5"
# # }'
#
# # Example: Extract data
# # RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
# #   "code": "document.querySelector(\".user-data\").textContent"
# # }')
# # echo "Data: $(echo $RESULT | jq -r '.result')"
#
# # Example: Take screenshot of authenticated page
# # infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
# #   "full_page": true
# # }' > authenticated-page.json
#
# echo ""
# echo "Authenticated session complete"
templates/capture-workflow.sh›
#!/bin/bash
# Template: Content Capture Workflow
# Purpose: Extract content from web pages (text, screenshots, video)
# Usage: ./capture-workflow.sh <url> [output-dir]
#
# Outputs:
#   - page-screenshot.json: Page screenshot data
#   - page-full-screenshot.json: Full page screenshot data
#   - page-elements.txt: Interactive elements with refs
#   - page-text.txt: All text content
#   - page-links.txt: All links on the page
#   - session-video.json: Video recording (if enabled)

set -euo pipefail

TARGET_URL="${1:?Usage: $0 <url> [output-dir]}"
OUTPUT_DIR="${2:-.}"

echo "Content capture: $TARGET_URL"
echo "Output directory: $OUTPUT_DIR"

mkdir -p "$OUTPUT_DIR"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    echo "Closing session..."
    CLOSE_RESULT=$(infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || echo '{}')

    # Save video if available
    VIDEO=$(echo $CLOSE_RESULT | jq -r '.video // empty')
    if [ -n "$VIDEO" ]; then
      echo "$CLOSE_RESULT" > "$OUTPUT_DIR/session-video.json"
      echo "Video saved to: $OUTPUT_DIR/session-video.json"
    fi
  fi
}
trap cleanup EXIT

# ================================================================
# CONFIGURATION
# ================================================================
RECORD_VIDEO=false  # Set to true to record video
FULL_PAGE=true      # Set to true for full page screenshots
EXTRACT_LINKS=true  # Set to true to extract all links
SCROLL_PAGES=0      # Number of scroll actions for infinite scroll pages

# ================================================================
# CAPTURE WORKFLOW
# ================================================================

# Start session
echo "Opening page..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$TARGET_URL"'",
  "record_video": '$RECORD_VIDEO',
  "width": 1920,
  "height": 1080
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

# Get metadata
URL=$(echo $RESULT | jq -r '.url')
TITLE=$(echo $RESULT | jq -r '.title')
echo "Title: $TITLE"
echo "URL: $URL"

# Save elements
echo $RESULT | jq -r '.elements_text' > "$OUTPUT_DIR/page-elements.txt"
echo "Elements saved to: $OUTPUT_DIR/page-elements.txt"

# Handle infinite scroll (if configured)
if [ $SCROLL_PAGES -gt 0 ]; then
  echo "Scrolling through $SCROLL_PAGES pages..."
  for ((i=1; i<=SCROLL_PAGES; i++)); do
    infsh app run agent-browser --function interact --session $SESSION_ID --input '{
      "action": "scroll", "direction": "down", "scroll_amount": 800
    }' > /dev/null
    infsh app run agent-browser --function interact --session $SESSION_ID --input '{
      "action": "wait", "wait_ms": 1000
    }' > /dev/null
    echo "  Scrolled page $i/$SCROLL_PAGES"
  done

  # Re-snapshot after scrolling
  RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
fi

# Take viewport screenshot
echo "Taking viewport screenshot..."
infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{}' > "$OUTPUT_DIR/page-screenshot.json"
echo "Screenshot saved to: $OUTPUT_DIR/page-screenshot.json"

# Take full page screenshot (if configured)
if [ "$FULL_PAGE" = true ]; then
  echo "Taking full page screenshot..."
  infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
    "full_page": true
  }' > "$OUTPUT_DIR/page-full-screenshot.json"
  echo "Full screenshot saved to: $OUTPUT_DIR/page-full-screenshot.json"
fi

# Extract all text content
echo "Extracting text content..."
RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
  "code": "document.body.innerText"
}')
echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/page-text.txt"
echo "Text saved to: $OUTPUT_DIR/page-text.txt"

# Extract all links (if configured)
if [ "$EXTRACT_LINKS" = true ]; then
  echo "Extracting links..."
  RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
    "code": "Array.from(document.querySelectorAll(\"a[href]\")).map(a => a.href + \" | \" + (a.textContent || \"\").trim().slice(0,50)).join(\"\\n\")"
  }')
  echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/page-links.txt"
  echo "Links saved to: $OUTPUT_DIR/page-links.txt"
fi

# ================================================================
# CUSTOM EXTRACTION: Add your specific extraction logic here
# ================================================================

# Example: Extract specific elements by selector
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "Array.from(document.querySelectorAll(\"h2\")).map(h => h.textContent).join(\"\\n\")"
# }')
# echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/headings.txt"

# Example: Extract JSON data from script tag
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "JSON.parse(document.querySelector(\"script[type=application/json]\").textContent)"
# }')
# echo $RESULT | jq '.result' > "$OUTPUT_DIR/json-data.json"

# Example: Extract table data
# RESULT=$(infsh app run agent-browser --function execute --session $SESSION_ID --input '{
#   "code": "Array.from(document.querySelectorAll(\"table tr\")).map(tr => Array.from(tr.cells).map(td => td.textContent.trim()).join(\",\")).join(\"\\n\")"
# }')
# echo $RESULT | jq -r '.result' > "$OUTPUT_DIR/table-data.csv"

# ================================================================
# SUMMARY
# ================================================================
echo ""
echo "Capture complete!"
echo "Files created:"
ls -la "$OUTPUT_DIR"/*.txt "$OUTPUT_DIR"/*.json 2>/dev/null || true
templates/form-automation.sh›
#!/bin/bash
# Template: Form Automation Workflow
# Purpose: Fill and submit web forms with validation
# Usage: ./form-automation.sh <form-url>
#
# This template demonstrates the snapshot-interact-verify pattern:
# 1. Navigate to form
# 2. Snapshot to get element refs
# 3. Fill fields using refs
# 4. Submit and verify result
#
# Customize: Update the refs (@e1, @e2, etc.) based on your form's snapshot output

set -euo pipefail

FORM_URL="${1:?Usage: $0 <form-url>}"

echo "Form automation: $FORM_URL"

# Cleanup handler
cleanup() {
  if [ -n "${SESSION_ID:-}" ]; then
    infsh app run agent-browser --function close --session $SESSION_ID --input '{}' 2>/dev/null || true
  fi
}
trap cleanup EXIT

# Step 1: Navigate to form
echo "Opening form..."
RESULT=$(infsh app run agent-browser --function open --session new --input '{
  "url": "'"$FORM_URL"'"
}')
SESSION_ID=$(echo $RESULT | jq -r '.session_id')

# Step 2: Display form structure
echo ""
echo "Form elements:"
echo "---"
echo $RESULT | jq -r '.elements_text'
echo "---"
echo ""

# ================================================================
# DISCOVERY MODE: Shows form structure
# After running once, update the FORM FILL section below with your refs
# then delete or comment out this section
# ================================================================
echo "Discovery mode: Form structure shown above"
echo ""
echo "Next steps:"
echo "  1. Note the refs for your form fields (e.g., @e1 for name, @e2 for email)"
echo "  2. Update the FORM FILL section below"
echo "  3. Set environment variables for form data"
echo "  4. Comment out this discovery section"
echo ""
exit 0

# ================================================================
# FORM FILL: Uncomment and customize after discovery
# ================================================================
# echo "Filling form..."
#
# # Text input
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e1", "text": "'"${FORM_NAME:-John Doe}"'"
# }'
#
# # Email input
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e2", "text": "'"${FORM_EMAIL:[email protected]}"'"
# }'
#
# # Dropdown/select
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "select", "ref": "@e3", "text": "Option 1"
# }'
#
# # Checkbox
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "check", "ref": "@e4"
# }'
#
# # Textarea
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "fill", "ref": "@e5", "text": "'"${FORM_MESSAGE:-Hello, this is a test message.}"'"
# }'
#
# # Submit button
# echo "Submitting form..."
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "click", "ref": "@e6"
# }'
#
# # Wait for submission
# infsh app run agent-browser --function interact --session $SESSION_ID --input '{
#   "action": "wait", "wait_ms": 2000
# }'
#
# # Step 3: Verify result
# echo ""
# echo "Verifying submission..."
# RESULT=$(infsh app run agent-browser --function snapshot --session $SESSION_ID --input '{}')
#
# URL=$(echo $RESULT | jq -r '.url')
# TITLE=$(echo $RESULT | jq -r '.title')
# echo "Final URL: $URL"
# echo "Page title: $TITLE"
#
# # Check for success indicators
# ELEMENTS=$(echo $RESULT | jq -r '.elements_text')
# if echo "$ELEMENTS" | grep -qi "thank you\|success\|submitted"; then
#   echo "SUCCESS: Form submitted successfully"
# elif echo "$URL" | grep -qi "error\|fail"; then
#   echo "ERROR: Form submission may have failed"
#   exit 1
# else
#   echo "UNKNOWN: Check the result manually"
# fi
#
# # Optional: Capture evidence
# infsh app run agent-browser --function screenshot --session $SESSION_ID --input '{
#   "full_page": true
# }' > form-result-screenshot.json
# echo "Screenshot saved to form-result-screenshot.json"

echo "Done"