Test Automation

User says something like: "Verify the login form accepts valid credentials."

Break it down:

• Setup: Navigate to the login page
• Actions: Fill email, fill password, click submit
• Assertions: Verify redirect to dashboard, verify welcome message visible

2. Setup (Navigate)

browser(action="navigate", url="<target-url>")

Wait for page load if needed:

browser(action="wait_for", selector="<ready-indicator>")

3. Actions (Interact)

Use self-healing selectors (see references/selector-strategies.md for the full fallback chain).

Prefer semantic selectors over brittle CSS:

browser(action="type", selector="input[name='email']", text="[email protected]")
browser(action="type", selector="input[name='password']", text="secret")
browser(action="click", selector="button:has-text('Sign in')")

If a selector fails, try alternatives in this order:

1. [name] or [id] attribute
2. [role] + accessible name: [role="button"][aria-label="Submit"]
3. Text content: button:has-text("Submit")
4. Placeholder: input[placeholder="Email address"]
5. Label association: input linked by <label>

4. Assertions (Verify)

Read page state and compare against expected values:

browser(action="get_text", selector="h1.welcome")

Expected: contains "Welcome back". If mismatch, the test fails.

For URL checks:

browser(action="evaluate", expression="window.location.pathname")

Expected: /dashboard.

For element existence:

browser(action="get_elements", selector=".success-banner")

Expected: at least one element returned.

5. Screenshot on Failure

On any assertion failure, capture evidence:

browser(action="screenshot")

6. Report Result

Output a structured result per test:

## Test: <test name>
- **Status:** PASS | FAIL
- **URL:** <tested url>
- **Steps completed:** <n>/<total>
- **Failed step:** <description> (if FAIL)
- **Expected:** <expected value>
- **Actual:** <actual value>
- **Screenshot:** <attached if FAIL>

Visual Regression

Detect layout and style drift by comparing screenshots across runs using vision.

1. Capture Baseline

Navigate to the target page and take a screenshot:

browser(action="navigate", url="<target-url>")
browser(action="wait_for", selector="<ready-indicator>")
browser(action="screenshot")

Store baseline metadata in KB:

kb(action="store", topic="projects", title="test baseline: <page-name>", content="URL: <url>\nCaptured: <date>\nViewport: <width>x<height>\nDescription: <what the page should look like>", tags=["test-baseline", "<project>"])

2. Run Regression Check

Navigate to the same page and screenshot again:

browser(action="navigate", url="<target-url>")
browser(action="screenshot")

3. Compare via Vision

Use the current screenshot and the baseline description to assess differences. Compare:

• Layout: Are elements in the same positions? Any overlapping or misaligned items?
• Content: Is text the same? Any missing or extra elements?
• Styling: Are colors, fonts, spacing consistent?

Report differences with specifics:

• "The navigation bar has shifted 20px down"
• "The CTA button color changed from blue to green"
• "A new banner element appeared above the hero section"

4. Update Baseline

If changes are intentional, update the KB entry:

kb(action="search", query="test baseline: <page-name>", topic="projects")
kb(action="store", topic="projects", title="test baseline: <page-name>", content="<updated metadata>", tags=["test-baseline", "<project>"])

Test Suite Management

Tests are stored as markdown files in ~/.picobot/workspace/tests/. Each file defines one or more test cases.

Test File Format

---