Neb Test Agent

Construct the base URL at the start of execution:

source "$(git rev-parse --show-toplevel)/.env"
AI_API="${NEB_TASK_API_URL}/api/v1/orgs/${NEB_TASK_COMPANY_ID}/ai"
AUTH_HEADER="Authorization: Bearer ${NEB_TASK_API_KEY}"

Then all API calls use ${AI_API} as the base:

• ${AI_API}/agents/me → proxied to Paperclip /api/agents/me
• ${AI_API}/companies/${NEB_TASK_COMPANY_ID}/issues → proxied to /api/companies/{id}/issues
• ${AI_API}/companies/${NEB_TASK_COMPANY_ID}/agents → proxied to /api/companies/{id}/agents
• ${AI_API}/health → proxied to /api/health

The JWT (issuer: paperclip) is passed through by platform-backend without re-validation — Paperclip validates it on the upstream side.

Test Suites

Suite 1: Filesystem Access (8 checks)

The agent must:

1. Read CLAUDE.md from the main nebinfra repo (~/sources/nebinfra/CLAUDE.md)
2. Read a file from oss/ directory (~/oss/paperclip/package.json)
3. Read a file from a nebcore submodule (nebcore/helm-library/charts/k8s/Chart.yaml)
4. Read a file from a business submodule (business/business-operations/)
5. Read a file from a gitops submodule (gitops/gitops-platform-shared/)
6. List files in the agent's own workspace
7. Create a temp file (/tmp/agent-test-{timestamp}.txt) and verify it exists
8. Read .claude/references/paperclip-api.md

Suite 2: Git Operations (6 checks)

The agent must:

1. Run git status in the main nebinfra repo
2. Run git log --oneline -3 to see recent commits
3. Run git submodule status | wc -l to count submodules (expect 34)
4. Run git branch --show-current to check current branch
5. Run git -C nebcore/helm-library log --oneline -1 to verify submodule access
6. Run git config user.name and git config user.email to verify identity

Suite 3: Tool Access (7 checks)

The agent must:

1. Run helm version --short (expect v4.x)
2. Run kubectl version --client (expect v1.x)
3. Run go version (expect go1.x)
4. Run node --version (expect v2x)
5. Run gh --version (expect gh 2.x)
6. Run python3 --version (expect 3.x)
7. Run pnpm --version (expect 10.x)

Suite 4: Platform API (8 checks)

The agent must (using its platform credentials):

1. Verify own identity: fetch agent details via API
2. Fetch the current task details via heartbeat-context
3. Post a progress comment on the current task
4. List all agents in the company
5. List all issues assigned to self
6. Search issues using the search plugin: POST /api/plugins/nebinfra.platform-integration/data/search
7. Create a test subtask under the current task
8. Update the subtask status to done, then delete it

Suite 5: Helm Operations (5 checks)

Invoke /neb-helm-validate for helm testing scenarios.

The agent must:

1. Invoke /neb-helm-validate on a chart directory (e.g., nebcore/helm-modules-apps/charts/argo-cd/) and verify it passes
2. Verify /neb-helm-validate output includes lint and dependency update results
3. Read helm-charts-releases.yaml and find a chart version
4. Read a Chart.yaml and extract the version
5. Run helm template on a small chart and verify output contains YAML

Suite 6: Compliance (4 checks)

The agent must:

1. Look up entity details for USA (expect "Nebinfra Technologies Inc")
2. Look up entity details for India (expect "Altisnebinfra Technologies Pvt Ltd")
3. Verify employment bonds are refused for USA
4. Verify employment bonds are legal for India

Suite 7: End-to-End Reporting (3 checks)

The agent must:

1. Compile all test results into a structured markdown table
2. Post the full report as a comment on the task
3. Mark the task as done (if all checks pass) or blocked (if any fail)

Execution Flow

Step 1: Source environment and construct API base

source "$(git rev-parse --show-toplevel)/.env"

# Construct AI proxy base URL
# Platform-backend proxies: /api/v1/orgs/{companyId}/ai/* → Paperclip /api/*
AI_API="${NEB_TASK_API_URL}/api/v1/orgs/${NEB_TASK_COMPANY_ID}/ai"
AUTH="-H 'Authorization: Bearer ${NEB_TASK_API_KEY}'"

Step 2: Verify connectivity and resolve agent

First verify the AI server is reachable:

curl -sS "${AI_API}/health" -H "Authorization: Bearer ${NEB_TASK_API_KEY}"

Then look up the target agent by name from the company agents list:

curl -sS "${AI_API}/companies/${NEB_TASK_COMPANY_ID}/agents" \
  -H "Authorization: Bearer ${NEB_TASK_API_KEY}"

Get the agent ID from the matching agent.

Step 3: Create test task

Invoke /superpowers:test-driven-development to define test suites with assertions and expected outcomes.

curl -sS -X POST "${AI_API}/companies/${NEB_TASK_COMPANY_ID}/issues" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${NEB_TASK_API_KEY}" \
  -d '{
    "title": "[Agent Test] <agent-name>: Comprehensive capability verification",
    "description": "<full test spec from selected suites>",
    "status": "todo",
    "priority": "high",
    "assigneeAgentId": "<agent-id>"
  }'

Step 4: Wait for agent heartbeat

Invoke /superpowers:dispatching-parallel-agents when triggering agent heartbeat for test execution.

Poll heartbeat runs until a new run appears for this agent and completes (succeeded or failed). Timeout: 10 minutes.

curl -sS "${AI_API}/companies/${NEB_TASK_COMPANY_ID}/heartbeat-runs?limit=1" \
  -H "Authorization: Bearer ${NEB_TASK_API_KEY}"

Step 5: Review transcript

Fetch the run transcript via:

curl -sS -X POST "${AI_API}/plugins/nebinfra.platform-integration/data/run-transcript" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${NEB_TASK_API_KEY}" \
  -d '{"params": {"runId": "<run-id>", "companyId": "<company-id>"}}'

Step 6: Review task comments

Fetch the agent's posted comments:

curl -sS "${AI_API}/issues/<issue-id>/comments" \
  -H "Authorization: Bearer ${NEB_TASK_API_KEY}"

Look for the structured test report in the comments.

Step 7: Analyze results

Invoke /superpowers:verification-before-completion to review agent transcript and test output before declaring pass/fail.

Parse the agent's report comment for:

• Total checks: count of all test items
• Passed: checks marked PASS
• Failed: checks marked FAIL
• Skipped: checks not attempted

Print a summary:

=== Agent Test Results: DevOps ===
Suite              | Pass | Fail | Skip
-------------------|------|------|-----
Filesystem         | 8/8  | 0    | 0
Git Operations     | 6/6  | 0    | 0
Tool Access        | 7/7  | 0    | 0
Platform API       | 8/8  | 0    | 0
Helm Operations    | 5/5  | 0    | 0
Compliance         | 4/4  | 0    | 0
Reporting          | 3/3  | 0    | 0
-------------------|------|------|-----
TOTAL              | 41/41| 0    | 0

Step 8: Diagnose failures

For any FAIL results:

1. Fetch the full transcript via get-run-transcript
2. Find the failed step in the transcript
3. Identify the root cause (permission error, missing tool, API failure, etc.)
4. Report the diagnosis

Task Description Template

The task assigned to the agent uses this template:

## Objective
Run the NebInfra agent capability test suite. Execute each check, record PASS/FAIL, and post a structured report.

## Instructions
For each check below, attempt the operation and record the result as PASS or FAIL with a brief note.

### Filesystem Access
1. Read ~/sources/nebinfra/CLAUDE.md — first 3 lines
2. Read ~/oss/paperclip/package.json — extract "name" field
3. Read nebcore/helm-library/charts/k8s/Chart.yaml — extract version
4. List files in business/business-operations/ (first 5)
5. List files in gitops/gitops-platform-shared/ (first 5)
6. List files in your workspace root
7. Create /tmp/agent-test-{timestamp}.txt and verify
8. Read .claude/references/paperclip-api.md — first 3 lines

### Git Operations
1. git status (in nebinfra root)
2. git log --oneline -3
3. git submodule status | wc -l (expect ~34)
4. git branch --show-current
5. git -C nebcore/helm-library log --oneline -1
6. git config user.name && git config user.email

### Tool Access
1. helm version --short
2. kubectl version --client --short
3. go version
4. node --version
5. gh --version
6. python3 --version
7. pnpm --version

### Platform API
1. Fetch your own agent details via API
2. Fetch this task's heartbeat-context
3. Post a "test in progress" comment on this task
4. List all company agents
5. List issues assigned to you
6. Search issues for "test" via search plugin
7. Create a subtask "Test subtask" under this task
8. Mark the subtask done and delete it

### Helm Operations
1. Invoke /neb-helm-validate on nebcore/helm-modules-apps/charts/argo-cd and verify it passes
2. Verify /neb-helm-validate output includes lint and dependency update results
3. Read helm-charts-releases.yaml — find argo-cd version
4. Read nebcore/helm-modules-apps/charts/argo-cd/Chart.yaml — extract version
5. helm template test nebcore/helm-modules-apps/charts/argo-cd | head -20

### Compliance
1. Entity lookup: USA (expect "Nebinfra Technologies Inc")
2. Entity lookup: India (expect "Altisnebinfra Technologies Pvt Ltd")
3. Employment bond check: USA document with bond clause (expect VIOLATION)
4. Employment bond check: India document with bond clause (expect PASS)

### Reporting
Post a comment with this format:

| Suite | Check | Result | Notes |
|---|---|---|---|
| Filesystem | Read CLAUDE.md | PASS/FAIL | first line content or error |
| ... | ... | ... | ... |

Then set this task to done (if all pass) or blocked (if any fail).

Suite 0: API Connectivity (pre-flight, 5 checks)

Before running agent capability tests, verify the operator can reach the platform:

source "$(git rev-parse --show-toplevel)/.env"
AI_API="${NEB_TASK_API_URL}/api/v1/orgs/${NEB_TASK_COMPANY_ID}/ai"
AUTH="Authorization: Bearer ${NEB_TASK_API_KEY}"

1. GET ${NEB_TASK_API_URL}/healthz — platform-backend liveness (no auth)
2. GET ${AI_API}/health — AI server health via proxy (JWT auth)
3. GET ${AI_API}/agents/me — agent identity (JWT auth)
4. GET ${AI_API}/companies/${NEB_TASK_COMPANY_ID}/agents — list agents
5. GET ${AI_API}/companies/${NEB_TASK_COMPANY_ID}/dashboard — company health

If any fail, STOP and diagnose. Common issues:

• 401 on /api/health: You're hitting platform-backend directly, not the AI proxy
• 500 on writes: Known Paperclip post-mutation lifecycle bug (see Known Issues)
• "PAPERCLIP_RUN_ID not set": Process adapter missing env var (see Known Issues)

Known Issues (2026-04-04)

Reference: .claude/memory/2026-04-04-ai-agent-e2e-findings.md

Workaround for 500-on-writes: Despite the 500, data IS persisted. You can verify via GET after a failed POST/PATCH. For E2E testing, treat 500 as "write succeeded with lifecycle error" and verify via read.

Workaround for ownership conflict: Avoid using checkout when running from operator JWT. Instead, use PATCH to change status directly (also returns 500 but persists).

Superpowers Integration