Phase 3 of deep research - executes searches using web-search-agent with strategy modules. Supports parallel batch execution and resume.
Execute searches to gather data for each entity, dimension, or angle.
state.json from the output directoryCheck data/ directory for existing JSON files. Skip any entity/angle that already has a completed JSON file.
All searches use these tools (zero cost, no API keys):
mcp__searxng__searxng_web_search — primary web searchmcp__searxng__web_url_read — read URL contentmcp__chrome__new_page + mcp__chrome__take_snapshot — fallback URL readinggh CLI via Bash — GitHub repository dataweb_url_read| Source | Failure | Action |
|---|---|---|
| SearXNG | Down/no results | Retry once, then warn and continue |
gh CLI | Auth error/not found | Log error, mark dimension incomplete |
| HN Algolia | Rate limit/timeout | Skip, use SearXNG Reddit/X instead |
| Chrome MCP | Cannot render | Warn, use SearXNG snippet only |
No single failure blocks the pipeline.
Parse the entity list and field definitions.
Group entities by batch_size from outline.yaml.
For each batch:
--auto, present the batch to the user via AskUserQuestion for approvalRead the web search agent instructions at ~/.claude/skills/research/agents/web-search-agent.md and follow them.
Your research topic is: {entity_name}
Your output path is: {output_dir}/data/{entity_slug}.json
Load these strategy modules before searching:
- ~/.claude/skills/research/agents/modules/general-web.md
{if entity is a GitHub project:}
- ~/.claude/skills/research/agents/modules/github-repo.md
Here are the fields you need to populate (from fields.yaml):
{paste fields.yaml content}
The current date is: {date from state.json}
List completed entities, failed entities, and total coverage.
Parse the dimension list and repo info.
For each dimension, research sequentially:
project_health (source: gh_cli):
gh repo view {repo} --json name,description,stargazerCount,forkCount,pushedAt,createdAt,licenseInfo,primaryLanguage
gh api repos/{repo}/commits --jq '.[].commit.committer.date' | head -10
gh api repos/{repo}/contributors --jq 'length'
gh api repos/{repo}/releases --jq '.[0:3] | .[] | {tag: .tag_name, date: .published_at}'
code_quality (source: clone_analysis):
community_sentiment (source: web_search):
alternatives (source: web_search):
integration_risk (source: gh_cli):
Write each dimension's results to data/{dimension_name}.json.
Parse the angle list.
For each angle, spawn an Agent with this prompt:
Read the web search agent instructions at ~/.claude/skills/research/agents/web-search-agent.md and follow them.
Your research angle is: {angle.query}
Focus area: {angle.focus}
Your output path is: {output_dir}/data/{angle_slug}.json
Load these strategy modules:
- ~/.claude/skills/research/agents/modules/general-web.md
{if focus is "academic":}
- ~/.claude/skills/research/agents/modules/academic.md
{if query contains Chinese:}
- ~/.claude/skills/research/agents/modules/chinese-tech.md
No fields.yaml — write free-form research findings.
The current date is: {date}
Run all agents in parallel.
List completed angles and any failures.