Deep Web Research

Unified Research Workflow

Phase 1: Discovery

Goal: Find relevant sources across academic and web domains.

Academic Search (Semantic Scholar):

1. autocomplete_query("topic") → refine terms
2. search_papers(query, year=YYYY, limit=30, fields=[title, authors, year, abstract, citationCount, url])
3. get_paper(paper_id, fields=[...references, citations]) → for key papers

Web Search (Tavily):

1. tavily_search(query, search_depth="advanced", include_raw_content=true, max_results=20)
2. tavily_extract(urls=[...], extract_depth="advanced") → full content from top results

Evidence Quality Hierarchy (prioritize sources):

• Tier 1: Peer-reviewed journals, systematic reviews, official standards (ISO, IEEE, W3C)
• Tier 2: Conference proceedings, technical documentation, industry reports
• Tier 3: Expert blog posts, case studies, reputable news, Stack Overflow
• Tier 4: Social media, marketing materials, unverified claims

Cross-Verification: 2+ independent sources for critical claims. Check original sources, not secondary citations.

Output: Candidate source list with quality tier annotations.

Phase 2: Acquisition

For each valuable source, acquire metadata and archive content.

2a. Metadata Extraction

Auto-detect source type and extract accordingly:

BibTeX generation goes to notes/references.bib. Citekey format: @lastnameYEAR (e.g., @hong2023).

2b. arXiv Protocol (MANDATORY for arXiv papers)

# 1. Extract metadata
tavily_extract(["https://arxiv.org/abs/[ID]"], extract_depth="advanced")

# 2. Download PDF
curl -L -o "notes/literature/Author-Year-Title.pdf" "https://arxiv.org/pdf/[ID].pdf"

# 3. Read FULL PDF (not just abstract)
pdftotext notes/literature/Author-Year-Title.pdf -

# 4. Verify download
file notes/literature/Author-Year-Title.pdf

Why full PDF: Abstracts miss 80%+ of value (methodology, limitations, figures, ablations).

2c. Bookmark Creation (MANDATORY for every source)

Every source gets a bookmark via mcp_raindrop_create_bookmarks.

Auto-categorize using collection matcher — see bookmark-collections.md.

Bookmark format:

{
  "link": "URL",
  "title": "Paper Title - Author Year",
  "excerpt": "2-3 sentence summary (MAX 250 chars for Raindrop)",
  "note": "## Key Insights\n- ...\n## Key Extracts\n> \"...\"\n## Connections\n- ...",
  "collection": COLLECTION_ID
}

Storage Strategy:

• Raindrop excerpt (~250 chars) → quick reference
• Raindrop note field → structured insights and extracts
• Zettelkasten note → comprehensive analysis (unlimited)
• PDF download → full source preservation

Phase 3: Processing

Read and analyze each source using evidence-based methods.

Reading Method: Three-Pass (Keshav 2007)

Five Cs Assessment (always do on first pass):

• Category: Measurement / Analysis / Survey / Implementation / Theory
• Context: How it relates to other work
• Correctness: Are assumptions valid?
• Contributions: Main contributions
• Clarity: Writing quality

AIC Content Abstraction (Pacheco-Vega):

• Abstract: Key claims
• Introduction: Problem, contribution, research questions
• Conclusion: Findings, limitations, future work

For detailed reading methodology, see reading-methods.md.

Source Zettel Creation (MANDATORY)

Every reading produces a comprehensive literature note via zk_create_note(note_type="literature").

Use the template at source-zettel-template.md.

Minimum requirements:

• Full citation with DOI/URL
• Verbatim abstract
• 5+ direct quotes with page numbers
• Five Cs assessment
• Methodology documented
• 2+ strengths and 2+ weaknesses
• 2+ assumptions identified
• 2+ hypotheses generated
• 3+ research questions formulated
• Links to existing Zettelkasten notes

Critical Evaluation

Apply proportionate to source importance:

Quick check (all sources): Evidence tier, cross-verification status, bias red flags.

Standard evaluation (key sources): Methodology critique, bias detection, argument structure.

Deep evaluation (critical sources): Full GRADE assessment, statistical validity, logical fallacy check.

For detailed evaluation frameworks, see evidence-hierarchy.md.

Phase 4: Synthesis

Transform information into interconnected knowledge.

4a. Permanent Notes

Extract atomic concepts from literature notes into permanent notes:

zk_create_note(
  title="Declarative Concept Title",    # e.g., "Role Specialization Improves Multi-Agent Task Decomposition"
  content="[Concept in own words + evidence from multiple sources]",
  note_type="permanent",
  tags="concept-type, domain, application"
)

Quality gate: Single idea, own words, self-contained, declarative title, 3-7 tags, linked to 2+ notes.

4b. Semantic Linking

Connect notes using typed relationships:

Always use bidirectional=true. Always include description explaining WHY notes connect.

4c. Structure Notes

Create Maps of Content (MOCs) when 7+ notes cluster around a theme:

zk_create_note(
  title="MOC: Topic Name",
  note_type="structure",
  content="## Core Concepts\n- [[id1]]: Concept A\n- [[id2]]: Concept B\n..."
)

4d. Triple Cross-Reference

Every source maintains three linked records:

Bookmark (Raindrop) ⟷ BibTeX (@citekey in references.bib) ⟷ Literature Note (Zettelkasten)

Phase 5: Expansion

Grow the knowledge graph through citation mining and gap identification.

Citation Mining:

get_paper_references(paper_id) → foundational work (backward)
get_paper_citations(paper_id)  → subsequent developments (forward)

Knowledge Discovery:

zk_find_similar_notes(note_id, threshold=0.3)  → semantic neighbors
zk_find_central_notes(limit=10)                → knowledge hubs
zk_find_orphaned_notes()                       → unconnected notes to link or delete

Gap Analysis: Identify unanswered questions, contradictions, and areas needing deeper investigation. Document in structure notes.

Research Report Template

# Research Report: [Topic]

## Executive Summary

[2-3 paragraphs: key findings and recommendations]

## Research Question(s)

## Methodology (sources, date range, quality threshold)

## Key Findings (with evidence, cross-verification, confidence)

## Alternative Approaches Considered

## Recommended Approach (with rationale)

## Zettelkasten Integration (notes created, links established)

## Bookmarks (collection URL, total sources)

## References (academic, professional, documentation)

## Gaps and Future Research

Workflow Patterns

Pattern 1: Single Paper Processing

Discovery → Get metadata → Download PDF (if arXiv) → Bookmark →
First pass (Five Cs) → Second pass (excerpts) → Third pass (if important) →
Source zettel → BibTeX entry → Link to existing notes

Pattern 2: Literature Survey

Seed search → Identify 3-5 quality papers → Citation mining →
Batch first-pass all candidates → Filter → Second-pass selected →
Third-pass key papers → Create MOC → Map themes and gaps

Pattern 3: Technology Evaluation

Define requirements → Academic search → Web search for implementations →
Bookmark all → Extract features → Create comparison notes →
Link: supports/contradicts → Recommend with evidence

Pattern 4: Quick Assessment

Fast first pass (5 min) → Five Cs → Targeted read (10-20 min) →
Decision (deep dive / bookmark / discard) → Fleeting note if keeping

Quality Checklist

Evidence Quality

• All claims backed by 2+ sources for critical claims
• Sources assigned quality tier (1-4)
• Contradicting evidence acknowledged
• Confidence level stated per finding

Documentation Completeness

• All sources bookmarked with rich notes
• Academic papers have literature notes in Zettelkasten
• PDFs archived to notes/literature/ with standardized names
• BibTeX entries in notes/references.bib
• Key concepts extracted to permanent notes
• Notes linked with semantic relationships

Synthesis Quality

• Findings in own words (not copy-paste)
• Atomic concepts properly separated
• Structure notes created for mature topics
• Research questions documented for future work

Common Pitfalls

Integration with Other Skills

• scientific-brainstorming: Question → this skill for research → Insight → Hypothesis
• scientific-critical-thinking: Use for deep methodological critique beyond standard evaluation
• zettelkasten-management: Core knowledge operations; this skill is the primary research producer

Integration with Deep Research Agents

This skill is the knowledge foundation for the Deep Research Orchestrator agent system. Agents that load this skill:

Orchestrator: deep-research-orchestrator coordinates all agents.

Quick Start: See docs/deep-research-quick-start.md for usage guide.

Reference Files

Templates