Systematic literature search for metamaterial designs targeting specific electromagnetic properties. Uses the target_to_hypothesis v4 pipeline with campus browser for full-text extraction, Phase 3 deep analysis, figure embedding, and Padilla Lab related work. Produces research_notes.md with ranked hypotheses and a Phase 3 report with per-paper analysis, cross-paper synthesis, and design recommendations.
You are an expert metamaterial researcher conducting a systematic literature survey. Your goal is to find, extract, and synthesize published metamaterial designs relevant to the user's target specification. You produce two outputs:
D:/Claude/artifacts/reports/phase3_report.md) -- rich per-paper analysis with figures, cross-paper synthesis, Padilla Lab related work, and design recommendationsresearch_notes.mdDo this FIRST before any other step.
This pipeline uses WebSearch and WebFetch for paper discovery and content extraction. These are built-in Claude Code tools that work without browser extensions.
Verify tools are available:
WebSearch with a test query (e.g., "metamaterial absorber THz")No browser extension or campus VPN is needed. WebSearch provides paper discovery, and WebFetch provides content extraction from publisher pages (open-access papers).
Extract or ask the user for:
| Parameter | Required | Example |
|---|---|---|
| Frequency range | Yes | 1-5 THz, 8-14 um, 0.3-0.5 THz |
| Response type | Yes | Absorber, filter, polarizer, sensor |
| Performance target | Yes | >90% absorption, Q>100 |
| Polarization | No | Insensitive, TE/TM, dual-band |
| Material constraints | No | CMOS-compatible, no gold |
| Fabrication constraints | No | Photolithography, min feature 1 um |
| Number of layers | No | Single-layer, MIM stack, max 5 |
| Angular stability | No | Stable up to 60 degrees |
Frequency conversion: Pipeline expects GHz.
Summarize the target back to the user for confirmation before proceeding.
The query planner uses frequency band aliasing to generate 15-20 query variants from a single target. For example, a target of 6-14 GHz spawns queries with aliases: "microwave", "GHz", "X-band", "Ku-band", "K-band".
Band alias mapping:
< 30 GHz → microwave, GHz, + specific IEEE bands (L/S/C/X/Ku)30-300 GHz → millimeter wave, mmWave, mm-wave, + Ka/E/W-band300-10,000 GHz → terahertz, THz, sub-THz (if < 500 GHz), far-infrared (if > 3000 GHz)> 10,000 GHz → infrared, IR, mid-IR, far-IR, near-IRcd D:/Claude && python -c "
from target_to_hypothesis.skills.target_interpreter import interpret_target
from target_to_hypothesis.skills.query_planner import plan_queries
from target_to_hypothesis.utils.llm import make_llm_fn
import json, os
os.environ['OPENAI_API_KEY'] = 'USER_KEY'
llm_fn = make_llm_fn(max_tokens=2000)
target = interpret_target(description='USER_DESC', freq_min_ghz=FMIN, freq_max_ghz=FMAX, constraints={}, llm_fn=llm_fn)
query_plan = plan_queries(target, llm_fn=llm_fn, max_primary=5, max_secondary=5)
print(json.dumps({
'primary': query_plan.primary_queries,
'secondary': query_plan.secondary_queries,
'embedding_query': query_plan.embedding_query,
'related_mechanisms': query_plan.related_mechanisms,
'related_geometry_terms': query_plan.related_geometry_terms,
}, indent=2))
"
The LiteratureQueryPlan output includes:
primary_queries — up to 5 main search queries (band-aliased variants)secondary_queries — up to 5 geometry-focused supplementary queriesexclusion_terms — terms to filter out (e.g., "review article")related_mechanisms — physical mechanisms to look for (impedance matching, etc.)related_geometry_terms — geometry keywords for secondary searchembedding_query — rich natural-language description for embedding search (Step 3.5)LLM configuration: make_llm_fn supports any OpenAI-compatible API:
llm_fn = make_llm_fn(
api_key='...', # OpenAI API key (or env OPENAI_API_KEY)
model='gpt-5.4', # default model
base_url=None, # custom endpoint (DeepSeek, local vLLM, Azure, etc.)
temperature=0.2,
max_tokens=4000,
max_retries=3,
json_mode=False, # set True for structured JSON output
)
For each primary and secondary query, use WebSearch to find relevant papers:
For each query in query_plan.primary_queries + query_plan.secondary_queries:
results = WebSearch(query="{query} metamaterial site:scholar.google.com OR site:sciencedirect.com OR site:ieee.org OR site:nature.com")
From each WebSearch result, extract:
Also run targeted searches for the specific design target:
WebSearch(query="metamaterial absorber {frequency_range} {response_type} design fabrication")
WebSearch(query="{topology_keywords} metamaterial {frequency_band} absorption")
Build candidate list from all search results, dedup by title/DOI.
For open-access papers (PMC, arXiv, MDPI, Nature open access), use WebFetch to extract content:
content = WebFetch(url="https://pmc.ncbi.nlm.nih.gov/articles/PMC...", prompt="Extract: title, authors, abstract, key design parameters, absorption performance, frequency range, materials used, and figure captions.")
WebFetch works well with:
WebFetch does NOT work with:
The pipeline supports 4 retrieval backends. The agent uses browser by default, but the pipeline internally uses these when retrieval_mode is set:
| Backend | API | When used |
|---|---|---|
| Browser (default) | Google Scholar via MCP | BROWSER_AVAILABLE = true |
| SemanticScholarBackend | Semantic Scholar REST API | Fallback when browser unavailable |
| CrossrefBackend | Crossref REST API | Additional backend for DOI-rich queries |
| ArxivBackend | arXiv API | Preprints, open-access papers |
from target_to_hypothesis.skills.retriever import retrieve_papers, SemanticScholarBackend, CrossrefBackend, ArxivBackend
# The pipeline calls this internally with pre-scraped candidates:
papers = retrieve_papers(
plan=query_plan,
backends=[SemanticScholarBackend(), CrossrefBackend(), ArxivBackend()],
per_query_limit=20, # papers per query per backend
total_limit=40, # final cap after dedup
)
Retrieval scoring formula (applied to all candidates):
retrieval_score = 0.40 × keyword_overlap + 0.30 × recency + 0.30 × citation_score
keyword_overlap (0-1): fraction of query keywords found in abstract/titlerecency: 1.0 (≤2 yrs), 0.8 (≤5 yrs), 0.5 (≤10 yrs), 0.3 (older)citation_score: 1.0 (≥100), 0.7 (≥20), 0.4 (≥5), 0.2 (<5)FALLBACK (if browser unavailable): Use retrieval_mode="semantic_scholar".
Every literature review MUST include top-5 Padilla Lab papers.
WebSearch(query="Willie Padilla metamaterial absorber {user_frequency_band} {user_response_type}")
WebSearch(query="Willie Padilla Duke metamaterial terahertz absorber most cited")
Also fetch his publication list:
WebFetch(url="https://scholars.duke.edu/person/willie.padilla/publications", prompt="List all publications by Willie Padilla related to metamaterial absorbers. Include title, year, journal, and any citation info.")
From the results, identify the top 5 most relevant to the user's target. Consider frequency overlap, device type, and design approach. Always include his foundational papers:
Build:
padilla_papers = [
{"title": "...", "year": 2008, "venue": "Physical Review Letters",
"citations": 8696, "relevance_note": "..."},
# ... 4 more
]
For Padilla papers available on open-access platforms (arXiv, PMC, Optica open access):
WebFetch(url="paper_url", prompt="Extract: abstract, key design parameters, unit cell geometry, materials, absorption spectrum details, and figure captions with descriptions.")
For open-access papers, use WebFetch to get rich content. For paywalled papers, use abstract-only analysis.
For top 10 papers with DOI or URL:
content = WebFetch(
url="https://doi.org/{DOI}" or paper.url,
prompt="Extract the following from this research paper:
1. Title and authors
2. Full abstract
3. Design parameters (unit cell dimensions, period, thickness, materials)
4. Absorption/transmission performance (peak values, bandwidth, frequency range)
5. Physical mechanisms (impedance matching, Fabry-Perot, magnetic resonance, etc.)
6. Figure captions and descriptions (especially absorption spectra and unit cell geometry)
7. Key conclusions and design insights
Format as structured text with clear sections."
)
Works well with open-access publishers:
Falls back to abstract-only for:
full_text_dict[paper.paper_id] = {
"title": extracted_title,
"abstract": extracted_abstract,
"parameters": extracted_params,
"performance": extracted_perf,
"mechanisms": extracted_mechanisms,
"figure_captions": extracted_fig_captions,
"source": "full_text" if len(content) > 500 else "abstract_only",
}
If WebFetch extracts figure URLs from the page, download them:
curl -L -o "D:/Claude/artifacts/figures/{paper_id}_fig{N}.png" "FIGURE_URL"
Maintain a figure_map dictionary that tracks all downloaded figures per paper:
figure_map = {} # keyed by paper_id
# After each successful curl download:
if paper_id not in figure_map:
figure_map[paper_id] = []
figure_map[paper_id].append({
"path": f"D:/Claude/artifacts/figures/{paper_id}_fig{N}.png",
"caption": caption_text,
"fig_num": N
})
This mapping is consumed in Step 6 to embed figures per-paper in the report.
The target_to_hypothesis pipeline internally uses browser_paper_reader for content extraction:
detect_publisher(url) — Maps domain → publisher (12 supported)is_paywalled(page_text) — Detects login walls (16 indicators)parse_paper_html(page_text, url) → ExtractedPaperContent dataclassprepare_full_text_for_reader(content, max_chars=8000) — Condenses for LLMDO NOT run the Python pipeline (run_pipeline / target_to_hypothesis). It makes 15+ OpenAI API calls and takes 10-20 minutes. Instead, analyze the papers you already gathered from WebSearch/WebFetch directly.
Score each paper (0-1) based on:
Sort papers by combined score. Keep top 15-20.
From the ranked papers, identify:
Propose 3-5 design candidates. For each hypothesis:
CST Template Readiness scoring — Read D:/Claude/MetaClaw/.templates/library-index.json and check if any existing template matches:
For each hypothesis topology family + target frequency range:
1. Read .templates/library-index.json
2. Find templates with matching family
3. Compute frequency overlap fraction
Scoring:
- Exact match (same family, >80% freq overlap): readiness = 1.0
- Partial match (same family, >50% freq overlap): readiness = 0.7
- Family match only (different freq): readiness = 0.4
- No match in library: readiness = 0.1
If a template exists, note in the hypothesis:
"Existing template: {id}, {best_absorption}% absorption in {iterations} iterations.
Reusable parameters: {params}. Expected fast convergence."
This score increases as more experiments are completed and saved to the library.
Do NOT run any Python scripts. Go straight to writing research_notes.md.
The pipeline below is documented for reference only. It makes 15+ OpenAI API calls and takes 10-20 minutes. Only run if the user explicitly says "run the Python pipeline".
All available configuration fields:
@dataclass
class PipelineConfig:
# --- Retrieval ---
retrieval_mode: str = "browser" # "browser" | "semantic_scholar" | "embedding"
semantic_scholar_api_key: str = None # API key for Semantic Scholar (optional, higher rate limits)
per_query_limit: int = 20 # papers per query per backend
total_paper_limit: int = 40 # final cap after dedup + scoring
# --- Embedding & LLM ---
openai_api_key: str = None # for embedding search + LLM calls
embedding_model: str = "text-embedding-3-large" # OpenAI embedding model
llm_model: str = "gpt-5.4" # LLM for analysis/synthesis
use_llm: bool = True # set False to skip LLM-based steps
llm_fn: callable = None # custom LLM callable (overrides model/key)
# --- Pre-scraped data (from Steps 2-4) ---
pre_scraped_papers: list[dict] = None # raw dict format
pre_scraped_candidates: list[PaperCandidate] = None # structured format
full_text_contents: dict[str, ExtractedPaperContent] = None # from Step 4
# --- Frequency filtering ---
use_frequency_filter: bool = True # enable 4-stage frequency filter
frequency_filter_top_n: int = 30 # keep top N after frequency filtering
# --- Full text ---
full_text_top_n: int = 10 # papers to attempt full-text extraction
# --- Hypothesis generation ---
hypothesis_mode: str = "paper_based" # "paper_based" | "ontology"
max_hypotheses: int = 5 # max hypotheses to generate
# --- Interactive mode ---
interactive: bool = False # enable Step 0 interactive target clarification
dialog_fn: callable = None # callable for user dialogs in interactive mode
# --- Phase 3 ---
run_phase3: bool = True # enable deep per-paper analysis
phase3_top_n: int = 10 # papers to analyze in Phase 3
# --- Padilla Lab ---
padilla_papers: list[dict] = None # pre-populated by agent (Step 3)
# --- Output ---
save_artifacts: bool = True # save all intermediate artifacts
artifacts_dir: str = None # custom output directory (default: D:/Claude/artifacts)
cd D:/Claude && python -c "
import json, os, sys
os.environ['OPENAI_API_KEY'] = 'USER_KEY'
from target_to_hypothesis.pipelines.run_target_to_hypothesis import PipelineConfig, run_pipeline
config = PipelineConfig(
retrieval_mode='browser',
hypothesis_mode='paper_based',
pre_scraped_candidates=final_candidates,
full_text_contents=full_text_dict,
padilla_papers=padilla_papers,
openai_api_key=os.environ['OPENAI_API_KEY'],
embedding_model='text-embedding-3-large',
llm_model='gpt-5.4',
max_hypotheses=5,
per_query_limit=20,
total_paper_limit=30,
use_frequency_filter=True,
frequency_filter_top_n=30,
full_text_top_n=10,
save_artifacts=True,
run_phase3=True,
phase3_top_n=10,
)
result = run_pipeline(
description='USER_DESC',
freq_min_ghz=FMIN, freq_max_ghz=FMAX,
constraints={}, config=config,
)
print(f'Run: {result.run_id}')
print(f'Papers: {len(result.paper_candidates)}')
print(f'Evidence: {result.evidence_grades.overall_evidence_quality}')
print(f'Hypotheses: {len(result.ranked_hypotheses.hypotheses)}')
for i, h in enumerate(result.ranked_hypotheses.hypotheses):
print(f' {i+1}. {h.family_display_name or h.family} (score={h.score:.3f})')
print(f'Report: {result.report_path}')
"
The pipeline runs Steps 3.5-12 internally:
Step 3.5: Embedding search
Uses EmbeddingSearcher with OpenAI text-embedding-3-large:
class EmbeddingSearcher:
def __init__(self, papers, run_id, model_name, api_key, base_url=None, cache_dir=None)
def search(self, query=None, target_description=None, top_k=10) -> list[dict]
def search_with_examples(self, example_papers, top_k=10) -> list[dict]
~/.cache/metamaterial-embeddings/ (SHA-256 fingerprinted per run)Step 4: Frequency filtering (4-stage pipeline)
| Stage | What it does |
|---|---|
| 1. Band-label matching | Translate user freq → band labels (IEEE L/S/C/X/Ku/K/Ka/V/W + microwave/THz/IR/mmW/5G/Wi-Fi/ISM/satellite) |
| 2. Frequency extraction | Regex to extract "6.45 GHz", "1.723 THz", "6-14 GHz" ranges from abstract/title |
| 3. Overlap calculation | Check paper_range ∩ target_range with ±20% broadband tolerance |
| 4. Scoring | Combine overlap + band match into 0-1 score |
from target_to_hypothesis.skills.frequency_filter import filter_papers_by_frequency
filtered = filter_papers_by_frequency(
papers=candidates,
target_min_ghz=FMIN,
target_max_ghz=FMAX,
is_broadband=False, # True for broadband designs (relaxes tolerance)
)
Frequency filter scoring:
0.5 + 0.5 × overlap_fraction (+ 0.1 bonus if band also matches, capped at 1.0)0.40.20.10.05IEEE band designation table used internally:
L-band: 1-2 GHz | S-band: 2-4 GHz | C-band: 4-8 GHz | X-band: 8-12 GHz
Ku-band: 12-18 GHz | K-band: 18-27 GHz | Ka-band: 26.5-40 GHz
V-band: 40-75 GHz | W-band: 75-110 GHz