LangExtract Skill

Claude's environment can only reach specific domains. Gemini API (generativelanguage.googleapis.com) and OpenAI API (api.openai.com) may not be reachable. If extraction fails with network errors, inform the user the LLM endpoint is unreachable from this environment and suggest they run the generated script locally.

Core Workflow

Every LangExtract task follows three steps:

1. Define the Extraction Task

Write a prompt_description string and one or more ExampleData objects.

The prompt tells the LLM what to extract. The examples show how — they are the schema definition via demonstration. This is the most important part; extraction quality depends almost entirely on prompt + example quality.

import langextract as lx
import textwrap

prompt = textwrap.dedent("""\
    Extract [entity types] in order of appearance.
    Use exact text for extractions. Do not paraphrase or overlap entities.
    Provide meaningful attributes for each entity to add context.""")

examples = [
    lx.data.ExampleData(
        text="[representative sample text]",
        extractions=[
            lx.data.Extraction(
                extraction_class="[entity_type]",
                extraction_text="[verbatim text from example]",
                attributes={"key": "value"}
            ),
            # ... more extractions, in order of appearance in text
        ]
    )
]

Critical rules for examples:

• extraction_text must be verbatim from the example text — no paraphrasing
• List extractions in order of appearance in the text
• Include at least one example per entity type you want extracted
• Attributes define what metadata to capture per entity — the LLM will mirror this structure
• Multiple examples improve quality; one good example is the minimum

2. Run Extraction

result = lx.extract(
    text_or_documents=input_text,    # str, list[str], URL, or list[Document]
    prompt_description=prompt,
    examples=examples,
    model_id="gemini-2.5-flash",     # Default; see Model Selection below
)

lx.extract() — Full Parameter Reference:

3. Save and Visualize Results

# Access extractions programmatically
for ext in result.extractions:
    print(f"{ext.extraction_class}: '{ext.extraction_text}' — {ext.attributes}")

# Save to JSONL
lx.io.save_annotated_documents(
    [result],
    output_name="results.jsonl",
    output_dir="."
)

# Generate interactive HTML visualization
html_content = lx.visualize("results.jsonl")
with open("visualization.html", "w") as f:
    if hasattr(html_content, 'data'):
        f.write(html_content.data)
    else:
        f.write(html_content)

The result object is an AnnotatedDocument with fields:

• extractions: list of Extraction objects (each has extraction_class, extraction_text, attributes, char_interval)
• text: the original input text
• document_id: auto-generated ID

Model Selection

Tuning for Long Documents

For documents over ~5,000 characters, adjust these parameters:

result = lx.extract(
    text_or_documents=long_text_or_url,
    prompt_description=prompt,
    examples=examples,
    model_id="gemini-2.5-flash",
    extraction_passes=3,       # Multiple passes improve recall
    max_workers=20,            # More parallelism for speed
    max_char_buffer=1000,      # Smaller chunks = better accuracy per chunk
)

Tuning guidelines:

• max_char_buffer: 800-1000 for high accuracy, 2000-5000 for speed. Smaller means more chunks but each chunk gets more focused LLM attention.
• extraction_passes: 1 for quick/simple tasks, 2-3 for thorough extraction of long documents. Each pass costs proportionally more.
• max_workers: 10-20 is typical. More workers = faster but higher API concurrency. Watch rate limits.

Vertex AI Batch Processing

For cost optimization on large-scale tasks:

result = lx.extract(
    ...,
    language_model_params={
        "vertexai": True,
        "project": "your-project-id",
        "location": "global",
        "batch": {"enabled": True}
    }
)

Prompt and Example Design Patterns

Read references/prompt-patterns.md for detailed guidance on designing prompts and examples for different domains (medical, legal, financial, literary analysis, NER).

Error Handling

Common issues and fixes:

• "API key not found": Set LANGEXTRACT_API_KEY or pass api_key=
• Prompt alignment warnings: Your example extraction_text doesn't appear verbatim in the example text. Fix the example.
• Empty extractions: Prompt is too vague or examples don't match the input domain. Improve examples.
• Rate limit errors: Reduce max_workers or add retry logic.
• Network errors: The LLM API endpoint may be unreachable. See Network Constraint above.

When to Generate a Script vs. Run Inline

• If the user provides an API key and the target LLM endpoint is reachable: write and execute a Python script directly.
• If the network blocks the LLM endpoint: generate the script, save it to outputs, and tell the user to run it locally.
• For complex multi-document pipelines: always generate a standalone script.