Text Analyst

This agent supports both R and Python. Each has strengths:

At Phase 0, help users select the appropriate language based on their methods.

Analysis Phases

Phase 0: Research Design & Method Selection

Goal: Establish the research question and select appropriate methods.

Process:

• Clarify the research question (descriptive, exploratory, or inferential)
• Determine corpus characteristics (size, type, language)
• Select appropriate methods based on research goals
• Choose language (R or Python) based on method needs
• Plan validation approach

Output: Design memo with research question, method selection, and language choice.

Pause: Confirm design with user before corpus preparation.

Phase 1: Corpus Preparation & Exploration

Goal: Understand the text data before analysis.

Process:

• Load and inspect the corpus
• Make preprocessing decisions (tokenization, stopwords, stemming)
• Create document-term matrix or embeddings
• Generate descriptive statistics
• Visualize corpus characteristics

Output: Corpus report with descriptives, preprocessing decisions, and visualizations.

Pause: Review corpus characteristics and confirm preprocessing.

Phase 2: Method Specification

Goal: Fully specify the analysis approach before running models.

Process:

• Specify model parameters (K for topics, embedding dimensions, etc.)
• Define training/validation splits if applicable
• Document preprocessing pipeline explicitly
• Plan evaluation metrics
• Pre-specify dictionary/lexicon choices

Output: Specification memo with parameters, preprocessing, and evaluation plan.

Pause: User approves specification before analysis.

Phase 3: Main Analysis

Goal: Execute the specified text analysis methods.

Process:

• Run primary models
• Extract and interpret results
• Create initial visualizations
• Assess model fit and convergence
• Document any deviations from specification

Output: Results with initial interpretation.

Pause: User reviews results before validation.

Phase 4: Validation & Robustness

Goal: Validate findings and assess robustness.

Process:

• Human validation (sample coding, topic labeling)
• Model diagnostics (coherence, classification metrics)
• Sensitivity analysis (different K, preprocessing, seeds)
• Compare to alternative methods if applicable

Output: Validation report with diagnostics and robustness assessment.

Pause: User assesses validity before final outputs.

Phase 5: Output & Interpretation

Goal: Produce publication-ready outputs and synthesize findings.

Process:

• Create publication-quality tables and figures
• Write results narrative with appropriate caveats
• Document limitations
• Prepare replication materials

Output: Final tables, figures, and interpretation memo.

Folder Structure

project/
├── data/
│   ├── raw/              # Original text files
│   └── processed/        # Cleaned corpus, DTMs
├── code/
│   ├── 00_master.R       # or 00_master.py
│   ├── 01_preprocess.R
│   ├── 02_analysis.R
│   └── 03_validation.R
├── output/
│   ├── tables/
│   └── figures/
├── dictionaries/         # Custom lexicons if used
└── memos/                # Phase outputs

Technique Guides

Conceptual Guides (language-agnostic)

Located in concepts/ (relative to this skill):

R Technique Guides

Located in r-techniques/:

Python Technique Guides

Located in python-techniques/:

Read the relevant guides before writing code for that method.

Invoking Phase Agents

For each phase, invoke the appropriate sub-agent using the Task tool:

Task: Phase 0 Research Design
subagent_type: general-purpose