Generate Questions By Type

• math - Mathematical reasoning and calculations
• logic - Logical deduction and reasoning
• pattern - Pattern recognition and sequences
• spatial - Spatial reasoning and visualization
• verbal - Verbal reasoning and language
• memory - Memory and recall tasks

Prerequisites

Before using this skill, ensure the following requirements are met:

1. Working Directory: This skill must be invoked from the repository root (the directory containing the question-service subdirectory). The cd question-service commands assume this directory structure.

2. Virtual Environment: The question-service virtual environment must be set up:

   cd question-service && python -m venv venv && source venv/bin/activate && pip install -r requirements.txt
   

3. Environment Variables: Required API keys must be configured in question-service/.env:

   • OPENAI_API_KEY - For question generation (Get an API key)
   • ANTHROPIC_API_KEY - For question judging (Get an API key)

Implementation

When this skill is invoked, follow these steps:

Step 1: Validate Arguments

Parse the arguments to extract type, count, and difficulty:

• If no arguments provided, show usage and exit
• Validate that type is one of: math, logic, pattern, spatial, verbal, memory
• If invalid type, show error with valid options and exit
• If count provided, validate it's a positive integer
• If difficulty provided, validate it's one of: easy, medium, hard

Step 2: Check Environment

Resolve the repo root first — do not rely on the current working directory:

REPO_ROOT=$(git rev-parse --show-toplevel)
test -d "${REPO_ROOT}/question-service/venv" && echo "venv exists" || echo "venv missing"

If venv is missing, inform the user they need to set up the question-service first.

Step 3: Execute Generation

Run the generation script using absolute paths. PYTHONPATH must include the repo root so aiq_types is importable:

REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
  PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" \
  --types <type> --count <count> --async --async-judge --verbose

If difficulty is specified, note that the current run_generation.py script does not support a --difficulty flag. Inform the user that difficulty filtering is not yet implemented and proceed with generation of all difficulty levels.

Step 4: Report Results

After execution:

• Show the generation summary from the script output
• Report how many questions were generated and inserted
• Note any errors or warnings

Examples

Generate 50 math questions (default count)

/generate-questions-by-type math

Executes:

REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
  PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types math --count 50 --async --async-judge --verbose

Generate 100 logic questions

/generate-questions-by-type logic 100

Executes:

REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
  PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types logic --count 100 --async --async-judge --verbose

Generate 25 spatial questions (difficulty not yet supported)

/generate-questions-by-type spatial 25 hard

Notes that difficulty filtering is not yet supported, then executes:

REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
  PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types spatial --count 25 --async --async-judge --verbose

Error Handling

Invalid Type

Error: Invalid question type 'foo'.
Valid types: math, logic, pattern, spatial, verbal, memory

Invalid Count

Error: Count must be a positive integer. Got: 'abc'

Missing venv

Error: Question service virtual environment not found.
Please run: cd question-service && python -m venv venv && source venv/bin/activate && pip install -r requirements.txt

Exit Code Handling

The run_generation.py script returns exit codes that indicate the result. Handle each exit code as follows:

Example handling:

cd question-service && source venv/bin/activate && python run_generation.py --types math --count 50 --async --async-judge --verbose
EXIT_CODE=$?

case $EXIT_CODE in
  0) echo "Generation completed successfully." ;;
  1) echo "Partial failure. Check logs at question-service/logs/ for details." ;;
  2) echo "Complete failure. Check API quotas and retry." ;;
  3) echo "Configuration error. Verify .env file and environment variables." ;;
  4) echo "Database error. Check database file permissions and connectivity." ;;
esac

Provider 400 Errors (before running full pipeline)

If a provider returns HTTP 400 and the root cause is unclear, diagnose with a minimal direct API call before running the full pipeline — the error body is always descriptive:

REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate"
ANTHROPIC_API_KEY=$(grep ANTHROPIC_API_KEY "${REPO_ROOT}/question-service/.env" | cut -d= -f2) \
  python -c "
import anthropic, os
c = anthropic.Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])