Generate questions of a specific type using the local question-service. Supports type filtering, count, and difficulty options for targeted question generation.
Generate IQ test questions of a specific type using the local question-service.
/generate-questions-by-type <type> [count] [difficulty]
| Argument | Required | Default | Description |
|---|---|---|---|
type | Yes | - | Question type: math, logic, pattern, spatial, verbal, memory |
count | No | 50 | Number of questions to generate |
difficulty | No | all | Difficulty level: easy, medium, hard |
math - Mathematical reasoning and calculationslogic - Logical deduction and reasoningpattern - Pattern recognition and sequencesspatial - Spatial reasoning and visualizationverbal - Verbal reasoning and languagememory - Memory and recall tasksBefore using this skill, ensure the following requirements are met:
Working Directory: This skill must be invoked from the repository root (the directory containing the question-service subdirectory). The cd question-service commands assume this directory structure.
Virtual Environment: The question-service virtual environment must be set up:
cd question-service && python -m venv venv && source venv/bin/activate && pip install -r requirements.txt
Environment Variables: Required API keys must be configured in question-service/.env:
OPENAI_API_KEY - For question generation (Get an API key)ANTHROPIC_API_KEY - For question judging (Get an API key)When this skill is invoked, follow these steps:
Parse the arguments to extract type, count, and difficulty:
math, logic, pattern, spatial, verbal, memoryeasy, medium, hardResolve the repo root first — do not rely on the current working directory:
REPO_ROOT=$(git rev-parse --show-toplevel)
test -d "${REPO_ROOT}/question-service/venv" && echo "venv exists" || echo "venv missing"
If venv is missing, inform the user they need to set up the question-service first.
Run the generation script using absolute paths. PYTHONPATH must include the repo root so
aiq_types is importable:
REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" \
--types <type> --count <count> --async --async-judge --verbose
If difficulty is specified, note that the current run_generation.py script does not support a --difficulty flag. Inform the user that difficulty filtering is not yet implemented and proceed with generation of all difficulty levels.
After execution:
/generate-questions-by-type math
Executes:
REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types math --count 50 --async --async-judge --verbose
/generate-questions-by-type logic 100
Executes:
REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types logic --count 100 --async --async-judge --verbose
/generate-questions-by-type spatial 25 hard
Notes that difficulty filtering is not yet supported, then executes:
REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate" && \
PYTHONPATH="${REPO_ROOT}" python "${REPO_ROOT}/question-service/run_generation.py" --types spatial --count 25 --async --async-judge --verbose
Error: Invalid question type 'foo'.
Valid types: math, logic, pattern, spatial, verbal, memory
Error: Count must be a positive integer. Got: 'abc'
Error: Question service virtual environment not found.
Please run: cd question-service && python -m venv venv && source venv/bin/activate && pip install -r requirements.txt
The run_generation.py script returns exit codes that indicate the result. Handle each exit code as follows:
| Exit Code | Meaning | Action |
|---|---|---|
| 0 | Success | Report the number of questions generated and inserted. No action needed. |
| 1 | Partial failure | Some questions were generated but others failed. Report the count of successful insertions and warn the user to check logs at question-service/logs/ for details on failures. Suggest retrying with a smaller count if many failures occurred. |
| 2 | Complete failure | No questions were generated. Check the error output for the cause. Common causes: API rate limits, network issues, or invalid prompts. Suggest the user wait and retry, or check API quotas. |
| 3 | Configuration error | Environment or argument configuration is invalid. Check that .env file exists with valid API keys. Verify all required environment variables are set. |
| 4 | Database error | Cannot connect to or write to the database. Verify database connectivity and check database configuration (DATABASE_URL). For local SQLite, ensure question-service/questions.db exists and is writable. |
Example handling:
cd question-service && source venv/bin/activate && python run_generation.py --types math --count 50 --async --async-judge --verbose
EXIT_CODE=$?
case $EXIT_CODE in
0) echo "Generation completed successfully." ;;
1) echo "Partial failure. Check logs at question-service/logs/ for details." ;;
2) echo "Complete failure. Check API quotas and retry." ;;
3) echo "Configuration error. Verify .env file and environment variables." ;;
4) echo "Database error. Check database file permissions and connectivity." ;;
esac
If a provider returns HTTP 400 and the root cause is unclear, diagnose with a minimal direct API call before running the full pipeline — the error body is always descriptive:
REPO_ROOT=$(git rev-parse --show-toplevel)
source "${REPO_ROOT}/question-service/venv/bin/activate"
ANTHROPIC_API_KEY=$(grep ANTHROPIC_API_KEY "${REPO_ROOT}/question-service/.env" | cut -d= -f2) \
python -c "
import anthropic, os
c = anthropic.Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])