Live testing skill for real backend data + AI integration tests. Use this for end-to-end validation with production-like conditions.
Announce at start: "I'm using the live-test skill to perform integration tests with real backend data and AI."
Execute integration and E2E tests using real Java backend data (not mock) and real AI/LLM calls to validate complete system behavior under production-like conditions.
Before running live tests, verify:
Java Backend Availability
curl -s http://localhost:8082/dify/teacher/t-test/classes/me | head -20
Expected: JSON response with class data
Environment Configuration
grep -E "USE_MOCK_DATA|SPRING_BOOT_BASE_URL" .env
Expected:
USE_MOCK_DATA=false (for live tests)SPRING_BOOT_BASE_URL=http://localhost:8082LLM API Key
grep "OPENAI_API_KEY\|ANTHROPIC_API_KEY" .env | head -1
Expected: Valid API key configured
# Ensure real backend mode
export USE_MOCK_DATA=false
# Start Python service if not running
python main.py &
sleep 3
curl -s http://localhost:8000/api/health
# Run all Phase 7 related tests
pytest tests/test_rag_service.py tests/test_knowledge_service.py tests/test_rubric_service.py tests/test_assessment_tools.py tests/test_question_pipeline.py -v --tb=short
# Run with live LLM (mark: live_llm)
pytest tests/ -m "live_llm" -v --tb=short
# Full E2E with real data
pytest tests/test_e2e_page.py tests/test_e2e_phase6.py -v --tb=short
# Specific live tests
pytest tests/test_live_integration.py -v --tb=short
Use these API calls to manually verify behavior:
# Test conversation with real data routing
curl -X POST http://localhost:8000/api/conversation \
-H "Content-Type: application/json" \
-d '{"message": "分析 1A 班的英语成绩", "language": "zh", "teacherId": "t-test"}'
# Test question generation pipeline
curl -X POST http://localhost:8000/api/workflow/generate \
-H "Content-Type: application/json" \
-d '{"userPrompt": "给 1A 班生成一套针对阅读理解的练习题", "language": "zh", "context": {"teacherId": "t-test"}}'
tests/test_data_tools.py, tests/test_adapters.py@pytest.mark.integrationtests/test_rag_service.py, tests/test_knowledge_service.py, tests/test_rubric_service.py@pytest.mark.ragtests/test_question_pipeline.py, tests/test_executor.py@pytest.mark.live_llmtests/test_e2e_*.py, tests/test_live_integration.py@pytest.mark.e2etests/
├── test_live_integration.py # Real backend + AI tests (new)
├── test_rag_service.py # RAG service unit tests
├── test_knowledge_service.py # Knowledge point tests
├── test_rubric_service.py # Rubric loading tests
├── test_assessment_tools.py # Assessment analysis tests
├── test_question_pipeline.py # Question generation tests
└── test_e2e_phase6.py # Phase 6 E2E tests
| Symptom | Likely Cause | Resolution |
|---|---|---|
ConnectionRefusedError | Java backend not running | Start Java service on port 8082 |
CircuitOpenError | Backend repeatedly failing | Wait 60s for circuit reset, check Java logs |
LLM timeout | API rate limit or network | Retry with backoff, check API key |
ValidationError on response | Model schema mismatch | Update Pydantic models to match Java API |
USE_MOCK_DATA=falsedocs/testing/ with timestampdocs/testing/use-cases.mdAfter test completion, generate a summary:
# Generate test report
pytest tests/ -v --tb=short --html=docs/testing/report-$(date +%Y%m%d).html
# Or text summary
pytest tests/ -v --tb=short > docs/testing/report-$(date +%Y%m%d).txt 2>&1
# Expected flow: Router → build_workflow → PlannerAgent → Blueprint
message = "分析 1A 班的英语成绩表现"
# Verify: Blueprint contains data_contract with get_teacher_classes + get_class_detail
# Expected flow: assess weakness → fetch rubric → question pipeline
message = "根据学生弱项为 1A 班生成练习题"
# Verify: Questions target identified weak knowledge points
# Expected flow: Router (followup) → refine → PatchAgent → patch plan
message = "把分析内容缩短一半"
context = {"blueprint": {...}, "pageContext": {...}}
# Verify: Returns patch_plan with scope=patch_compose