Smoke Test

Standard Operating Procedure (SOP)

Phase 1: Code Update Check

1. Confirm current directory - Verify that the current working directory is the DeerFlow project root
2. Check Git status - See whether there are uncommitted changes
3. Pull the latest code - Use git pull origin main to get the latest updates
4. Confirm code update - Verify that the latest code was pulled successfully

Phase 2: Deployment Mode Selection and Environment Check

Choose deployment mode:

• Ask for user preference, or choose automatically based on network conditions
• Default to local installation mode

Local mode environment check:

1. Check Node.js version - Requires 22+
2. Check pnpm - Package manager
3. Check uv - Python package manager
4. Check nginx - Reverse proxy
5. Check required ports - Confirm that ports 2026, 3000, 8001, and 2024 are not occupied

Docker mode environment check (if Docker is selected):

1. Check whether Docker is installed - Run docker --version
2. Check Docker daemon status - Run docker info
3. Check Docker Compose availability - Run docker compose version
4. Check required ports - Confirm that port 2026 is not occupied

Phase 3: Configuration Preparation

1. Check whether config.yaml exists
   • If it does not exist, run make config to generate it
   • If it already exists, check whether it needs an upgrade with make config-upgrade
2. Check the .env file
   • Verify that required environment variables are configured
   • Especially model API keys such as OPENAI_API_KEY

Phase 4: Deployment Execution

Local mode deployment:

1. Check dependencies - Run make check
2. Install dependencies - Run make install
3. (Optional) Pre-pull the sandbox image - If needed, run make setup-sandbox
4. Start services - Run make dev-daemon (background mode, recommended) or make dev (foreground mode)
5. Wait for startup - Give all services enough time to start completely (90-120 seconds recommended)

Docker mode deployment (if Docker is selected):

1. Initialize Docker environment - Run make docker-init
2. Start Docker services - Run make docker-start
3. Wait for startup - Give all containers enough time to start completely (60 seconds recommended)

Phase 5: Service Health Check

Local mode health check:

1. Check process status - Confirm that LangGraph, Gateway, Frontend, and Nginx processes are all running
2. Check frontend service - Visit http://localhost:2026 and verify that the page loads
3. Check API Gateway - Verify the http://localhost:2026/health endpoint
4. Check LangGraph service - Verify the availability of relevant endpoints
5. Frontend route smoke check - Run bash .agent/skills/smoke-test/scripts/frontend_check.sh to verify key routes under /workspace

Docker mode health check (when using Docker):

1. Check container status - Run docker ps and confirm that all containers are running
2. Check frontend service - Visit http://localhost:2026 and verify that the page loads
3. Check API Gateway - Verify the http://localhost:2026/health endpoint
4. Check LangGraph service - Verify the availability of relevant endpoints
5. Frontend route smoke check - Run bash .agent/skills/smoke-test/scripts/frontend_check.sh to verify key routes under /workspace

Optional Functional Verification

1. List available models - Verify that model configuration loads correctly
2. List available skills - Verify that the skill directory is mounted correctly
3. Simple chat test - Send a simple message to verify the end-to-end flow

Phase 6: Generate Test Report

1. Collect all test results - Summarize execution status for each phase
2. Record encountered issues - If anything fails, record the error details
3. Generate the final report - Use the template that matches the selected deployment mode to create the complete test report, including overall conclusion, detailed key test cases, and explicit frontend page / route results
4. Provide follow-up recommendations - Offer suggestions based on the test results

Execution Rules

• Follow the sequence - Execute strictly in the order described above
• Idempotency - Every step should be safe to repeat
• Error handling - If a step fails, stop and report the issue, then provide troubleshooting suggestions
• Detailed logging - Record the execution result and status of each step
• User confirmation - Ask for confirmation before potentially risky operations such as overwriting config
• Mode preference - Prefer local mode to avoid network-related issues
• Template requirement - The final report must use the matching template under templates/; do not output a free-form summary instead of the template-based report
• Report clarity - The execution summary must include the overall pass/fail conclusion plus per-case result explanations, and frontend smoke check results must be listed explicitly in the report
• Optional phase handling - If functional verification is not executed, do not present it as a separate skipped phase in the final report

Known Acceptable Warnings

The following warnings can appear during smoke testing and do not block a successful result:

• Feishu/Lark SSL errors in Gateway logs (certificate verification failure) can be ignored if that channel is not enabled
• Warnings in LangGraph logs about missing methods in the custom checkpointer, such as adelete_for_runs or aprune, do not affect the core functionality

Key Tools

Use the following tools during execution:

1. bash - Run shell commands
2. present_file - Show generated reports and important files
3. task_tool - Organize complex steps with subtasks when needed

Success Criteria

Smoke test pass criteria (local mode):

• Latest code is pulled successfully
• Local environment check passes (Node.js 22+, pnpm, uv, nginx)
• Configuration files are set up correctly
• make check passes
• make install completes successfully
• make dev starts successfully
• All service processes run normally
• Frontend page is accessible
• Frontend route smoke check passes (/workspace key routes)
• API Gateway health check passes
• Test report is generated completely

Smoke test pass criteria (Docker mode):

• Latest code is pulled successfully
• Docker environment check passes
• Configuration files are set up correctly
• make docker-init completes successfully
• make docker-start completes successfully
• All Docker containers run normally
• Frontend page is accessible
• Frontend route smoke check passes (/workspace key routes)
• API Gateway health check passes
• Test report is generated completely

Read Reference Files

Before starting execution, read the following reference files:

1. references/SOP.md - Detailed step-by-step operating instructions
2. references/troubleshooting.md - Common issues and solutions
3. templates/report.local.template.md - Local mode test report template
4. templates/report.docker.template.md - Docker mode test report template