Diagnosis and resolution guides for common issues in the system, including Airflow failures, database connection errors, API authentication, and data integrity problems.
Symptom: DAG not appearing in Airflow UI
# Test DAG parsing locally
python dags/multi_sport_betting_workflow.py
# Check Airflow logs
docker logs $(docker ps -qf "name=scheduler") 2>&1 | grep -i error
Common causes:
Fix: Ensure plugins directory is in Python path:
plugins_dir = Path(__file__).parent.parent / "plugins"
if str(plugins_dir) not in sys.path:
sys.path.insert(0, str(plugins_dir))
Symptom: connection refused or
authentication failed# Check PostgreSQL is running
docker ps | grep postgres
# Test connection
psql -h localhost -U airflow -d airflow -c "SELECT 1"
Environment variables:
export POSTGRES_HOST=localhost
export POSTGRES_PORT=5432
export POSTGRES_USER=airflow
export POSTGRES_PASSWORD=airflow
export POSTGRES_DB=airflow
Symptom: 401 Unauthorized or Invalid API key
Check credentials:
with open("kalshkey") as f:
api_key = f.readline().strip()
private_key = f.read()
print(f"API Key length: {len(api_key)}")
print(f"Private key starts with: {private_key[:50]}")
Common issues:
Symptom: Bets not matching, empty results
from naming_resolver import NamingResolver
resolver = NamingResolver()
# Debug team name resolution
print(resolver.resolve("LAL", "nba"))
print(resolver.resolve("Lakers", "nba"))
print(resolver.resolve("Los Angeles Lakers", "nba"))
Fix: Add missing mappings to naming_resolver.py:
NBA_ALIASES = {
"LA Lakers": "Los Angeles Lakers",
"LAL": "Los Angeles Lakers",
# Add missing aliases
}
Symptom: Missing games, stale data
# Check last successful run
docker exec $(docker ps -qf "name=scheduler") \
airflow dags list-runs -d multi_sport_betting_workflow --limit 5
Debug data freshness:
from db_manager import default_db
df = default_db.fetch_df("""
SELECT sport, MAX(game_date) as latest
FROM unified_games
GROUP BY sport
""")
print(df)
Get task logs:
docker exec $(docker ps -qf "name=scheduler") \
airflow tasks logs multi_sport_betting_workflow download_games_nba 2024-01-15
Clear failed task:
docker exec $(docker ps -qf "name=scheduler") \
airflow tasks clear multi_sport_betting_workflow -t download_games_nba -s 2024-01-15 -e 2024-01-15
Symptom: OOM errors, slow performance
# Process data in chunks
for chunk in pd.read_sql(query, engine, chunksize=10000):
process(chunk)
Symptom: ModuleNotFoundError
# Check if module exists
ls plugins/ | grep module_name
# Check Python path
python -c "import sys; print('\n'.join(sys.path))"
TROUBLESHOOTING.md - Additional troubleshootingdocker-compose.yaml - Service configurationlogs/ - Airflow logs directory