Scrape Quality

Goal

Ensure scraped studio records are accurate, complete, and trustworthy so customers can confidently compare studios.

Workflow

1. Start with a recent scrape artifact (JSON/JSONL) and define audit scope:
   • Full dataset or targeted source/city.
2. Validate schema and required fields:
   • Confirm each record has required keys and valid types.
   • Fail fast on malformed URLs, impossible coordinates, or invalid enums.
3. Measure completeness for customer-critical fields:
   • Studio name, location, dance styles, level signal, schedule signal, pricing signal, contact method.
   • Report completion rates and flag fields below threshold.
4. Check normalization quality:
   • Style and level vocabularies are standardized.
   • Address/city/state formatting is consistent.
5. Check deduplication quality:
   • Detect near-duplicates by name + address + URL patterns.
   • Confirm obvious duplicates are merged or marked.