Data Loading Traceability

Data Loading Traceability Skill

Use When

• Creating or modifying loaders in scratchpad, src, or pipeline workflows.
• Integrating new datasets or changing dataset versions.
• Debugging provenance or reproducibility issues.

Inputs

• Dataset location(s) and ownership.
• Version/date snapshot or retrieval timestamp.
• Split/filter/transformation rules.
• Output schema expectations.

Workflow

1. Define source-of-truth path and acquisition metadata.
2. Record raw-to-processed transformation steps with deterministic ordering.
3. Document split logic (train/val/test) and random seed policy.
4. Validate loaded schema and count summaries.
5. Save trace artifacts in experiment folder (data_lineage.md + quick stats).