Scaffold new PySpark notebooks for the Contoso Fabric Platform. Selects the correct template based on the target layer, fills placeholders, registers the schema, and creates test stubs.
Use this skill whenever a new notebook needs to be created for any layer of the medallion architecture. This skill ensures consistency across all notebooks.
Identify the target layer based on the notebook's purpose:
Choose the appropriate template:
templates/bronze_ingestion.py.tmpltemplates/silver_transform.py.tmpltemplates/gold_aggregate.py.tmplReplace these placeholders in the template:
| Placeholder | Description | Example |
|---|---|---|
{{ENTITY}} | Entity/table name in snake_case | pos_transactions |
{{ENTITY_UPPER}} | Entity name in UPPER_CASE | POS_TRANSACTIONS |
{{SOURCE_PATH}} | Input data path | lakehouse/bronze/raw/pos_transactions/ |
{{TARGET_PATH}} | Output Delta table path | lakehouse/bronze/pos_transactions/ |
{{DESCRIPTION}} | One-line description | Ingest POS transactions from raw CSV |
{{MERGE_KEY}} | Business key column(s) | transaction_id |
{{LAYER}} | Layer name | bronze |
Add the new table's StructType schema to notebooks/_shared/schema_registry.py in the
SCHEMAS dictionary.
Create tests/unit/test_{entity}.py with:
Run: python -m py_compile notebooks/{layer}/{notebook_file}.py
Fix any syntax errors before reporting completion.
Summarise: