Execute code and manage compute on Databricks. Use this skill when the user mentions: "run code", "execute", "run on databricks", "serverless", "no cluster", "run python", "run scala", "run sql", "run R", "run file", "push and run", "notebook run", "batch script", "model training", "run script on cluster", "create cluster", "new cluster", "resize cluster", "modify cluster", "delete cluster", "terminate cluster", "create warehouse", "new warehouse", "resize warehouse", "delete warehouse", "node types", "runtime versions", "DBR versions", "spin up compute", "provision cluster".
Run code on Databricks. Three execution modes—choose based on workload.
| Aspect | Databricks Connect ⭐ | Serverless Job | Interactive Cluster |
|---|---|---|---|
| Use for | Spark code (ETL, data gen) | Heavy processing (ML) | State across tool calls, Scala/R |
| Startup | Instant | ~25-50s cold start | ~5min if stopped |
| State | Within Python process |
| None |
| Via context_id |
| Languages | Python (PySpark) | Python, SQL | Python, Scala, SQL, R |
| Dependencies | withDependencies() | CLI with environments spec | Install on cluster |
Spark-based code? → Databricks Connect (fastest)
└─ Python 3.12 missing? → Install it + databricks-connect
└─ Install fails? → Ask user (don't auto-switch modes)
Heavy/long-running (ML)? → Serverless Job (independent)
Need state across calls? → Interactive Cluster (list and ask which one to use)
Scala/R? → Interactive Cluster (list and ask which one to use)
Read the reference file for your chosen mode before proceeding.
python my_spark_script.py
execute_code(file_path="/path/to/script.py")
# Check for running clusters first (or use the one instructed)
list_compute(resource="clusters")
# Ask the customer which one to use
# Run code, reuse context_id for follow-up MCP call
result = execute_code(code="...", compute_type="cluster", cluster_id="...")
execute_code(code="...", context_id=result["context_id"], cluster_id=result["cluster_id"])
| Tool | For | Purpose |
|---|---|---|
execute_code | Serverless, Interactive | Run code remotely |
list_compute | Interactive | List clusters, check status, auto-select running cluster |
manage_cluster | Interactive | Create, start, terminate, delete. COSTLY: start takes 3-8 min—ask user |
manage_sql_warehouse | SQL | Create, modify, delete SQL warehouses |