Torch Compile

2. Graph Break Audit

1. Run torch._dynamo.explain on the target function.
2. Record graph break counts and reasons.
3. Move non-tensor logic outside the compiled region.
4. Re-run the explain pass to confirm fewer breaks.

3. Speedup Expectation Check

1. Confirm the workload is Python-overhead bound.
2. If the workload is GPU compute bound, expect lower gains.
3. Adjust batch size or fuse operations to increase gains.

Non-Obvious Insights

• Compilation overhead shows up on the first few executions, so warmup is required before benchmarking.
• Speedup depends on reducing Python overhead and GPU read/writes; architecture and batch size affect the outcome.
• Graph breaks trade optimization opportunities for correctness rather than crashing.

Evidence

• "torch.compile makes PyTorch code run faster by JIT-compiling PyTorch code into optimized kernels, while requiring minimal code changes." - PyTorch
• "torch.compile takes extra time to compile the model on the first few executions." - PyTorch
• "reducing Python overhead and GPU read/writes, and so the observed speedup may vary on factors such as model architecture and batch size." - PyTorch
• "Graph breaks result in lost optimization opportunities, which may still be undesirable, but this is better than silent incorrectness or a hard crash." - PyTorch

Scripts

• scripts/torch-compile_tool.py: CLI for probing torch.compile availability, benchmarking, and explain output.
• scripts/torch-compile_tool.js: Node.js wrapper for the same CLI.

Dependencies

• Python 3.11+ or Node 18+.
• PyTorch 2.0+ for torch.compile.

References

• references/README.md