Jit Catch

If there is no diff, tell the user and stop.

Step 2 — Infer Intent and Risks

Infer intent

Summarize in 1-2 sentences what the change is trying to accomplish. Use the branch name, commit message, and the diff itself as signals. Be specific: "Adding rate limiting to the OrderController.create/2 action" not "Modifying the order controller."

Identify risks

List 3-5 specific risks — concrete ways the implementation could introduce bugs. Consider these Elixir-specific risk categories:

• Pattern match failures — New or changed data shapes that existing function clauses won't match, leading to FunctionClauseError or MatchError
• Missing function clauses — A new code path added without a corresponding clause for edge cases (empty list, nil, error tuples)
• Changed return tuples — Functions that previously returned {:ok, value} now returning {:error, reason} or a bare value, breaking callers
• Broken pipe chains — A function in a pipeline changed its return type, causing the next function in the pipe to fail
• GenServer state corruption — handle_call/handle_cast/handle_info changes that alter the state shape without updating all handlers
• Supervision tree impacts — Child spec changes, init/1 changes, or restart strategy modifications that affect fault tolerance
• Guard clause gaps — New guards that don't cover all cases, or removed guards that widen accepted inputs unexpectedly
• Ecto changeset/query changes — Modified validations, changed associations, altered query conditions that silently change what data is accepted or returned

Step 3 — Generate Catching Tests (Dodgy Diff Approach)

Treat the diff as potentially buggy. For each identified risk, generate an ExUnit test that:

1. Exercises the changed code path directly
2. Asserts the parent behavior — what should still hold true after the change
3. Would fail if the risk has materialized — the test is specifically designed to catch this class of bug

Test format

defmodule MyApp.CatchingTest do
  use ExUnit.Case, async: true

  # Import or alias the modules under test
  alias MyApp.ChangedModule

  @moduletag :catching

  describe "catching tests for [brief description of change]" do
    setup do
      # Minimal setup needed to exercise the code paths
      %{...}
    end

    test "risk: [specific risk description]", ctx do
      # Arrange — set up the specific scenario
      # Act — call the changed function
      # Assert — verify the parent behavior still holds
    end
  end
end

Guidelines

• Use @moduletag :catching at the module level so all tests in the module are tagged and can be run or excluded as a group. Prefer this over per-test @tag :catching to reduce repetition
• Suggest a file path matching the project's existing test structure (e.g., test/my_app/changed_module_catching_test.exs)
• Use idiomatic ExUnit: assert, refute, assert_raise, catch_error, pattern matching in assertions
• Prefer assert match?(pattern, value) or assert {:ok, _} = result over generic equality checks
• Keep tests focused — one risk per test, minimal setup
• If the code under test requires external dependencies (database, HTTP), use mocks or suggest the appropriate test setup but keep it simple
• Include comments explaining what parent behavior the test is verifying and why it matters

Step 4 — Flag True Positive Patterns

After generating tests, scan the diff for suspicious patterns adapted from the paper's Table 3. These are the most common true positive signals in Elixir code:

Reference the detailed descriptions in references/true-positive-patterns.md.

Flag any pattern you detect with its name and a brief explanation of why it's suspicious in this specific diff.

Step 5 — Sense Check Output

Present your findings in this format:

Intent Summary

One or two sentences describing what the change is trying to accomplish.

Sense Checks

Present each finding as a plain-language question, grouped by confidence:

High confidence — patterns that are very likely unintended:

[Pattern name]: ModuleName.function/arity used to [old behavior], but now [new behavior] — is that expected?

Worth checking — patterns that might be intentional but are worth verifying:

[Pattern name]: ModuleName.function/arity [description of the behavioral change] — was this deliberate?

Generated Tests

Present the full ExUnit test module in a code block. Preface it with:

These tests assert the parent behavior. If any fail on your branch, it confirms a behavioral change for that scenario — review whether it's intended.

Suggest where to save the file:

Suggested path: test/my_app/changed_module_catching_test.exs

How to Run

mix test --only catching

Important Constraints

• Do NOT generate tests for changes to test files themselves — only test production code changes
• Do NOT generate tests for purely cosmetic changes (formatting, comments, docs)
• If the diff is very large (>500 lines changed), focus on the highest-risk modules rather than trying to cover everything
• Keep the total number of catching tests to 3-8 per invocation — enough to be useful, not so many they're overwhelming
• If you're unsure about a risk, include it as "worth checking" rather than omitting it