Auditing Math Calculations

Supported Book Types

For test question banks (Bank 1 or Bank 2), also consult the auditing_question_bank_accuracy skill for the bank-specific audit-review PDF workflow and the deeper MC-choice-expansion procedure.

Procedure

Step 0: Determine Scope

Ask the user (or infer from the request) two things:

1. Which book type? — This determines which folder(s) and answer commands to look for.
2. Which chapter, topic, or file? — Could be a single file, a chapter, or all files of a book type.

If the user says "check the workbook chapter 3", that means all files in topics_workbook/ that start with ch03-. If they say "check this file", use the currently open file.

Step 1: Discover the Files

Single file

If the user specifies a file or has one open, use that directly.

By chapter

Run the discovery script to list all topics for that chapter:

python scripts/get_chapter_topic_facts.py --chapter {N}

Then map the output to the correct book type folder. The script returns paths relative to topic type (topics/, topics_additional/, topics_modified/). Prepend the book type prefix:

Not every file listed by the script will exist in every book type. Check existence before reading.

Step 2: Read Every Source File in Full

Read each file completely. Do not skim or sample. Every problem, example, and answer must be checked.

Step 3: Verify All Calculations

Work through the file top-to-bottom, checking every piece of verifiable math. The checks differ by what you find:

3a. Worked Examples (workedExample, sideBySideExample, stepExample, quickLesson)

These are teaching sections with step-by-step solutions.

1. Re-solve the problem independently before reading the example's solution.
2. Walk through every intermediate step. Verify each arithmetic operation, algebraic manipulation, and simplification.
3. Check the final result matches what the example claims.
4. Verify formulas used are correct (e.g., area of a circle is $\pi r^2$, not $\pi d^2$; volume of a cylinder is $\pi r^2 h$, not $\pi r h$).

3b. \answerExplain{answer}{explanation}

Used in: Study Guide, Workbook, Quiz, In 10 Days, In 30 Days.

1. Solve the problem independently. Compute the answer from the question text alone.
2. Compare your answer with the first argument (the answer field).
3. Walk through the explanation (second argument) and verify every intermediate calculation.
4. Check consistency: Does the explanation arrive at the same answer stated in the first argument?

For Quiz questions — the answer field includes the choice letter (e.g., "B ($\sqrt{5}$)"): 5. Verify the correct answer matches the choice text. If \answerExplain{B (...)}{} is given, confirm \choiceB{} actually contains that value. 6. Evaluate ALL four choices. Confirm the three wrong choices are genuinely wrong. This catches bugs where two choices simplify to the same value.

3c. \worksheetAnswer{text}

Used in: Worksheet.

1. The bold portion is the answer and the rest is the explanation. Parse both.
2. Solve the problem independently from the question/activity instructions above.
3. Verify the bold answer is correct.
4. Walk through the inline explanation and check each calculation.
5. For multi-part activities (circleIt, sortCut, matchIt, errorDetective): verify every individual item, not just the first one.

3d. \puzzleAnswer{text}

Used in: Puzzles.

1. Read the puzzle setup carefully — clues, constraints, and difficulty level.
2. Solve the puzzle yourself using only the clues given.
3. Verify the explanation in \puzzleAnswer{} follows logically from the clues.
4. For sort/classify puzzles: check every single item is correctly categorized.
5. For mystery number puzzles: verify each clue eliminates what the answer claims.

3e. \answer{text}

Used in: Steps, In 30 Days.

1. Solve the problem independently.
2. Compare your result with the answer text.
3. For multi-sentence answers (e.g., "No. Since 15 is not a perfect square..."), verify the reasoning too.

3f. \answerTF{True/False} and \answerMC{letter}

Used in: Study Guide, In 30 Days.

1. Evaluate the statement (for True/False) or work through all choices (for MC).
2. Confirm the keyed value is correct.

3g. \correctAnswer{X} and \explanation{}

Used in: Test Question Banks 1 and 2.

Follow the detailed procedure in the auditing_question_bank_accuracy skill, which covers:

• Independently computing the answer before looking at choices
• Expanding/evaluating EVERY multiple-choice option
• Walking through the explanation step by step
• Checking for contradictions between explanation and keyed answer

3h. Concept Boxes, Tips, and Definitions

Also verify math in non-question environments:

• \begin{conceptBox} — formulas, equations, identities stated as facts
• \begin{tipBox} / \mascotSays{} / \begin{keyIdea} — shortcuts, rules of thumb
• \begin{errorBox} — the "wrong work" must actually contain the stated error, and the correction must be right
• \stepResult{} — final result in step cards
• Inline math in prose — any equation or calculation woven into explanatory text

Step 4: Classify Findings

Only flag issues where the math is objectively wrong:

• A calculation error (arithmetic, algebra, formula)
• The answer doesn't match the explanation
• A formula is stated incorrectly
• A worked example produces the wrong result
• A True/False or MC answer is keyed incorrectly
• An explanation contradicts the stated answer
• For worksheets: the bold answer doesn't match the explanation logic
• For puzzles: the clues don't actually lead to the stated answer

Do NOT flag (and do NOT edit):

• Style or wording preferences
• Alternative solution methods that still produce the correct answer
• Minor phrasing differences in explanations
• Rounding variations when both values are acceptable
• Distractor choices in MC (don't rewrite wrong answers — only flag if the correct answer is wrong or if two choices are identical)
• Formatting, whitespace, comments, or LaTeX structure
• Anything that is not a mathematical error

If the math is correct, leave the file unchanged — no exceptions.

Step 5: Fix ONLY Miscalculations — Nothing Else

If the audit found no miscalculations, skip this step entirely and do not edit any file.

For each confirmed error:

1. Change only what is necessary. If the answer is wrong but the explanation is correct, change only the answer. If both are wrong, fix both.
2. Preserve the original phrasing — change numbers and math, not prose.
3. Do not restructure surrounding content that is correct.
4. For MC questions: Change only \correctAnswer and/or \explanation. Never modify \choiceA–\choiceD. If a distractor happens to equal the correct answer, change only the key and explanation to point to the right choice.
5. For worksheetAnswer: Change only the bold answer and/or the calculation within the explanation.

Step 6: Compile and Validate (Optional)

If the user wants visual confirmation, compile the changed files:

For test question banks — use the audit-review PDF workflow:

cp audit_review_template.tex tests/audit_review.tex
# Edit tests/audit_review.tex to \input{} only the changed files
latexmk -xelatex -interaction=nonstopmode -halt-on-error -outdir=build tests/audit_review.tex
open build/audit_review.pdf

See the auditing_question_bank_accuracy skill for detailed instructions on the audit-review template.

For other book types — compile the relevant main file:

Or, for a quick single-file check, use the audit-review template with an \input{} of the changed file.

Step 7: Report Results

Provide a structured summary:

1. Files audited — list every file read (with count).
2. Issues found and fixed — for each fix:
   • File name and location in the file (problem number or environment)
   • What was wrong (the specific calculation error)
   • What it was changed to (the corrected math)
3. Issues found but NOT fixed — if any require user judgment (e.g., ambiguous question wording that could affect the answer).
4. Clean bill of health — explicitly state if no errors were found, so the user knows the audit is complete, not abandoned.

Common Error Patterns

These are the most frequent calculation bugs across all book types:

Workflow by Request Type

"Check this file" (single file)

→ Read it → Verify all math → Fix errors → Report

"Check [book type] chapter N"

→ Run get_chapter_topic_facts.py --chapter N → Map to folders → Read all files → Verify → Fix → Report

"Check all [book type] files"

→ List all files in the folder(s) → Read each → Verify → Fix → Report (this may be very large; suggest doing it chapter by chapter)

"Check the answers in [book type] for topic X"

→ Find the specific file by topic slug → Read → Verify only the answers → Fix → Report