Skill: automated-test-suite-synthesis

[Case 2]

• Initial Environment: A development environment for a 'Modular Inverse' mathematical problem where $N$ up to $10^{18}$ is provided. The task is to calculate the modular inverse under a prime modulus $P$.
• Real Question: Create a test suite to detect if an implementation correctly handles the case where $N$ and $P$ are not coprime, or where the result overflows a 64-bit integer during intermediate calculations.
• Real Trajectory: The agent analyzes the problem constraints and identifies that naive implementations often fail when the prime is extremely large ($> 2^{31}-1$). It performs a differential analysis by comparing a 'naive' extended Euclidean algorithm and a 'robust' version using arbitrary-precision arithmetic. It identifies that the common error is an integer overflow that occurs during the intermediate multiplication step. The agent writes a generator script targeting boundary values near $10^{18}$ and uses math explanations to specifically verify the 'no inverse exists' logic. The resulting test suite includes several prime and non-prime $P$ values at the maximum scale.
• Real Answer: A test suite targeting overflow boundaries and non-coprime edge cases for modular arithmetic.
• Why this demonstrates the capability: The agent identifies a specific 'cognitive bias' (the assumption that inputs fit in a standard integer) and uses differential analysis to generate cases that specifically challenge that assumption. This requires mapping abstract mathematical constraints to concrete, executable input scripts.

[Case 3]

• Initial Environment: A contest-style repository containing a complex string-matching problem involving regular expressions and multi-language character sets. The environment includes 10 correct solutions and 50 'Wrong Answer' submissions from humans.
• Real Question: Identify the distinct error patterns in the provided failed submissions and synthesize a minimal set of test cases that covers all unique failure modes.
• Real Trajectory: The agent executes all incorrect submissions against a large pool of random inputs to identify 'failing seeds'. It then uses a diversity-ratio analysis to cluster these failures into unique 'Error Patterns', such as failure to handle nested brackets or multi-byte UTF-8 sequences. For each unique pattern, it implements a specialized Python script that generates the simplest possible input that reveals that specific error. It verifies the suite's efficiency by confirming that the number of tests is minimized while the 'Detection Rate' remains maximized. Finally, it uses a self-validation pass to ensure every generated string is a valid challenge according to the problem rules.
• Real Answer: A minimized, highly discriminative test suite where each test case is designed to capture a unique, non-redundant logical error pattern identified from human bugs.
• Why this demonstrates the capability: This shows the agent's ability to manage 'inter-test correlation'. By focusing on 'distinct error pattern coverage', the agent avoids redundant testing and builds a more efficient verifier that is optimized for both speed and diagnostic power.

Pipeline Execution Instructions

To synthesize data for this capability, you must strictly follow a 3-phase pipeline. Do not hallucinate steps. Read the corresponding reference file for each phase sequentially:

1. Phase 1: Environment Exploration Read the exploration guidelines to discover raw knowledge seeds: references/EXPLORATION.md

2. Phase 2: Trajectory Selection Once Phase 1 is complete, read the selection criteria to evaluate the trajectory: references/SELECTION.md

3. Phase 3: Data Synthesis Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data: references/SYNTHESIS.md