Cts Triage

This gives you the pass rate and number of failures. Document this as your baseline.

Step 2: Identify Test Subcategories

Review the output from the running the CTS (with or without --list) to identify any subcategories that may exist within the suite being analyzed. Subcategories typically have an additional :- or ,-delimited word in the test name. Running tests by subcategory may be more manageable than running with the entire suite at once or running individual tests.

Step 3: Run Each Subcategory

Test each subcategory individually to identify which ones are passing vs failing:

cargo xtask cts 'webgpu:api,validation,category:subcategory:*' 2>&1 | grep -E "(Passed|Failed|Skipped)" | tail -3

Track the pass rate for each subcategory. This helps you identify:

• What's already working (don't break it!)
• Where the failures are concentrated
• Which issues affect multiple categories

Step 4: Analyze Failure Patterns

For failing subcategories, look at what tests are failing:

cargo xtask cts 'webgpu:api,validation,category:subcategory:*' 2>&1 | grep "\[fail\]" | head -20

Look for patterns:

• Format-specific failures: May indicate missing format capability checks
• Parameter-specific failures: Validation missing for specific parameter combinations

Step 5: Examine Specific Failures

Pick a representative failing test and run it individually to see the error:

cargo xtask cts 'webgpu:api,validation,category:subcategory:specific_test' 2>&1 | tail -30

Look for:

• "EXPECTATION FAILED: DID NOT REJECT": wgpu is accepting invalid input (validation gap)
• "Validation succeeded unexpectedly": Similar to above
• "Unexpected validation error occurred": wgpu is rejecting valid input
• Error message content: Tells you what validation is triggering or missing

Step 6: Check CTS Test Source

To understand what the test expects, read the TypeScript source:

grep -A 40 "test_name" cts/src/webgpu/api/validation/path/file.spec.ts

The test source shows:

• What configurations are being tested
• What the expected behavior is (pass/fail)
• The validation rules from the WebGPU spec
• Comments explaining the rationale

Step 7: Categorize Issues

Group failures into categories:

High Priority - Validation Gaps:

• wgpu accepts invalid configurations that should fail
• Security or correctness implications
• Example: Accepting wrong texture formats, missing aspect checks

Medium Priority - Spec Compliance:

• Edge cases not handled correctly
• Optional field validation issues
• Example: depthCompare optional field handling

Low Priority - Minor Gaps:

• Less common scenarios
• Limited real-world impact
• Example: Depth bias with non-triangle topologies

Known Issues - Skip:

• Known failure patterns (documented in AGENTS.md)
• Track count but don't try to fix

Step 8: Identify Root Causes

For validation gaps, find where validation should happen:

1. Search for existing validation:

   grep -n "relevant_keyword" wgpu-core/src/device/resource.rs
   

2. Look for render/compute pipeline creation:

   • Render pipeline: wgpu-core/src/device/resource.rs around create_render_pipeline
   • Compute pipeline: Similar location
   • Look for existing validation patterns you can follow

3. Check for helper functions:

   grep "fn is_" wgpu-types/src/texture/format.rs
   

4. Find error enums:

   grep "pub enum.*Error" wgpu-core/src/pipeline.rs
   

Step 9: Implement Fixes

When implementing fixes:

1. Add error variants if needed (in wgpu-core/src/pipeline.rs)
2. Add helper methods (in wgpu-types if checking properties)
3. Add validation checks (in wgpu-core/src/device/resource.rs)
4. Test the fix with specific failing tests
5. Run full subcategory to verify all related tests pass
6. Check you didn't break passing tests

Step 10: Document Findings

Create or update a triage document (e.g., category_triage.md).

Do not write information about changes you have made to the triage document. Only capture the state of the tests and any investigation into open issues.

# Category CTS Tests - Triage Report

**Overall Status:** XP/YF/ZS (%/%/%)

## Passing Sub-suites ✅

[List sub-suites that have no failures (all pass or skip)]

## Remaining Issues ⚠️

[List sub-suites that have failures and if it can be stated concisely, a summary of the issue]

## Issue Detail

[List detail of any investigation into failures. Do not go into detail about passed suites, just list the failures.]

### 1. title, e.g. a distinguishing word from the test selector

**Test selector:** `webgpu:api,validation,render_pipeline,depth_stencil_state:format:*`
**What it tests:** [Description]
**Example failure:**
[a selector for a single failing test, e.g.:]

```