Guide design, review, audit, and implementation of contract conformance testing using Schemathesis integration with from_asgi() for FastAPI applications, stateful testing with OpenAPI link-based state machines, test database management for conformance test suites, response schema validation, and coverage measurement against the OpenAPI specification. Use when configuring Schemathesis, auditing contract test coverage, reviewing stateful test strategies, or setting up conformance test infrastructure. Triggered by: contract testing, Schemathesis, conformance testing, schema validation, stateful testing, OpenAPI links, from_asgi, property-based testing, API testing, contract test, spec conformance, response validation.
Comprehensive governance for verifying that API implementations conform to their OpenAPI specifications using automated, property-based testing.
A spec and an implementation are two separate artifacts that can silently diverge. The spec says a field is required; the implementation omits it. The spec defines a 201 response; the implementation returns 200. Manual test suites cannot cover the combinatorial space of valid and invalid inputs. This skill enforces automated conformance testing using Schemathesis, which generates test cases directly from the OpenAPI spec and validates that the implementation matches the contract.
Boundary: This skill governs conformance testing between spec and implementation. For the spec itself, see openapi-specification-governance. For the CI pipeline that orchestrates testing alongside other stages, see governing-enforcement-pipeline.
Use this skill when setting up contract conformance testing, auditing test coverage against the spec, or configuring stateful API testing.
Schemathesis reads an OpenAPI specification and automatically generates test cases that exercise every documented endpoint, parameter combination, and edge case. It then validates that responses conform to the documented schemas.
# Python package
pip install schemathesis
# Or via Docker
docker pull schemathesis/schemathesis
# Test against a running server
st run https://api.example.com/openapi.json
# Test with base URL override
st run openapi.json --base-url http://localhost:8000
# Test specific endpoint
st run openapi.json --endpoint /projects
from_asgi()import schemathesis
from myapp.main import app
schema = schemathesis.from_asgi("/openapi.json", app=app)
@schema.parametrize()
def test_api_conformance(case):
response = case.call_asgi()
case.validate_response(response)
from_asgi() Over Network Testing| Aspect | from_asgi() | Network Testing |
|---|---|---|
| Speed | Fast (in-process) | Slow (HTTP overhead) |
| Server required | No | Yes |
| Database control | Direct access | Requires fixtures |
| Debugging | Standard debugger | Remote debugging |
| CI simplicity | Single process | Requires server lifecycle |
# conftest.py
import schemathesis
from myapp.main import app
schema = schemathesis.from_asgi("/openapi.json", app=app)
# tests/test_conformance.py
@schema.parametrize()
def test_api(case):
response = case.call_asgi()
case.validate_response(response)
# Test only specific operations
@schema.parametrize(endpoint="/projects")
def test_projects(case):
response = case.call_asgi()
case.validate_response(response)
# Exclude operations
@schema.parametrize(endpoint="/projects", method="GET")
def test_list_projects(case):
response = case.call_asgi()
case.validate_response(response)
OpenAPI links define relationships between operations. Schemathesis uses these links to generate stateful test sequences:
# In the OpenAPI spec