Real Pytest No Mocks Real Tests

• ✅ PASS - the code works correctly
• ❌ FAIL - the code is broken and needs fixing

There is no third state. Skipped tests are:

• Technical debt pretending to be documentation
• Broken code that someone gave up on
• False confidence in test coverage metrics

If a test can't run:

• Fix the environment/dependencies so it can run
• Fix the code so the test passes
• Delete the test if it's testing something that doesn't exist

NEVER commit a skipped test. Either make it pass or delete it.

PHASE 1: Contract-First Analysis (DO THIS FIRST)

NEVER write tests by reading implementation. That's how you write tests that mirror what code does instead of what it should do.

Protocol: Analyze Contract Without Reading Implementation

Step 1: Read ONLY the module's public interface

# Read THIS (public interface)
class ReminderTool:
    def run(self, operation: str, **kwargs) -> Dict[str, Any]:
        """Execute reminder operations."""
        pass

# DO NOT read implementation details
# DO NOT look at internal methods
# DO NOT read how it's implemented

Step 2: Document the contract

Before writing any test, answer these questions in writing:

MODULE CONTRACT ANALYSIS
========================

1. What is this module's PURPOSE?
   - What problem does it solve?
   - Why does it exist?

2. What GUARANTEES does it provide?
   - What promises does the API make?
   - What invariants must hold?
   - What post-conditions are guaranteed?

3. What should SUCCEED?
   - Valid inputs
   - Happy path scenarios
   - Boundary cases that should work

4. What should FAIL?
   - Invalid inputs
   - Boundary conditions that should error
   - Security violations
   - Resource constraints

5. What are the DEPENDENCIES?
   - What does this module depend on?
   - Are there too many dependencies?
   - Could this be simpler?

6. ARCHITECTURAL CONCERNS:
   - Is this module doing too much?
   - Is it papering over design failures elsewhere?
   - Does the contract make sense or is it convoluted?
   - Should this module even exist?

Step 3: Design test cases from contract

Based on contract analysis (NOT implementation):

• List positive test cases (what should work)
• List negative test cases (what should fail)
• List boundary conditions
• List security concerns
• List performance concerns

See "CANONICAL EXAMPLE" section below for complete contract analysis walkthrough.

PHASE 1.5: Contract Verification (VALIDATE YOUR ASSUMPTIONS)

CRITICAL: Do NOT read the implementation file yourself. Use the contract-extractor agent as an abstraction barrier.

Why This Phase Exists

You've formed expectations about the contract from the interface. Now verify those expectations against actual implementation WITHOUT seeing the implementation yourself. The agent reads the code and reports ONLY contract facts (not implementation details).

Protocol: Invoke Agent → Compare → Identify Gaps

Step 1: Invoke the contract-extractor agent

# Use Task tool to invoke the agent
Task(
    subagent_type="contract-extractor",
    description="Extract contract from module",
    prompt="""Extract the contract from: path/to/module.py

Return:
- Public interface (methods, signatures, types)
- Actual return structures (dict keys, types)
- Exception contracts (what raises what, when)
- Edge cases handled
- Dependencies and architectural concerns"""
)

Step 2: Compare your expectations against agent report

Create a comparison:

EXPECTATION vs REALITY
======================

Expected return structure:
{
    "status": str,
    "results": list
}

Actual return structure (from agent):
{
    "status": str,
    "confidence": float,  # I MISSED THIS
    "results": list,
    "result_count": int   # I MISSED THIS
}

Expected exceptions:
- ValueError for empty query

Actual exceptions (from agent):
- ValueError for empty query ✓
- ValueError for negative max_results  # I MISSED THIS

Expected edge cases:
- Empty results returns []

Actual edge cases (from agent):
- Empty results returns status="low_confidence", confidence=0.0, results=[]
  # More nuanced than I expected

Step 3: Identify discrepancies and their implications

For each discrepancy, ask:

• Is the code wrong (doesn't match intended contract)?
• Is the contract unclear (missing documentation)?
• Did I misunderstand the requirements?
• Is this an undocumented feature (needs test)?

Example Analysis:

DISCREPANCY: Agent reports confidence field in return, I didn't expect it
IMPLICATION: This is part of the contract - add test to verify confidence in [0.0, 1.0]

DISCREPANCY: Agent reports ValueError for negative max_results, I didn't expect it
IMPLICATION: Good edge case handling - add negative test

DISCREPANCY: Agent reports 8 dependencies, I expected 3-4
IMPLICATION: ARCHITECTURAL CONCERN - too many deps, report to human

Step 4: Update test plan based on verified contract

Now you know:

• What the code actually returns (test these exact structures)
• What exceptions are actually raised (test these exact cases)
• What edge cases are actually handled (test these behaviors)
• What architectural problems exist (report these to human)

Step 5: Design comprehensive test cases

# Based on VERIFIED contract (not assumptions):

# Positive tests
- test_search_returns_exact_structure  # Verify all keys agent reported
- test_search_confidence_in_valid_range  # Agent said 0.0-1.0
- test_search_respects_max_results  # Agent confirmed this guarantee

# Negative tests
- test_search_rejects_empty_query  # Agent confirmed ValueError
- test_search_rejects_negative_max_results  # Agent revealed this

# Edge cases
- test_search_empty_results_structure  # Agent showed exact structure
- test_search_with_no_user_data  # Based on RLS info from agent

# Architectural concerns
- Report to human: "Module has 8 dependencies - possible SRP violation"

See "CANONICAL EXAMPLE" section below for complete agent invocation, comparison, and gap analysis walkthrough.

When to Read Implementation

Only AFTER writing tests based on verified contract. Then you can read implementation for context, debugging, or refactoring - but tests are already protecting the contract.

PHASE 2: Fail-First Verification (PROVE TESTS CAN FAIL)

A test that always passes proves nothing. You must see it fail.

Protocol: Write → Fail → Verify

Step 1: Write test based on contract expectations

Don't look at implementation. Write assertions based on what the contract says SHOULD happen.

def test_search_returns_confidence_score(search_tool, authenticated_user):
    """Contract: search must return confidence score between 0.0 and 1.0"""
    user_id = authenticated_user["user_id"]
    set_current_user_id(user_id)

    # Based on contract, not implementation
    result = search_tool.run(
        operation="search",
        query="Python async patterns",
        max_results=5
    )

    # Contract expectations
    assert "confidence" in result
    assert 0.0 <= result["confidence"] <= 1.0
    assert "results" in result
    assert len(result["results"]) <= 5

Step 2: Run the test - expect failure or question success

pytest tests/test_search_tool.py::test_search_returns_confidence_score -v

If test FAILS:

• Is this the expected failure? (No data exists yet)
• Is the failure message clear?
• Is this exposing a bug in the code?
• Is this exposing a problem with the contract?

If test PASSES immediately:

• Is the code actually correct?
• Are my assertions too weak?
• Am I testing a trivial case?
• Did I set up test data somewhere I forgot about?

Step 3: Verify the test can actually catch bugs

Temporarily break the code and verify the test fails:

# In the actual implementation, temporarily break it:
def run(self, operation, **kwargs):
    return {"confidence": 2.5}  # INTENTIONAL BUG: exceeds 1.0

Run test - it should fail. If it doesn't, your assertions are too weak.

Step 4: Remove the intentional bug, test should pass

Now you have confidence the test actually works.

Common Testing Anti-Patterns

When writing tests, surface design problems - don't paper over them.

When you find architectural red flags, report them:

ARCHITECTURAL CONCERN: ToolName

PROBLEM: [Specific issue]
EVIDENCE: [What you observed]
IMPACT: [Why this matters]
RECOMMENDATION: [Specific fix]

Core Principle: NEVER MOCK

MIRA's testing philosophy: NEVER MOCK. Use actual services, real databases, real APIs.

This is a hard rule because mocking is where I slip. The test will seem "hard" without mocks and I'll think "just this once..." DON'T.

Why No Mocking?

Mocks test mocks, not code:

# This tests nothing about real behavior:
@mock.patch('tool.database')
def test_reminder_tool(mock_db):
    mock_db.query.return_value = [{"id": 1, "title": "Test"}]
    result = tool.get_reminders()
    # You have NO IDEA if real code works

Real tests catch real bugs:

# This will fail if database schema changes:
def test_reminder_tool(sqlite_test_db):
    tool = ReminderTool()
    tool.run("add_reminder", title="Test", date="2025-01-01")

    # Real database query
    rows = sqlite_test_db.execute("SELECT * FROM reminders WHERE title = ?", ("Test",))
    assert len(rows) == 1

If it's hard to test without mocks, the design is wrong. Fix the design, don't mock it away.

MIRA Test Infrastructure

🚨 USE authenticated_user - IT'S FULLY SET UP

The authenticated_user fixture gives you EVERYTHING you need:

def test_anything(authenticated_user):
    # These are ALL set up and ready:
    user_id = authenticated_user["user_id"]           # Test user ID
    continuum_id = authenticated_user["continuum_id"] # Test user's continuum (ALREADY EXISTS)
    email = authenticated_user["email"]               # [email protected]
    token = authenticated_user["access_token"]        # Valid session token

    # User context is ALREADY SET - just use user_id
    # Continuum ALREADY EXISTS - just add messages to it
    # Cleanup happens AUTOMATICALLY - no manual teardown needed

What authenticated_user does for you:

1. ✅ Creates test user (if doesn't exist)
2. ✅ Creates test user's continuum (if doesn't exist)
3. ✅ Sets user context for RLS
4. ✅ Creates valid session token
5. ✅ Cleans up all test data before/after each test
6. ✅ Preserves user record for reuse

How to use it (SIMPLE):

# ✅ CORRECT - Just use the continuum_id that's already there
def test_add_messages(authenticated_user, test_db):
    user_id = authenticated_user["user_id"]
    continuum_id = authenticated_user["continuum_id"]  # Use this!
    set_current_user_id(user_id)  # Set context

    repo = get_continuum_repository()
    msg = Message(role="user", content="Test message")
    repo.save_message(msg, continuum_id, user_id)  # Just save directly

    # Message is in the database, ready to test

# ❌ WRONG - Don't create new continuums (test user already has one)
def test_add_messages_wrong(authenticated_user):
    user_id = authenticated_user["user_id"]
    repo = get_continuum_repository()
    continuum = repo.create_continuum(user_id)  # DON'T DO THIS!
    # This creates a SECOND continuum - user should only have ONE

Common Patterns:

# Most common: Just add messages to test user's continuum
def test_tool(authenticated_user):
    user_id = authenticated_user["user_id"]
    continuum_id = authenticated_user["continuum_id"]
    set_current_user_id(user_id)

    # Add test data
    repo = get_continuum_repository()
    msg = Message(role="user", content="Test")
    repo.save_message(msg, continuum_id, user_id)

    # Test your code
    result = tool.run("search", query="Test")
    assert result["status"] == "success"

# API testing: authenticated_client has headers pre-set
def test_api(authenticated_client):
    response = authenticated_client.get("/v0/api/endpoint")
    assert response.status_code == 200

Test User Constants (for reference):

TEST_USER_EMAIL = "[email protected]"
SECOND_TEST_USER_EMAIL = "[email protected]"
# User IDs vary, always use authenticated_user["user_id"]

🔒 Using second_authenticated_user for RLS Testing

The second_authenticated_user fixture provides a SECOND fully-configured test user:

When you need to verify Row-Level Security (RLS) or multi-user scenarios, use both fixtures together:

def test_user_isolation(authenticated_user, second_authenticated_user):
    """Verify RLS prevents cross-user data access."""
    user1_id = authenticated_user["user_id"]
    user1_continuum_id = authenticated_user["continuum_id"]

    user2_id = second_authenticated_user["user_id"]
    user2_continuum_id = second_authenticated_user["continuum_id"]

    # User 1 creates private data
    set_current_user_id(user1_id)
    repo = get_continuum_repository()
    msg1 = Message(role="user", content="User 1 secret data")
    repo.save_message(msg1, user1_continuum_id, user1_id)

    # User 2 tries to access User 1's data
    set_current_user_id(user2_id)
    result = search_tool.run("search", query="secret", max_results=10)

    # Verify User 2 cannot see User 1's data
    assert len(result["results"]) == 0, "RLS violation: User 2 can see User 1's data"

Both fixtures provide identical structure:

authenticated_user = {
    "user_id": str,           # First test user ID
    "continuum_id": str,      # First test user's continuum
    "email": str,             # [email protected]
    "access_token": str       # Valid session token
}

second_authenticated_user = {
    "user_id": str,           # Second test user ID (different UUID)
    "continuum_id": str,      # Second test user's continuum (different UUID)
    "email": str,             # [email protected]
    "access_token": str       # Valid session token
}

When to use second_authenticated_user:

• Testing RLS isolation between users
• Verifying user-specific queries filter correctly
• Testing collaborative features (if MIRA adds them)
• Confirming data leakage prevention
• Any scenario requiring multiple distinct users

Cleanup is automatic for both users:

• Both users' data cleaned before/after each test
• Both user records preserved for reuse
• No manual cleanup needed

SQLite for Tool Testing:

@pytest.mark.schema_files(['tools/implementations/reminder_tool_schema.sql'])
def test_reminder_tool(sqlite_test_db):
    tool = ReminderTool()
    tool.run("add_reminder", title="Test", date="2025-01-01")

    rows = sqlite_test_db.execute("SELECT * FROM reminders WHERE title = ?", ("Test",))
    assert len(rows) == 1

Automatic Cleanup:

• cleanup_test_user_data() runs after each test (autouse)
• Deletes user's data, preserves user account for reuse
• No manual teardown needed

Test Organization - Mirror the Codebase

Test files MUST mirror the codebase directory structure: