Testing Standard

I think unittest should not be used as the primary testing framework for this repository. The only unittest APIs that should remain common are the mocking helpers in unittest.mock such as patch, MagicMock, and AsyncMock.

Why pytest

• The current codebase already uses pytest everywhere in tests/, observal-server/tests/, and observal_cli/tests/.
• Root configuration already enables pytest and pytest-asyncio, so async tests fit the repo without extra boilerplate.
• pytest gives consistent fixtures, parametrization, and clearer assertions for the mix of API, CLI, service, and integration tests in Observal.
• Keeping one test style makes review simpler and reduces friction when contributors move between backend, CLI, and eval tests.

Where unittest still fits

• Use unittest.mock when it is the most direct way to isolate a dependency.
• Do not introduce new unittest.TestCase-based test suites unless a third-party dependency forces that structure.
• If a future test needs compatibility with an external example, keep it isolated and prefer converting it to plain pytest style afterward.

Template Example

Use this as the default shape for new test files.

from unittest.mock import AsyncMock, MagicMock, patch

import pytest

from observal_cli.some_module import SomeService


@pytest.fixture
def service() -> SomeService:
    return SomeService()


def test_happy_path(service: SomeService) -> None:
    result = service.run()

    assert result == "expected"


@pytest.mark.asyncio
async def test_async_flow() -> None:
    dependency = AsyncMock()
    dependency.fetch.return_value = {"ok": True}

    assert await dependency.fetch() == {"ok": True}

File Structure

• Keep unit tests close to the code they exercise, using the existing top-level test directories already in the repo.
• Prefer one file per feature area or endpoint.
• Use descriptive test names that explain the behavior, not the implementation detail.

Style Rules

• Prefer plain assert statements over framework-specific assertion helpers.
• Use fixtures for setup that repeats across multiple tests.
• Use parametrize when the same behavior should be checked across multiple inputs.
• Keep patching small and local to the test that needs it.
• Use pytest.mark.asyncio for async tests.

Running Tests

# Backend and CLI tests
cd observal-server
uv run pytest ../tests/ -q

# Eval tests
uv run pytest ../tests/eval/ -q

# A single file
uv run pytest ../tests/test_agent_composition.py -v

Decision

For Observal, the standard test framework is pytest. unittest is only acceptable as a support library through unittest.mock, not as the default style for new tests.