Skip to content

Collect and report coverage

Collect lines and branches, save .coverage, and generate reports without rerunning tests. Install vfy in your CPython 3.12+ test environment. Run commands from the project root and replace src with your source directory. Coverage and test execution support Linux and macOS.

Run tests with coverage

Check suite compatibility before adopting vfy. vfy test supports a limited set of runtime plug-ins.

uv run vfy test --all --cov=src --cov-branch --cov-report=term-missing

This runs the full suite, saves .coverage, and prints missing lines and branches. vfy test collects coverage by default and records per-test contexts. For affected-test runs, follow test selection; keep the same collection settings between the baseline and selected runs.

Collect a script or module

uv run vfy coverage run --branch --source src app.py
uv run vfy coverage run --branch --source src -m pytest

Use your entry point in place of app.py. Each command saves .coverage and returns the program's exit status. Arguments after the script or module name pass through unchanged. Existing Coverage.py jobs can replace coverage with vfy coverage for run, report, combine, erase, and file-report commands.

Choose files and collection mode

--source includes unexecuted source files in reports; repeat it for more roots. Narrow paths with glob filters:

uv run vfy coverage run --source src --include '*/service/*.py' --omit '*/generated/*' -m pytest

Without an explicit source or include filter, vfy skips the standard library and installed packages. [run] cover_pylib = true or the Python API's cover_pylib=True also measures the standard library; installed packages remain excluded. Collection defaults to lines. --branch records arcs, including function entry and exit; a data file contains lines or arcs, not both. For condition outcomes and short-circuit paths, use --decisions (or pytest's --cov-decisions), which also enables branches. See semantic coverage for collection, JSON output, and gates.

Append or erase data

Runs replace saved data by default. To accumulate compatible runs from two scripts (replace their paths with yours):

uv run vfy coverage run --append --source src scripts/check_units.py
uv run vfy coverage run --append --source src scripts/check_integration.py
uv run vfy coverage erase --parallel

The last command deletes the main database and its shards; omit --parallel to delete only the main file. Both vfy test and stock pytest with vfy's coverage plug-in accept --cov-append. Appending with vfy test runs the full requested scope without test selection. Combine only data from the same source revision and collection mode.

Generate reports and enforce a threshold

After collection:

uv run vfy coverage report --show-missing --precision 1 --fail-under 95
uv run vfy coverage html --directory htmlcov --title "Project coverage"
uv run vfy coverage json -o coverage.json --pretty-print --show-contexts
uv run vfy coverage xml -o coverage.xml
uv run vfy coverage lcov -o coverage.lcov
uv run vfy coverage annotate --directory annotations

The first command prints totals and fails with status 2 below 95%; invalid configuration or data exits 1. pytest's --cov-fail-under=95 instead fails with status 1. The remaining commands write the named files. All formats share exclusions, context filters, and totals. Use --cov-report= during a test run to save data for a later report step.

Filter reports with --include, --omit, or repeatable --contexts; hide files without statements with --skip-empty. Terminal reports also support --format=markdown and --format=total. See the CLI reference for flags and sort values, and Python API for report methods.

Report content and source limits

JSON contains line and branch totals in Coverage.py's format-3 shape plus functions and classes. XML contains rates, counts, package groups, and branch condition coverage; check it with the service that consumes it. LCOV contains per-file line hits, branches, and summary counts. Annotated source prefixes are > for executed statements, ! for missing statements, - for excluded statements, and two spaces for non-executable source.

HTML writes an index, stylesheet, and one page per file. Each file is replaced atomically; a failed write can leave a mix of old and new files. Reports read source at its current path. A file-reporter plug-in can supply virtual source; test reports for generated or deleted files before making them a gate.

Find which tests reach a line

Per-test contexts connect measured lines to test node IDs. Selection and mutation testing use that mapping; ordinary coverage or one static run label cannot identify the reaching tests. vfy test records contexts automatically. For stock pytest, collect a full suite explicitly:

uv run pytest -p no:pytest_cov -p vfy.pytest_plugin --cov=src --cov-context=test --cov-report=
uv run vfy explain src/service.py:42
uv run vfy coverage report --contexts '^tests/unit/'

Replace src/service.py:42 with an executable source line. explain lists tests that reached it, fastest first when durations are available; it does not run those tests or explain selection decisions. With semantic data, it also shows the pairs of recorded runs that demonstrate a condition changing a decision's outcome (MC/DC; see semantic coverage). If neither reaching tests nor such pairs are recorded, it exits with an error.

Contexts label each test phase as nodeid|setup, nodeid|run, and nodeid|teardown, for phases that ran. Schema-7 data and combined xdist shards retain them. Report filters are regular expressions; they filter lines and arcs before calculating totals and leave saved data unchanged.

To label an entire run, use vfy coverage run --context unit .... With per-test contexts too, the label is unit|nodeid|phase. For programmatic labels and dynamic frame contexts, see the Python API.

Reuse Coverage.py configuration

vfy reads .coveragerc, .coveragerc.toml, setup.cfg, tox.ini, and pyproject.toml. COVERAGE_FILE chooses the data basename; COVERAGE_RCFILE chooses the configuration. For example:

[tool.coverage.run]
branch = true
source = ["src"]

[tool.coverage.report]
show_missing = true
fail_under = 90
uv run vfy config show
uv run vfy config validate

These print resolved settings and validate their types and supported values. Unknown sections and keys are errors. Keep vfy-only semantic settings under [tool.vfy.*]. See configuration for discovery, precedence, path aliases, exclusions, format destinations, and plug-in options.

vfy reads and writes Coverage.py 7 schema-7 SQLite data. To compare engines, run the same suite with separate COVERAGE_FILE values, then compare files, lines or arcs, contexts, exclusions, totals, and artifacts. Private APIs and dumps() bytes are outside Coverage.py compatibility.

Collect workers and subprocesses

With pytest-xdist 3.8.0 installed:

uv run pytest -p no:pytest_cov -p vfy.pytest_plugin -n 2 --cov=src --cov-branch --cov-context=test --cov-report=term-missing

The controller combines worker shards into .coverage before reporting; a complete run needs no separate combine step. vfy test --engine=pytest-native can use xdist with --numprocesses=2 --dist=loadfile; see worker options for restrictions on combining worker modes. A worker crash or malformed shard preserves the previous database and report. Failing tests still save valid data.

Python subprocesses

uv run vfy coverage run --parallel-mode app.py
uv run vfy coverage report

Replace app.py with your program. Python children inherit collection bootstrap settings. Spawned and exec'd children save a shard on normal exit; forked children restart the collector and save their own shard. The report combines discovered shards. Application sitecustomize hooks run once under their usual module name, before usercustomize; CPython handles their exceptions normally. Code imported by an application startup hook is measured when it matches the collection filters. Preserve the child's Python environment; do not set vfy's managed environment variables yourself.

Collection saves on normal exit, including plain SystemExit. Set [run] sigterm = true to save before preserving SIGTERM's usual exit status. SIGKILL, os._exit, interpreter crashes, and power loss cannot save pending data.

Recover interrupted shards

uv run vfy coverage combine --strict --keep
uv run vfy coverage debug data --data-file .coverage.worker

--strict errors when no shards exist; --keep retains inputs. vfy validates all shards before replacing .coverage; failure leaves data and shards unchanged. Replace .coverage.worker with a real shard path. If a worker is missing, check plug-in loading (add -p xdist.plugin when autoload is disabled), child environment, and .coverage.* files before combining. Standalone runs need --parallel-mode to write the parent as a shard.

Use a Coverage.py plug-in

Install the plug-in in the collection and reporting environment and configure its module under [run] plugins. File tracers, file reporters, configurers, and dynamic contexts use the Coverage.py plug-in API. These differ from pytest plug-ins loaded with -p.

Reports reload the tracer named in saved data, so the module must remain importable. Compare your plug-in's virtual filenames, translated lines and arcs, unexecuted-file discovery, exclusions, source rendering, contexts, and saved tracer records against Coverage.py before changing CI. See the plug-in checks for the protocol tests and third-party package reports.

Collect with stock pytest

Keep your existing pytest runner and measure coverage with vfy:

uv run pytest -p no:pytest_cov -p vfy.pytest_plugin --cov=src --cov-branch

This saves .coverage and prints a terminal report. pytest's -p option loads vfy.pytest_plugin; -p no:pytest_cov blocks pytest-cov. vfy does not activate automatically, so plain pytest --cov uses pytest-cov when installed. Other plug-ins, including pytest-html and pytest-metadata, load as usual. vfy's own HTML/metadata replacements are a separate opt-in: -p vfy.pytest_reporting. See reporting compatibility before enabling them.

For repeated runs, add the flags to pyproject.toml, preserving existing addopts, or set them in PYTEST_ADDOPTS for your shell or CI job:

[tool.pytest.ini_options]
addopts = "-p no:pytest_cov -p vfy.pytest_plugin"

pytest reads these flags before installed plug-ins load. See compatibility for supported versions.

For flags, report destinations, and the cov and no_cover fixtures, see pytest coverage options.