Skip to content

Troubleshooting

Start with uv run vfy diagnostics. It shows the runtime, configuration, and default native engine; add --json for a bug report. Use --engine=pytest-native to inspect the optional installed-pytest route. Run the commands below from your project root.

uv run vfy diagnostics
uv run vfy diagnostics --json

vfy rejects a test option or plug-in

Choose the collection command that matches the failing engine and test scope:

uv run vfy diagnostics --collect -- tests
uv run vfy diagnostics --engine=pytest-native --collect -- tests

The first command checks the default native engine; the second checks the optional native route, which requires vfy[pytest]. Collection imports test modules and runs collection hooks, but not test bodies. Compare the collected count with pytest, then run the full suite to check fixtures and outcomes.

If vfy reports an unrecognized option such as --testmon, check addopts in pytest.ini, pyproject.toml, or tox.ini, and PYTEST_ADDOPTS. Remove pytest-testmon options from vfy's invocation; they do not configure vfy's affected-test selection. Keep them on a separate pytest command if that workflow still uses pytest-testmon. A clean Git commit and a passing vfy test --all create vfy's selection baseline.

An omitted plug-in can leave a requested fixture undefined, even when the package is installed. Native vfy diagnostics --engine=pytest-native --collect names missing fixtures and counts the tests requesting them. Independent collection can succeed before fixture setup fails, so also run the tests. For example, tests using pytest-aiohttp's aiohttp_client fixture need pytest execution if vfy does not admit that plug-in. Keep the plug-in and run uv run pytest -p no:pytest_cov -p vfy.pytest_plugin --cov=src tests to collect vfy coverage while pytest runs its hooks. Replace src and tests with your paths. Suite adoption explains how to compare results before changing CI.

If the native route rejects a custom pytest.Item or a pytest_runtest_call hook in conftest.py, try the independent engine with the same dependencies. It supports public custom items and call hooks; compare its collection and outcomes with pytest before adopting it. AnyIO tests and plug-ins that need pytest internals stay on pytest with vfy coverage. Narrowing vfy's scope with --ignore omits those tests from the run and disables affected-test selection for that invocation; it does not validate the whole suite.

vfy ran the whole suite when I expected a few tests

Independent selection needs an unfiltered safe coverage run and a passing full run on a clean Git checkout. Explicit selectors, watch, other policies, and --no-cov run the full requested scope. vfy also runs the full requested scope when it cannot justify a subset and prints Note: <reason>. Check that reason before rebuilding the baseline. See full-run reasons.

To establish a baseline, commit your changes and run the full suite with the same coverage and worker options you will use for selected runs. For example:

uv run vfy test --engine=native --all --cov=src   # on a clean Git checkout

A selected run fails

Read the failing test output and rerun that test on its own, or run everything. A failed selected run preserves the passing baseline but writes fresh partial coverage; reports from that failure do not include reused coverage for skipped tests. A passing selected run merges fresh and saved coverage.

uv run vfy test --engine=native --cov=src tests/test_bounds.py::test_positive_value
uv run vfy test --engine=native --all --cov=src

Check the installed build

Print the package version, build profile, and parser backend:

uv run vfy --version
uv run python -c "from vfy.native import build_profile, parser_backend; print(build_profile(), parser_backend())"

Use a release build for performance measurements; a debug build is suitable for development. Include this output when reporting a build or import problem.

sys.monitoring coverage tool id N is already in use

Another collector already holds CPython's coverage tool ID, often Coverage.py, SlipCover, or an auto-loaded pytest-cov. Run only one collector. Under pytest, turn pytest-cov off:

uv run pytest -p no:pytest_cov -p vfy.pytest_plugin --cov=src

With the Python API, pair every start() with stop(), preferably in try/finally or with coverage.collect().

No data file

Check the data path first: the run may have written elsewhere or left shards. Check whether COVERAGE_FILE points somewhere else and whether the effective data_file is the one you expect. Confirm that the measured interpreter exited normally, then inspect the data:

uv run vfy coverage debug data --data-file .coverage
uv run vfy coverage combine --strict --keep

Run combine only for parallel .coverage.* shards.

A source file is missing

vfy reports a file only if it was imported or is under source. Check the resolved settings and where each one came from:

uv run vfy config show

To discover unimported files, set source = ["src"] under [tool.coverage.run] in pyproject.toml. Check whether include or omit filters exclude the file; see file selection. For virtual sources, register the same file-tracer plug-in during reporting that was used during collection.

Totals differ from Coverage.py

A report table can't show where the engines differ. Keep separate data files and compare them in this order: measured_files(), then lines() or arcs() for one differing file, then analysis2() statements and exclusions, then branch_stats(), then context filters, then the report-format summaries.

uv run vfy coverage debug data --data-file .coverage

Cannot combine files

The shards don't match. Common causes: one shard has line data and another branch data, a shard uses a schema other than 7, two plug-ins claim different tracers for one file, the output path was also an input, or --strict found no files. Use --keep while you investigate so the input shards stay available.

uv run vfy coverage combine --keep

Invalid configuration

Run validation to locate the invalid setting, then correct its name, type, or value using the configuration reference. Use uv run vfy config show if the error refers to an unexpected file.

uv run vfy config validate

vfy coverage check reports no semantic data

Check that the command reads the data file from your decision-enabled run. Line or branch data cannot gain semantic observations during reporting. Recollect with --cov-decisions for tests or vfy coverage run --decisions for a script or module, then check. Pass --data-file PATH if collection used another path.

uv run vfy test --all --cov=src --cov-decisions --cov-report=term-missing
uv run vfy coverage check --data-file .coverage

vfy coverage check says the source is stale

A measured source file changed after collection. The check compares saved source digests with the current files, so even an edit outside a Boolean expression can make the data stale. Recollect from the current checkout, then check. Data from different source digests cannot be combined.

uv run vfy test --all --cov=src --cov-decisions
uv run vfy coverage check

A semantic threshold fails even though line coverage is high

Write the semantic report to JSON and inspect the missing outcomes and witness pairs. Line, decision, condition, and MC/DC thresholds are independent; high line coverage does not satisfy the other gates. Add tests for the missing cases, then recollect. See semantic coverage for interpreting the report:

uv run vfy coverage check --format json --output build/decisions.json

Semantic mode reports a plug-in error

Decision mode rejects Coverage.py file-tracer plug-ins, because their virtual source can't map to CPython decision bytecode. Collect line and branch coverage for that plug-in instead: remove --cov-decisions and any configured run:decisions = true (compatibility contract).

uv run vfy test --all --cov=src

A downstream service rejects XML, JSON, or LCOV

Keep the rejected file and the service's error message. Compare the fields your service requires with Compatibility, and regenerate the XML at a fixed path to inspect it.

uv run vfy coverage xml -o build/coverage.xml