Architecture¶
vfy implements Coverage.py and pytest interfaces with Rust coverage and test engines. Use this page to locate the component responsible for a change and check the invariants it must preserve.
Components and ownership¶
| Component | Owns |
|---|---|
python/vfy/ |
Public Coverage.py and pytest protocols, pytest adapters, stdout, and calls into third-party plug-ins |
python/vfy/cli*.py |
CLI parsing, command dispatch, diagnostics, and report output |
python/vfy/_pytest/ |
Private adapters between installed pytest and the native engine: collection, caches, JUnit, xdist transport, and run preparation |
python/vfy/run/ |
The vfy test command: engine choice, selection and replay, CI baselines, and watch mode |
python/vfy/independent/ |
The pytest-compatible interfaces the independent engine provides without installed pytest |
vfy-core |
The PyO3-free data, analysis, configuration, selection, mutation, and report engine |
crates/vfy (vfy._native) |
CPython monitoring and object conversion |
crates/vfy-pytest |
Test collection and execution, built into the same extension |
Rust also owns coverage collection, scheduling, aggregation, rendering, and
atomic filesystem commits. By default, vfy collects and runs tests with its independent
engine. The optional --engine=pytest-native mode uses installed pytest for
collection. Python adapts CPython and plug-in protocols for both engines.
Invariants¶
- Monitoring events go straight to Rust. Python is not on the event path.
- One Rust data and report model. Every renderer uses the same totals. No Python
fallbacks or duplicate algorithms.
pytest.approxis the exception: its tolerance checks call__eq__,__sub__, and__abs__on arbitrary user objects, sopython/vfy/independent/approx.pyimplements it in Python. - Validate data and report inputs before replacing destination files. Each file replacement is atomic; multi-file publication prepares rollback copies and reports any backup that could not be restored.
- Persisted and public ordering is deterministic.
- Stock pytest is a differential oracle, never an automatic fallback.
Runtime boundaries¶
Monitoring events enter Rust directly. Configured third-party Python plug-ins cross back into Python through the adapter.
Scroll horizontally to view the full diagram.
Python reporting adapters call configured file reporters separately.
Async pytest boundary¶
vfy.asyncio_plugin is the installed-pytest asyncio adapter. It classifies
callables per session, rechecks their code after instrumentation, and creates lazy
asyncio.Runner instances through pytest fixtures. In native runs, Rust plans
fixture dependencies and loop scopes. Python awaits coroutines, propagates context,
finalizes generators, and closes runners.
Test execution¶
vfy test --engine=native owns collection, fixture resolution, execution,
and reporting without requiring pytest. Its Python facade exposes the pytest
interfaces used by test suites and adapts Python objects to Rust operations.
Compatibility names identify those public protocols; they do not delegate
execution to pytest. See compatibility limits.
Invocation preparation and execution¶
The independent and selection controllers share a Rust plan for scope, workers, scheduling, and replay. Python supplies interpreter-specific locations, paths, and resolved pytest configuration. Each invocation owns its child processes, temporary results, coverage, and any pending dependency fingerprint scan.
Native lifecycle owners¶
Native capture owners retain redirected descriptors, stream objects, and phase state. Rust reads captured output; Python stream and logging operations remain observable through their public protocols. Logging handlers and their shared formatter belong to the configured session. Each test owns separate phase records, and phase guards unwind handler attachments on errors.
Each native fixture binding owns its callable, scoped outcome, and finalizers.
Reading cached_result exposes the same authoritative outcome as a public tuple;
later public replacements remain visible to fixture resolution and teardown.
Report history stores phase values under stable IDs. Public hooks and readers observe the same Python report objects and their mutations. Serial execution reads live reports at final publication so hook edits reach machine results; workers and JUnit writers retain the projections required at their earlier publication boundaries.
Hook dispatch uses the active conftest registrations; when none exist, it omits item-path filtering. Registration changes update that decision. Calls into tests, fixtures, plug-ins, descriptors, and streams cross the actual CPython boundary; native ownership preserves the public objects and callbacks observed there.
Native collection¶
vfy test --engine=pytest-native uses installed pytest to collect the test source. The
adapter compiles it into a validated Rust plan and Python bindings. Rust resolves
and runs fixtures. CPython calls the bound test and fixture functions.
The plan holds typed test and fixture IDs, scope and dependency chains, and validated references. The adapter keeps pytest items, callables, and live parameter values in aligned binding tables. Dynamic fixture discovery crosses this boundary, and Rust updates the plan before execution. A parameter change invalidates the affected dependency closure, and teardown runs in reverse setup order.
Each fixture has one live cached instance. A cache hit keeps the instance's original owner. Owners unwind from inner to outer, and finalizers run in reverse registration order within each owner. A test reads the live callable and its code after fixture setup.
Workers¶
One worker is the default. Native file-group workers (-n, -j, or
--workers) fork the collected session when eligible. Each worker owns fixture
lifetimes, capture files, and an empty coverage collector. Otherwise, fresh workers
collect independently; the controller checks collection agreement. It merges
results in collection order, combines collection coverage with worker shards, and
publishes final reports. Workers never publish them independently.
Pytest's --numprocesses selects pytest-xdist scheduling and transport. Each xdist
worker keeps a native session across dispatches and emits pytest phase reports.
An explicit fork request rejects an ineligible run. See
worker start policy
for eligibility and collection overrides.
A passing full run on a clean checkout stores source and execution identities with the baseline used by selection and mutation.
Stock pytest oracle¶
Stock pytest execution is an internal differential oracle. It shares the Rust coverage model, and pytest runs the tests. Rust keeps a bounded failure-confirmation history for both routes. The override is hidden and is never an automatic fallback.
Native HTML reporting passes the execution result handle to the Rust renderer. Python supplies metadata and supported customization-hook values. Attachments travel with validated worker result records, and the controller publishes the merged report.
Watch execution ownership¶
The Rust watch controller handles file events, debounce, run identity, recovery, and publication. One run is active per session. Before planning, the controller captures a content-based identity, and the caller cannot substitute another at publication. Python adapts pytest options, hooks, signals, and output, and starts a fresh test process for each run.
Rust checks file contents and delivered events before it accepts a result. An invalidated run stays obsolete, even if the edit is reverted. Accepted coverage stores its input identity, so later selection rejects coverage from different source. Only a current, passing full run clears recovery state.
Publication validates every staged artifact and destination first. Each file is replaced atomically, with rollback copies prepared on disk. If publication fails, or the run goes obsolete during finalization, Rust restores every replaced file. A failed restore does not stop the others, and the error names any backup it kept. Publication and completion share one commit point. Edits after acceptance belong to the next run.
The adapter releases the GIL while Rust waits for changes or publishes.
The Rust handle boundary¶
Readers and collectors return a shared Rust handle. Reports use it directly. A mutable API
call or a configured Python file reporter materializes FileCoverage views (dashed path).
Ordinary reporting never does.
Scroll horizontally to view the full diagram.
Rust resolves configuration and path aliases, combines shards, and commits report files. Python writes stdout or caller-owned file objects, and calls third-party reporters. Mixed line and branch data fails validation before any output changes. Every renderer uses the same Rust totals.
Collection hot path¶
Monitoring callbacks record observations without parsing source or walking branch
trails. PY_START resolves and caches file identity; LINE records a hit in the
current context. Line locations disable after their first hit and reactivate when
collection starts or context changes. Code compiled before and after collection starts
is collected.
Branch strategies preserve frame state for suspension and exception transitions. Direct branch collection deduplicates bytecode edges and resolves logical statement destinations at snapshot time. Event retirement is separate from frame lifecycle.
Interned contexts, sparse 64-line chunks, compact arc pairs, and copy-on-write partitions avoid dense allocation and duplicate unchanged data. Schema-7 line reads share equal line sets among the contexts of each file; later updates copy on write. Public API and schema-7 records materialize deterministic strings and sorted observations.
The extension holds coverage data behind an immutable shared handle. Reports acquire that handle before releasing the GIL. Successful mutations detach their copy; fallible mutations replace the shared value only after validation.
Source analysis reuse¶
Immutable source analysis is cached by source bytes, format version, exclusion and partial-branch expressions, branch mode, and interpreter annotation semantics. The cache is bounded by entries and estimated bytes. Compiled regular expressions bypass it because their text does not capture every matching option. New coverage observations rebuild aggregation without reparsing source.
After fork(), a child avoids the inherited cache mutex and analyses without
caching until exec. File selection precedes context matching; both filtering and
analysis run outside the GIL. Context validation precedes source analysis, with
errors ordered by context and path.
Semantic coverage collection¶
Decision coverage shares the collector and report model. It does nothing unless enabled. Source analysis defines stable decision and condition IDs. Monitoring events contribute vectors of true, false, or not-evaluated values. The report evaluator uses these vectors to check decision, condition, and MC/DC coverage without re-running user expressions.
The collector never calls bool(), keeps condition objects, or rewrites bytecode. If
condition evaluation raises, the incomplete invocation is discarded. If a short-circuit
operand is skipped, the vector records not evaluated, so masking analysis cannot claim
unobserved evidence.
Mutation-testing path¶
vfy mutate checks whether the suite detects changes to source expressions. It applies
each mutation in a child process and never edits source files.
Scroll horizontally to view the full diagram.
Rust builds the catalog, ranks tests from coverage contexts, and keys the cache. The Python adapters handle CPython and pytest lifecycle hooks and print results. A collected session is reused only when the mutation owner supports a code-object swap, POSIX fork is available, no other threads are active, and portable execution was not requested. The source hash and exact UTF-8 byte range must match before launch and during import, or the mutation is rejected. A test failure counts as protection only if the unchanged source passes the baseline tests. Import-time mutations run the full configured test scope. Body mutations can select tests from recorded coverage contexts.
Test selection¶
Rust compares Git changes with a passing full-run baseline. It joins edited source ranges, per-test coverage, and reverse import dependencies to produce a selection plan. With scoped collection, the plan names the test files the independent engine collects. The other files keep their baseline test IDs. Passing selected runs combine fresh coverage with saved data for skipped tests; failed runs keep the passing baseline. The selection guide owns the rules, fallback conditions, and limits.
Ruff parser and import graph¶
Coverage analysis, mutation discovery, and import-based selection use Ruff's
ruff_python_parser and ruff_python_ast at a pinned version. Their Rust APIs are
unstable, so the crates are exact-pinned. Updating them requires the full differential
analysis suite. The parser stays out of monitoring callbacks.
The import graph parses ordinary imports with the Ruff AST, resolves files within
configured source roots, computes reverse dependents, caches fingerprints, and joins file
identities to observed pytest node IDs. Dynamic imports such as importlib and
__import__ are not seen, so selection can miss them. Reflection, environment-based
resolution, and type-aware import semantics are outside this contract.