Writing
Notes from the test runner.
Why pytest collection is expensive, where drop-in claims usually break, and how to measure CI minutes saved without lying to yourself.
-
Measuring CI minutes saved, honestly
It is easy to publish a 'we are 3x faster' number. It is harder to publish one that survives a skeptical reading. Here is the methodology we use for rpytest's CI benchmarks, the mistakes we used to make, and how to measure your own.
benchmarkscimethodology -
Plugin compatibility: where drop-in claims usually break
Every faster-pytest tool says 'drop-in compatible.' Most break somewhere. This post lays out where the breakage usually happens, how we built rpytest to host real pytest so most plugins just work, and what --verify-dropin actually checks.
pytestpluginscompatibility -
What pytest's collection phase costs you
Collection is the phase between 'pytest started' and 'pytest ran a test.' It is also the phase most people are surprised to discover dominates their CLI wall clock. Here is what it actually does, why it is slow, and what removing it looks like.
pytestperformance