Master essential benchmarking techniques, from basic timing to production-ready performance testing.
time
commandtime
is only available on UNIX-based systems, so if you’re working with Windows, you can skip this first step.time
command gives you a bird’s-eye view of your script’s performance, measuring everything from CPU usage to memory consumption.
Let’s start with a practical example. Create a script that demonstrates different algorithmic approaches:
time
command in action:
time
command reveals three crucial metrics:
real
or total
): Wall-clock time from start to finishuser
): CPU time spent in user mode (your Python code executing—loops, calculations, memory operations)sys
or system
): CPU time spent in kernel mode (system calls, file I/O, memory allocation from the OS)hyperfine
time
gives you the basics, hyperfine transforms benchmarking into a science. It runs multiple iterations, provides statistical analysis, and even generates beautiful comparison charts.
After having installed hyperfine, you can get started pretty quicly:
time
run can’t provide.
You can also compare commands with hyperfine:
timeit
timeit
module becomes your microscope. It’s designed to minimize timing overhead and provide accurate measurements of small code snippets.
Here is an example measuring the functions we previously created:
lambda
functions to wrap our calls—this approach is cleaner than string-based timing and provides better IDE support. The data.copy()
call ensures each iteration works with fresh data, preventing any side effects from skewing our results.timeit
lies in its surgical precision. While our previous tools measured entire script execution, timeit
isolates the exact performance characteristics of individual functions. This granular approach becomes invaluable when you’re optimizing specific bottlenecks rather than entire applications.
pytest
:
uv init
and it will turn your directory into a Python project.pytest-codspeed
and transform these correctness tests into performance benchmarks with minimal changes:
benchmark
fixture as a parameter and using it to wrap the execution of the sort algorithm:
pytest-codspeed
automatically determined the optimal number of iterations: 6 runs for the slow bubble sort versus 1,005 runs for the lightning-fast quicksort. This intelligent adaptation ensures statistical significance regardless of your algorithm’s performance characteristics.
Learn more about the plugin in the pytest-codspeed
reference.
time
command when you need a quick sanity check of overall script performance or want to understand system resource usage. It’s perfect for comparing different implementations at the application level.hyperfine
when you need statistical rigor for command-line tools or want to track performance across different input parameters. Its warmup runs and statistical analysis make it ideal for detecting small performance changes.timeit
when you’re optimizing specific functions or comparing different algorithmic approaches. Its focus on eliminating timing overhead makes it perfect for micro-benchmarks.pytest-codspeed
when performance becomes a first-class concern in your development process. It transforms performance testing from an afterthought into an integral part of your test suite.