Benchmarking Framework Sheds Light on Quantum Software Performance

Nature Computational Science has published a new study introducing Benchpress, a benchmarking framework designed to evaluate the performance of quantum software development kits (SDKs) at scale. As quantum computing advances toward practical applications with devices approaching 100 qubits, this research aims to address a gap in the field. “How well current classical software infrastructures can support quantum computation.”

Quantum computers, while often highlighted for their potential advantages over classical systems, remain reliant on classical computing for key pre- and post-processing steps. As the paper notes “The promise of quantum computation lies in its ability to perform specific tasks more efficiently than otherwise possible using classical methods alone. However, quantum computers do not operate in isolation and require classical computing resources for data pre- and postprocessing.”

Despite the growing ecosystem of quantum SDKs, such as Cirq, Qiskit, Tket, and others, there has been no systematic method to compare their effectiveness in building and optimizing quantum circuits. Previous benchmarking efforts have been limited by inflexible formats and scalability issues. Benchpress aims to change that.

According to the study, Benchpress provides a “common framework that allows testing across three key areas; quantum circuit construction, manipulation and optimization.” It includes over 1,000 individual tests and supports multiple SDKs through a modular design. This allows researchers to assess not just performance, runtime, memory usage, and circuit quality, but also the breadth of functionality across platforms.

The tool relies on Qiskit for much of its internal infrastructure, due to its compatibility with a range of SDKs and its support for OpenQASM, a widely used intermediate representation of quantum circuits. This strategic choice enhances interoperability and reduces the risk of inconsistencies in comparisons.

The authors emphasize that Benchpress is open source, which they hope will encourage community engagement and provide a transparent basis for evaluating quantum software as hardware capabilities evolve. Given that fully fault-tolerant quantum computing is still some years away, tools like Benchpress serve as an important step toward making near-term quantum computing more accessible, reliable, and scalable.