Paper Title: Approaching 10M I/Ops on a Single CPU Core

Paper Abstract: One of the most common benchmarks in the storage industry is 4KiB random I/O per second. Over the years, the industry first saw the publication of 1M I/Ops on a single box, then 1M I/Ops on a single core (by SPDK). More recently, there have been publications outlining 10M I/Ops on a single box using high performance NVMe devices and more than 100 CPU cores. SPDK has developed a number of key innovations and is now able to demonstrate more than 9.3M I/Ops on a single thread. To accomplish this, SPDK is employing a large number of advanced techniques not seen in other storage stacks. These techniques include specific, careful management of the CPU cache, advanced MMIO doorbell batching, PCIe and DDIO considerations, and the use of non-temporal CPU instructions. Additionally, there remains a number of techniques that have not yet been employed that warrant future research. These techniques often push devices outside of their original intended operating mode, while remaining within the bounds of the specification, and so often require collaboration between NVMe controller and device designers, the NVMe specification body, and software developers such as the SPDK team.

Paper Author: Ben Walker, Storage Solutions Architect, Intel

Author Bio: Ben Walker is a core maintainer and technical lead at Intel for the Storage Performance Development Kit (SPDK). His work as a co-creator of the project and as a member of the NVMe Working Group has him deeply involved in the development of NVMe, NVMe over Fabrics, Open-Channel, and other advanced storage technologies. Ben has spent the last decade working on high performance storage systems at Intel, pioneering techniques for multi-queue, poll mode storage software designs for low latency media. He holds many storage related patents and is a regular speaker on the conference circuit. Ben holds a B.S. in Physics from Arizona State University.