Paper Title: Enabling IO Determinism in Hyperscale Data Centers

Paper Abstract: SSD I/O isolation and latency control in multi-tenant cloud applications have become increasingly important, especially as the capacity of the physical drives grows larger. Working with the major cloud providers and SSD vendors, Facebook has led the industry to standardize two NVMe Technical Proposals that strive to tackle the problems. NVM Sets and Read Recovery Levels enables partitioning of a physical SSD into multiple independent quality of service (QoS) isolated NVMe namespaces. I/O Determinism (IOD) gives the host software stack the control of SSD background operations in order to achieve deterministic I/O behavior. This talk presents a Linux-based software architecture and the test results using Facebook workloads that improves the read I/O latency distribution while leveraging the NVMe Sets and I/O Determinism standards. In a nutshell, the architecture builds on top of the basic RAID developed using IOD-capable SSDs. By mediating the deterministic and non-deterministic windows of the SSDs in the RAID group as defined in the I/O Determinism specification, we can service the read I/O from the SSDs in the deterministic window.

Paper Author: Chris Petersen, Hardware Systems Technologist, Facebook

Author Bio: Chris Petersen is a Hardware Systems Technologist leading flash and non-volatile memory solutions at Facebook. Chris has been designing and building servers, storage, and datacenter solutions for over 14 years. He has worked on a wide variety of architectures, including scale-up, scale-out, and containerized datacenters. Chris has been instrumental in the definition of “Lightning”, the NVMe JBOF (Just a Bunch of Flash) design that accepts SSDs in several form factors. He has also worked on standardizing the Lightning idea for the Open Compute Project. Chris is a board member of NVM Express, Inc, the organization that that developed the NVM ExpressTM specification for accessing SSDs on the PCIe bus. He has evangelized both NVMe in general and Lightning in particular at many conferences, including the Open Compute Summit and the Non-Volatile Memory Workshop. He has been granted six patents. Before joining Facebook, he held engineering positions at Dell and HP. He earned an MS and BS in electrical and computer engineering at Cornell University.