Automated Disk Drive Characterization
Contact: Greg Ganger
|Sophisticated disk schedulers and detailed disk simulators require
extensive and accurate characterizations of disk drive performance.
Such characterizations include data about mechanical delays,
on-board caching and prefetching algorithms, command and protocol
overheads, and logical-to-physical block mappings. Unfortunately,
accurate characterizations have traditionally been difficult
to acquire, relying on a collection of ad hoc techniques. As
a result, detailed characterizations exist (in the public domain)
for very few disk drives.
To address this problem, we have developed DIXtrac, a program for disk extraction running under Linux. Without human intervention, DIXtrac can automatically extract from SCSI disk drives accurate values for over 100 performance-critical parameters. It runs as a user-level application on linux, using the /dev/sg interface to pass SCSI commands directly to the device driver. DIXtrac uses a collection of pre-programmed test vectors to measure timings for mechanical and command processing overheads. It uses expert-system-like algorithms to identify layout and caching policies. Collectively, these parameters provide sufficient characterization to allow extremely accurate simulation of disk performance.
DIXtrac has been used to characterize six different disk models from three manufacturers. Performing a characterization consists of simply pointing DIXtrac at the disk of interest. The characterization process generally requires less than 3 minutes to complete (5 minutes in one case). Using the extracted characteristics to parameterize the DiskSim simulator, we observe very close matches between simulated and measured disk performance.
From The Newsletter on Parallel Data Systems, Fall 1999.
- NEW: Summer 2008 -- A new version of the DIXtrac code has been released as part of the DiskSim 4.0 simulator.
- Analysis of Methods for Scheduling Low Priority Disk Drive Tasks.
Jiri Schindler, Eitan Bachmat. Proceedings of SIGMETRICS 2002 Conference,
June 15-19, 2002, Marina Del Rey, California.
Abstract / Postscript [237K] / PDF [132K]
- Track-aligned Extents: Matching Access Patterns to Disk Drive
Characteristics. Jiri Schindler, John Linwood Griffin, Christopher
R. Lumb, Gregory R. Ganger. Conference on File and Storage Technologies
(FAST), January 28-30, 2002. Monterey, CA. Also available as CMU SCS
Technical Report CMU-CS-01-119.
Abstract / Postscript [682K] / PDF [159K]
- Automated Disk Drive Characterization. Schindler, J. and
Ganger, G.R. CMU SCS Technical Report CMU-CS-99-176, December 1999.
Abstract / Postscript [341K] / PDF [282K]
- DiskSim: An efficient, accurate, highly-configurable disk system simulator.
We thank the members and companies of the PDL Consortium: Broadcom, Ltd., Citadel, EMC Corporation, Facebook, Google, Hewlett-Packard Labs, Hitachi Ltd., Intel Corporation, Microsoft Research, MongoDB, NetApp, Inc., Oracle Corporation, Samsung Information Systems America, Seagate Technology, Tintri, Two Sigma, Uber, Veritas and Western Digital for their interest, insights, feedback, and support.