ABSTRACT

Carnegie Mellon University Parallel Data Lab Technical Report CMU-PDL-08-109, July 2008.

Characterizing HEC Storage Systems at Rest

Shobhit Dayal

School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213

http://www.pdl.cmu.edu/

High-performance parallel file systems are a critical component of the largest computer systems, are primarily proprietary, and are specialized to high end computing systems that have many access patterns known to be unusual in enterprise and productivity workplaces. Yet little knowledge of even the basic distributions of file systems and file ages are publicly available, even though significant effort and importance is increasingly associated with small files, for example. In this paper we report on the statistics of supercomputing file systems at rest from a variety of national resource computing sites, contrast these to studies of the 80s and 90s of academic and software development campuses and observe the most interesting characteristics in this novel data.

KEYWORDS: filesystems, storage, HPC, HEC, statistics

FULL TR: pdf