Thursday, May 10 , 2007
12:00 pm - 1:00 pm
Eliminating cross-server operations in scalable file systems
Distributed file systems that scale by partitioning files and directories among a collection of servers inevitably encounter cross-server operations. A common example is a rename that moves a file from a directory managed by one server to a directory managed by another. Systems that provide the same semantics for cross-server operations as for those that do not span servers traditionally implement dedicated protocols for these rare operations. This paper suggests an alternate approach that exploits the existence of dynamic redistribution functionality (e.g., for load balancing, incorporation of new servers, and so on). When a client request would involve files on multiple servers, the system can redistribute those files onto one server and have it service the request. Although such redistribution is more expensive than a dedicated cross-server protocol, the rareness of such operations makes the overall performance impact minimal. Analysis of NFS traces indicates that cross-server operations make up less than 0.006% of getattr client requests, usually much less, while experiments with a prototype implementation show that the performance impact is minimal when such operations are much more frequent (e.g., 0.05% of operations). Thus, when dynamic redistribution functionality exists in the system, cross-server operations can be handled with very little additional implementation complexity.
Shafeeq is a 5th year graduate student in the Parallel Data Lab. His current research interest is in metadata management for large-scale Self-* storage systems. Personal interests include hiking, photography, and trying to find the Zen in motorcycle maintenance.
or visit http://www.pdl.cmu.edu/SDI/