Thursday, March 17, 2016
TIME: 12:00 pm - 1:00 pm
PLACE: RMCIC 4th Floor Panther Hollow Room
SPEAKER: Marcus Fontoura,
TITLE: Harvesting Spare Cycles and Storage in Large Scale Datacenters
User-facing, latency-critical services account for a large fraction of the servers in large-scale datacenters, but typically exhibit low resource utilization during off-peak times. An effective approach for extracting more value from these servers is to co-locate the services with batch workloads. Thus, in this paper, we build systems that harvest spare compute cycles and storage space from datacenters for such batch workloads. The main challenge is minimizing the performance impact on the services, while being aware of and resilient to their utilization and management patterns. To overcome this challenge, we propose techniques for giving priority over the resources to the services, and leveraging historical information about them. Our results characterize the dynamics of how services are utilized and managed in ten large-scale production datacenters. Using real experiments and simulations, we also show that our techniques eliminate data loss and unavailability in many scenarios, while protecting the co-located services and significantly improving batch job execution time.
Marcus is currently a Partner Architect at Microsoft, where he works on the end-to-end architecture for Azure compute. In his previous roles at Microsoft, Marcus worked on the production infrastructure for Bing and in several Bing Ads projects. Prior to Microsoft, Marcus was a Research Scientist at Google (2011-2013) where he worked in the Search Infrastructure team. His focus was on the serving systems powering Google.com search. At Google, Marcus worked in many projects including performance and scalability of retrieval engines, novel compression schemes, indexing systems, and networking. He also worked in retrieval techniques for large-scale machine learning systems. Before joining Google, Marcus was a Research Scientist at Yahoo! Research (2005-2010) working on several advertising projects. Marcus also worked as the architect for a large-scale software platform for indexing and content serving, which is used in several of Yahoo!'s display and textual adverting systems. Yahoo! Superstar and was awarded with two Yahoo! You Rock awards. Prior to Yahoo!, Marcus worked at the IBM Almaden Research Center (2000-2005), where he co-developed a query processor for XPath queries over XML streams. This was one of key components of the implementation of the XML data type in the IBM DB2 Relational Database System. In another project at IBM, he was one of the key researchers developing an Enterprise Search Engine that resulted in a new software product for IBM - the IBM OmniFind Enterprise Search. Marcus finished his Ph.D. studies in 1999, at the Pontifical Catholic University of Rio de Janeiro, Brazil (PUC-Rio), in a joint program with the Computer Systems Group, University of Waterloo, Canada. The main contributions from his Ph.D. thesis have been condensed in the book The UML Profile for Framework Architectures, published by Addison-Wesley in 2001. After finishing his Ph.D. Marcus was a post-doctoral researcher in the Computer Science Department at Princeton University for one year (1999-2000). Marcus is an ACM Distinguished Member and an IEEE Senior Member. He has more than 25 issued patents and more than 50 published papers. Marcus has been in several program committees over the years, including SIGIR, WWW, WSDM, KDD, and CIKM. Recently Marcus was a co-chair of the WWW 2013 developers track. You can find more about him at his personal web-site http://fontoura.org.
VISITOR HOSTS: Majd Sakr, Garth Gibson
VISITOR COORDINATOR: Majd Sakr, firstname.lastname@example.org, 412-268-1161
SDI / ISTC SEMINAR QUESTIONS?
Karen Lindenfelser, 86716, or visit www.pdl.cmu.edu/SDI/