PDL PROJECTS

TetriScope

Contact: Xiaolin (Charlene) Zang, Alexey Tumanov, Greg Ganger

TetriScope is a combination of the application scheduler TetriSched and the visualization tool Atlas, both integrated into Hadoop's resource manager Yarn.

TetriSched is a high performance plan-ahead scheduler created by PDL researchers. It has been integrated into Yarn to serve as a pluggable scheduler. See the following diagram for a sketch of TetriSched.


[pdf version]

Atlas is web-based user interface developed by PDL. It has been integrated into Yarn's web site to provide a graphic view of the scheduled applications. Although it is initially designed to help researchers refine the TetriSched scheduler, Atlas is a scheduler independent UI, capable of working with any Yarn scheduler.

Following is the TetriScope system diagram, screenshots and interpretation.


[pdf version]

In addition to providing detailed view on how an application is scheduled on various nodes in a data center, Atlas provides a high-level view of aggregated load on per-rack basis for large data centers. It also provides a highly-manipulable timeline for the user to zoom and swipe to adjust the desired time window for viewing the applications.

Following are screencasts of TetriScops showing three scheduling scenarios. They also showcase certain features of Atlas, such as viewing aggregated load and timeline manipulation.

Plan ahead scheduling



For best results, use full screen to play. Click to download a full resolution mp4 version of this video.

Global scheduling



For best results, use full screen to play. Click to download a full resolution mp4 version of this video.

Pre-emption scheduling



For best results, use full screen to play. Click to download a full resolution mp4 version of this video.


People

FACULTY

Greg Ganger
Mor Harchol-Balter

STAFF

Bill Courtright
Xiaolin (Charlene) Zang

GRAD STUDENTS

Jun Woo Park
Alexey Tumanov
Timothy Zhu

INDUSTRY COLLABORATORS

Michael A. Kozuch (Intel Labs)

Publications

Acknowledgements

We thank the members and companies of the PDL Consortium: Broadcom, Ltd., Citadel, Dell EMC, Google, Hewlett-Packard Labs, Hitachi Ltd., Intel Corporation, Microsoft Research, MongoDB, NetApp, Inc., Oracle Corporation, Samsung Information Systems America, Seagate Technology, Tintri, Toshiba, Two Sigma, Uber, Veritas and Western Digital for their interest, insights, feedback, and support.

^TOP

 

 

© 2017. Last updated 29 August, 2016