, November 15, 2001
The dynamic nature of Internet services makes it difficult to engineer this problem away. Simply overprovisioning resources is infeasible when load spikes are orders of magnitude greater than the average. The load that a given service places on its underlying resources is also unpredictable, given that most services involve a combination of static content, dynamic scripting, database access, and other code. Traditional thread-based models of concurrency are designed for timesharing and do not scale to many thousands of simultaneous tasks.
We present a general-purpose software architecture that addresses the problem of supporting massive concurrency and load conditioning in a highly dynamic environment. We describe the Staged Event-Driven Architecture, or SEDA, in which applications consist of a network of event-driven stages connected by explicit queues. This architecture allows services to be well-conditioned to load, preventing resources from being overcommitted when demand exceeds service capacity. In addition, SEDA makes use of a set of dynamic resource controllers to keep stages within their operating regime despite large fluctuations in load.
This talk presents the SEDA design as well as an implementation of an Internet services platform based on this architecture. We discuss control mechanisms for automatic tuning and load conditioning, including thread pool sizing, event batching, and adaptive load shedding. Finally, we present several applications constructed using the SEDA framework, including a high-performance HTTP server that outperforms both Apache and Flash; a packet router for the Gnutella peer-to-peer file sharing network; and a Web-based email service. Our results show that SEDA applications exhibit higher performance than traditional service designs, and are robust to huge variations in load.
Further Seminar Info: