SDI Seminar

Speaker: Charles P. Shelton, Carnegie Mellon University

Date: February 10, 2000
Time: Noon
Place: Hamerschlag Hall D210

Robustness Testing of the Microsoft Win32 API


Although Microsoft Windows is being deployed in mission-critical applications, little quantitative data has been published about its robustness. We present the results of executing over two million Ballista-generated exception handling tests across 237 C library functions and Win32 API system calls involving six Windows variants, as well as similar tests conducted on the Linux operating system. Windows 98, Windows 95, and Windows CE were found to be vulnerable to complete system crashes caused by very simple C programs for several different functions. No system crashes were observed on Windows NT, Windows 2000, and Linux. Linux was significantly more graceful at handling exceptions from system calls in a program-recoverable manner than Windows NT and Windows 2000, but those Windows variants were more robust than Linux (with glibc) at handling C library exceptions. While the choice of operating systems cannot be made solely on the basis of one set of tests, it is hoped that such results will form a starting point for comparing dependability across heterogeneous platforms.

Charles Shelton is a second year graduate student in the Department of Electrical and Computer Engineering at Carnegie Mellon University. His advisor is Philip Koopman. He received his B.S. in computer engineering in 1998 from the University of Maryland Baltimore County. He has been working on software fault tolerance and fault injection issues in the Ballista project since his arrival here. He is currently finishing his Master's Degree and preparing for the PhD qualifying exam.