SDI Seminar

Speaker: Mitchell Tsai, University of California, Los Angeles

Date: May 20, 1999
Time: Noon
Place: Wean Hall 8220

Designing Systems for Next-Generation I/O Devices


Next-generation user input and sensor technologies (such as speech, handwriting, vision, and physical location) present new possibilities for adaptable and easy-to-use commands. They also create many problems currently either ignored or handled in an "ad hoc" manner: (1) sensor noise, (2) errors from users and sensor processors, (3) ambiguity, and (4) fragmentation among multiple sources. Even a 1% error rate in speech recognition can require the user to spend 20-40% of their time making corrections. Moreover, misinterpreted application commands can easily cause irreversible and dangerous errors. For these reasons, next-generation I/O requires richer communication and interaction between applications, sensors, and sensor processors. To facilitate this style of communication, the computer must begin to analyze and understand the active context of conversations and tasks.

Our system, BabySteps,is a first step towards a set of operating system services for dialogue management the intelligent handling of human-machine communication. One service is the Context Manager, which analyzes user behavior to create and manage contexts. Another, the Command Manager, coordinates and processes input streams into commands. Command Processing modules allow additional functionality, such as safety filters and multimodal integration. BabyStepsuses explicit representations of context to:

(1) support powerful multimodal commands combining speech and mouse input
(2) provide increased safety by filtering out dangerous commands
(3) improve sensor processor performance by providing better knowledge about expected inputs.

Dialogue management can use speech input and application knowledge to help disambiguate different types of mouse movement and actions; e.g. "Make these boxes red." Potentially, explicit contexts can also improve the performance of existing OS services. We can envision the disk drive sending the memory manager a message which means "The disk drive is context 7 (actively reading and writing small files). Are you in cache mode 3 (20 MB write-through cache)?" Since traditional metrics (e.g. accuracy and speed) for evaluating performance are often inadequate, we propose new metrics which involve:

(1) total time required to perform tasks
(2) error taxonomy which measures the negative impact of errors (e.g. Are the errors easily undoable or reversible?).


Mitchell Tsai is finishing his Ph.D. in an Adaptive and Mobile Computing laboratory at UCLA under advisors Peter Reiher and Gerald Popek. In 1986,he received an A.B. in Chemistry and Physics from Harvard University. Afterwards, he became vice-president of FMS, Inc., designing custom database applications for companies such as the International Monetary Fund and the National Academy of Sciences. A growing interest in Artificial Intelligence eventuated in a journey to UCLA for a Ph.D., with some assistance from a National Science Foundation Graduate Fellowship. He spent some time at UCLA working with Bayesian Nets and collaborative robotics, and feels that we're reaching a crucial nexus where many AI technologies will impact real-world systems. His other interests include gymnastics and modern dance, and he is fascinated by multimodal communication using voice and gesture in smart environments for business tasks, family life, and artistic expression.