The UISpeech System is designed to answer questions that are posed to it in natural language. UISpeech parses incoming questions, matches the queries created from the parse trees against its knowledge base and presents the appropriate information segments to the user. In this way, UISpeech provides untrained users with speedy access to knowledge that in many cases would take an expert some time to find.

A key technique called "natural language annotation" helps UISpeech connect information seekers to information sources. This technique employs natural language sentences and phrases -- annotations -- as descriptions of content that are associated with information segments at various granularities. An information segment is retrieved when its annotation matches an input question. This method allows UISpeech to handle all variety of media, including text, diagrams, images, video and audio clips, data sets, Web pages, and others.

The natural language processing component of UISpeech consists of two modules that share the same grammar. The understanding module analyzes English text and produces a knowledge base that encodes information found in the text. Given an appropriate segment of the knowledge base, the generating module produces English sentences. Used in conjunction with the technique of natural language annotation, these modules put the power of sentence-level natural language processing to use in the service of multi-media information access.