An increasing number of telephone services are offered in a fully automatic way with the help of speech technology. The underlying systems, called spoken dialogue systems (SDSs), possess speech recognition, speech understanding, dialogue management, and speech generation capabilities, and enable a more-or-less natural spoken interaction with the human user. Nevertheless, the principles underlying this type of interaction are different from the ones which govern telephone conversations between humans, because of the limitations of the machine interaction partner. Users are normally able to cope with the limitations and to reach the goal of the interaction, provided that both interlocutors behave in a cooperative way.
The present book gives a systematic overview of assessment, evaluation, and prediction methods for the quality of these innovative services. On the basis of cooperativity considerations, a new taxonomy of quality of service (QoS) aspects is developed. It identifies four types of factors influencing the quality aspects perceived by the user: Environmental factors resulting from the physical situation of use (transmission channels, ambient noise); factors directly related to the machine interaction partner; task factors covering the interaction goal; and non-physical contextual factors like the access conditions and the involved costs. These factors are shown to be in a complex relationship to different categories of perceived quality, like cooperativity, efficiency, usability, user satisfaction, and acceptability. The taxonomy highlights the relationships between the different factors and aspects. It is a very useful tool for classifying assessment and evaluation methods, for planning and interpreting evaluation experiments, and for estimating quality on the basis of system characteristics.
Quality is the result of a perception and a judgment process. Consequently, assessment and evaluation methods involving human test subjects are necessary in order to quantify the impact of system characteristics on perceived quality. The system characteristics can be described with the help of interaction parameters, i.e. parameters which are measured instrumentally or on the basis of expert annotations. A number of parameters and evaluation methods are defined, both on a system component level and for the fully integrated system. It is shown that technology-centered component assessment has to go hand in hand with user-centric evaluation, because both provide different types of information for the system developer. The resulting information about quality is needed in all phases of system specification, design, implementation, and operation, in order to efficiently set up systems which offer a high quality to their users.