Quality of Telephone-Based Spoken Dialogue Systems phần 9 pdf

Definition of Interaction Parameters 371 372 Definition of Interaction Parameters 373 374 Definition of Interaction Parameters 375 376 Definition of Interaction Parameters 377 378 Definition of Interaction Parameters 379 This page intentionally left blank [...]... Success Labelling Task success describes whether the aim of the dialogue was achieved or not In order to judge upon task success, the expert makes use of the restaurant slot of the “Modified Scenario AVM”, the restaurant slot of the “Changed AVM”, and the transcription of the dialogue The following categories are available: 1 S: Succeed The aim of the dialogue is achieved and the user obtains the requested... intentionally left blank Appendix C BoRIS Dialogue Structure Figure C 1 1 Dialogue flow in the BoRIS restaurant information system of experiment 6.3, part 384 Figure C.2 Dialogue flow in the BoRIS restaurant information system of experiment 6.3, part 2 For a legend see Figure C.1 BoRIS Dialogue Structure 385 Figure C.3 Dialogue flow in the BoRIS restaurant information system of experiment 6.3, part 3 For a... to be included in the system answer Instructions and Scenarios 397 4 SCuCs: Succeed with constraint relaxation by the user and by the system Combination of the previous two cases: The aim of the dialogue is achieved after the user had to change his/her query, but nevertheless the user was unable to give all specifications The aim of the dialogue can be regarded as achieved if all restaurants provided... please change the following specification: The price Restaurant name(s): 391 Instructions and Scenarios Dialogue no Please gather your information from the following hints: Price: Type of food: Location: Restaurant name(s): If BoRIS doesn’t find a matching restaurant, please change the following: Price: Restaurant name(s): 392 Dialogue no You plan to eat out in Bochum Because your favorite restaurant... functionality The aim of the dialogue can be regarded as achieved when all restaurants given by the system are contained in the restaurant slot of the “Modified Scenario AVM” However, not all restaurants of the “Modified Scenario AVM” need to be included in the system answer 2 SCu: Succeed with constraint relaxation by the user The user made a query which is covered by the functionality of the system, but... “Changed AVM” However, not all restaurants of the “Changed AVM” need to be included in the system answer 3 SCs: Succeed with constraint relaxation by the system The aim of the dialogue is achieved although the user was not able to provide all specifications All restaurants provided by the system have to be part of the “Modified Scenario AVM” However, not all restaurants of the “Modified Scenario AVM” need... have to be interpreted in the context of the restaurant information task It has to be noted that the criteria and recommendations are not strict rules Instead, the evaluation expert often has a certain degree of freedom for interpretation In order to take a decision in an individual case, the expert should consider the objective of the criteria, and the course of the interaction up to the specific... Be perspicuous; avoid obscurity of expression; avoid ambiguity; be brief; be orderly The following categories are available: 1 AP: Appropriate None of the four criteria violated 2 IA: Inappropriate One or more of the four criteria is violated, e.g in the following cases: 398 The system asks a counterquestion or ignores a user query completely Example: U: “Which types of food are available?” S: “At the... question with the aim of putting the dialogue goal forward Example: “What types of food may I choose from?” However, users sometimes ask for feedback on whether the system is still present (e.g “Hello?”) This question should not be classified as a user question because it is not meant to put the dialogue goal forward It has to be interpreted as a spontaneous user reaction in the consequence of excessive response... between bad and excellent In case of an unpredictable extreme judgment, you can use the thinly drawn edges of the scale as well Please also use the spaces between the grid marks, as depicted above Assess the system in a very self-confident way and remember during the whole test session: Not you are tested, but you test our system! And now: Have a lot of fun! 388 D.2 Scenarios Dialogue no You would like to . Definition of Interaction Parameters 371 372 Definition of Interaction Parameters 373 374 Definition of Interaction Parameters 375 376 Definition of Interaction Parameters 377 378 Definition of Interaction. C BoRIS Dialogue Structure Figure C. 1. Dialogue flow in the BoRIS restaurant information system of experiment 6.3, part 1. 384 Figure C.2. Dialogue flow in the BoRIS restaurant information system of. experiment 6.3, part 2. For a legend see Figure C.1. BoRIS Dialogue Structure 385 Figure C.3. Dialogue flow in the BoRIS restaurant information system of experiment 6.3, part 3. For a legend see Figure