... architecture underlying the
MATCH system (Johnston et al., 2002).
Multimodal UI
ASR
Server
ASR
Server
Multimodal
NLU
Multimodal
NLU
Movie DB
(XML)
NLU
Model
Grammar Template
ASR
Model
Words
Gestures
Speech
Client
Speech
Client
Meaning
Grammar
Compiler
Grammar
Compiler
F
A
C
I
L
I
T
A
T
O
R
Handwriting
Handwriting
Recognition
... finite-state multimodal language proc-
essing tech...
... structure and content of multimodal communi-
cations, that integrate natural language, pointing,
graphics, and animations. The central claim of
this talk is that the generation of multimodal dis-
course ... useful for multimodal discourse as
well. This means that an interesting methodologi-
cal transfer from the area of natural language pro-
cessing to a much broader computationa...