Discuss the following statement: “Multimodal systems are radically different than standard GUIs”
Multimodal Systems aren’t enterely different than GUIs, they build on GUI principles, but their paradigm shift lies in breaking the dependency on screens and hands.
They expand how users can interact by allowing natural, flexible, and inclusive input/output.
Core Element of GUI:
windows, icons, menu, pointers > WIMP model for GUI
not every element has to be a part
Explain the five medium / carrier / sense correspondences
Precisely define the term “modality” (using your own words)
A modality is the combination of a physical signal (e.g. sound) and the human sensory system (e.g. hearing) used to exchange information through a medium. It defines how an user gives input or receives feedback.
Why do we have to differentiate between input and output modalities?
to give a precise definition of the system at hand.
(e.g. spoken computer game >> is it a game controlled by speech? with a speech output? with speech input and output?)
Which modalities are used by interfaces which you use day by day (e. g. smart phone, computer, smart watch, smart TV, ticket terminal, etc.)
Visual: screen
haptic: vibration
auditory: sounds
Explain the two major approaches for categorizing different modalities. Which one is more intuitive for you?
based on the medium or senses modality grouping (graphic, acoustic, haptic)
based on structural modality type (linguistic, analogue, …)
>> the first because it starts with the sensory experience so it feels more like thinking of the user first and not of the system modelling
Please categroize your “day by day modalities” by one of the two approaches.
reading a text message: graphic modality >> static graphic language >> static text (5a) >> typed text (5a1)
typing a message: graphic modality >> dynamic graphic language >> dynamic text (8a) >> typed text (5a1)
listening to podacast: acoustic modality >> acoustic language >> spoken discourse (6a)
using emojis: arbitrary static graphics
Last changed5 hours ago