Kapitel 1 BMW iDrive

• Users typically mix unimodal with multimodal expressions; often do not use multiple senses

• only 14 % of multimodal gestures are speech-and-pointing gestures

• ~ 25 % of users commands actually contain a spoken input that overlaps with pointing; people don’t use multiple input modes at the same time

• non-speech modes occur before speech in 99% of the time

• multimodal expressions are more direct, shorter, linguistically unique and often easier to process

• multimodal inputs are usually complementary; barely redundant

• multimodal systems often have better error handling

• users differ in how they combine inputs > system should detect and adapt to useres preference

• modes like speech, handwriting, gestures, … are not interchangeable > input modes should be treated as specialized tools

• Efficiency is not main advantage > fewer errors (36-50% reduction), user preference for flexibility, adaption to different tasks, accessability and mobile use

• multimodality leads to better UX design

• it can enhance usability (product used by specified users in a specific context, reach specific goals, efficiently and effectively, achieve satisfaction)

• people naturally communicate with multiple senses

(Process from DIN EN ISO 9241-210)

1. Understand and specify the context of use > Who are the users, What are they doing, Under which conditions

2. Specify the user requirements > What does the unser need the system to do

3. Produce design solutions to meet these requirements > create concepts mockups and prototypes

4. Evaluate teh design against requirements > usabiltity testing, simultanious, feedback

> Iterative Process!

Users:

• Age / Personality

• Smartphone user

Tasks:

• primary tasks (talking to others via text or voice)

• secondary task (using apps / internet)

• tetiary tasks (entertainment / information)

Equipment:

• display

• camera

• sensors

• speakers

• (buttons?)

Surroundings:

• technical (computing power, sensors, camera)

• physical (environment of user > quiet, loud, traffic, time of day (lighting))

• social (values, surrounding people)

• functionallity > function with specified characteristics

• usability > used by user group

• efficiency > ratio of information needed from user and time to put information in

• security > handeling private data

• safety > expected functionalily

• 
  

Naviagion to destiation (Modalities):

• 1 / 2 iDrive Controller (speller/touch)

• 3 Direct Touch

• 4 Voice Input

• 5 Shortcut keys

Interations:

• 5 can not be used in multimodality

• switch from 1/2 to 3:

  • problem: want to touch sth on keyboard (screen) > switch to keyboard on screen > proximity sensor

  • switch back 3 > 1/2: touch/turn iDrive Knob (not able to use proximity sensor bc keyboard would constanly switch)

• input loss when switch from 1-3 to 4: small progress is lostl (state of selection)

• input loss when switch from 4 to 1-3: usually loss of whole progress

> integrate the user > test it!

Input:

• interactions between modalities

• switching between modalities (input loss)

• modality can be used multimodal, or not?

Output:

• environment (brightness, noise, where is person > can see screen, mental load (e.g. driving))

• switch modality during Output

• Does modality make sense as output modality in context?

• explain picture:

• Validity

• Phase of development

> yes (if done correctly)

Example on iDrive:

• Combining Strengths:

  • Efficientcy (touch screen)

  • Touchless (language)

  • Simplicity (shortcuts)

• Compensate weaknesses:

  • noisy envrionments (language)

  • social barriers (gestures)

  • poor accessibility (touchscreen)