Chapter 7: Virtual Environments

Virtual environments as interactive, virtual image displays enhanced by special processing and by non visual display modalities, such as auditory and haptic, to convince users that they are immersed in a synthetic space.

• Multiple users

• Interact with each other in Real time

• Works even if users are located around the world

• New type of media

• Generates realistic images/sounds/other sensations, replicate real environment

• Simultates the user in the environment and lets him*her interact with it

• Using specific display technology and projectors

• Typically Head-mounted-Display (HMD)

• 2 google sized screens, one per eye

• Angeling 2D pictures to mimic human sight -> stereoscopic 3D image

• Needs to track head position and orientation with a tracker

Synthetic content = syntheticaly created out of bacic 3D shapes + simulated physics

Captured content = imaging techniques

Uncomfortable symptoms when using a VR system. Intensity can differ from person to person

• Reducing the delay between motion and rendering of corresponding sensory information

• Improve screen refreshing rate

• Keep sensory information as synchronized as possible

• Interaural time difference (ITD) -> low freq.

• Interaural level difference (ILD) -> high freq.

• Dynamic changes of ITD and ILD

• Mimic human anthropometry

• Produce some sort of average HRTFs

• Upside: Straightforward

• Downside: Head-tracking cannot be applied upon playback/ dynamic changes due to head movements cannot be applied

-> customizing Head-Related Transfer Functions to match an individual's unique anatomy—like the shape of their ears, head, and torso

Generic HRTFs can lead to:

• Poor sound localization (e.g. front-back confusion)

• “Inside-the-head” audio effects

• Less immersive experiences in spatial audio

BUT: not essential for all tech

• Ventriloquist Effekt

• brain automatically matches visual + auditory cues to a certain degree

Uses amplitude and timing differences to produce phantom sound sources between the speakers

Downside: Works only reliably with the listener facing the speakers

• Projection surfaces are not acoustically transparent

• Strong reflections occur

• String resonances occur

The Soundfield Microphone

• (Near-)Coincident microphone capsules

• Capture the sound pressure and the sound velocity vector

• The sound pressure and the sound velocity vector can be re-created using a speaker array or headphones

• Downside: Low spatial resolution / small sweet spot (when reproduced with speakers)

Spherical Microphone Arrays

• Typically 24-64 microphones on rigid sphere

• Parameterize the impinging sound field

• Upside: High spatial resolution

• In theory, a continuous layer of microphones is required -> spatial aliasing

• Downside: Significant limitations in practice (best performance only over 2 octaves)