Chapter 2: Hearing & Speech

1. Pragmatics: Meaning / Intention of the user uttering something

2. Semantics: Content (factual information)/ meaning of single sentences -> relationship between sign and dedicated meaning

3. Syntax: Sentence structure

4. Morphology: Word forms (subentities builded up to form words)

5. Phonology: spoken Sound structure

6. Phonetics: Articulation / Acoustics / Perception -> How spoken sounds are articulated, how they are manifested in acoustics and how they are perceived

Phoneme:

• “...smallest segmental unit of sound employed to form meaningful contrasts between utterances...” (from International Phonetic Association, 1999)

• example: functional contrast between “pit” and “bit”

• Phoneme -> different Phonological classes

So phoneme is a functional class which is restricted to a given language

Phone:

• Smallest phonetic unit of speech

• Often used in terms of concrete realization of a phoneme (parole vs. langue). (used to produce phoneme)

• Phone -> different Phonetic classes

Allophone:

• Different phones of the same phoneme

• Distributed complimentarily

• Free (regional, social) variants

• phoneticaly different but functionally, phonologicaly not different

• example (german): ich vs ach

Morpheme: Smallest unit of speech carrying meaning.

• Speech: Sequence of phones of individual phonemes.

• Written language: Grapheme sequences.

Syllable: Phonetic, not meaning-related unit; smallest group of phones in natural speech production process.

• Contains exactly one syllable-peak, typically vowel or “double-vowel” (diphthong).

• Can contain one or more consonants.

Morpheme – syllable distinction: different decompositions; example “painter”.

• Syllable: “pain-ter”.

• Morpheme: “paint-er”

- Provides information about linguistic and paralinguistic aspects

• Type of expression (statement, question, etc.)

• Emotional state of speaker; irony/sarcasm; agreement, disagreement, etc.

  
  

- Linguistic consideration

• Quantity: Especially temporal structure of speech (pauses-, phone- & syllable durations, rhythm &speed)

• Accent(uation)

• Intonation: Speech melody

  
  

- Physical

• Amplitude

• Duration.

• Pitch / fundamental frequency F0.

directional hearing

Pinna: important for spatial hearing, causes diffraction at high frequencies

Ear canal: The sound is transmitted through the ear canal which is about 25 mm long and 7 mm high. As an approximation, it can be interpreted as a resonator with a closed end at the eardrum and an open end at the other side. If the length matches 𝜆⁄4, resonance occurs. The transmission of sound therefore reaches its maximum at the corresponding frequency (about 3 kHz for a length of 25 mm).

-> Pinna & ear canal are acoustic resonance system.

-> Resonance modes depend on distance and directional location of sound source(s) relative to the head

Ear drum: The incoming sound waves cause an oscillation of the ear drum which allows it to transfer them further to the middle ear

-> Pressure sensor.

-> Excitation of oscillation due to sound waves

Impedance matching air/liquid (physical medium

inner ear)

The middle ear transmits the oscillations from the ear drum via the oval window to the inner ear.

Because the inner ear contains fluid, an impedance matching between inner ear and outer ear is required to prevent strong reflections.

This impedance matching is achieved through a length difference between malleus and incus as well as through an area difference between eardrum and oval window

Signal pre-processing for further processing by

higher levels (related to Fourier analysis)

Inner hair cells: Transform the movements of the basilar membrane to nerve impulses

Outer hair cells: nonlinear amplification (most at low sound levels)

Basilar membrane: Frequency analysis – High frequencies cause a movement at the beginning of the basilar membrane, low frequencies cause a movement at the end of the basilar membrane

Spatial Hearing

- posible due to outer ear

- sound on both ears is compared and position of sound is calculated

- possible parameters: volume, pitch, time of sound arrival

Yes

Relation between phon and sone is logarithmic (from 1 sone):

• 1 sone at 40 phon

• 2 sone at 60 phon

ITD is the Interaural Time Difference and mostly imporant for lower frequencies (< 1,5 kHz). But at higher frequencies we use in complex sounds the envelope of broadband signals to localize the sound sources. (temporal difference of sound incidence)

ILD is the Interaural Level Difference and most important at higher frequencies (> 1,5 kHz) (sound pressure level difference between ears)

• Spectral superposition, spectral masking.

  → Increase of hearing threshold for test signal due to masker.

• Temporal aspects → temporal masking

- Periodicity of signal (spectral energy-distribution -> timbre)

- Sufficient number of periods (> 20)

- Pitch ≡ fundamental frequency

- Condition for pitch perception: Presence of 3 subsequent harmonics

- Physical/technical: Sound event scale of pitch (harmonic)

- 1 octave = doubling of frequency

- Perceptive: Auditory event scale of pitch (melodic) ≡ tonality