MMI 2 nicht gut#

A phoneme is the basic acoustic unit of speech (42 in American English), while a viseme is the basic unit for "visual" speech, representing a class of visually indistinguishable phonemes.

Appearance Based Features use pixel information and offer robust extraction but have high dimensionality (requiring PCA/LDA); Model-based features use specific parameters/vectors and have low dimensionality but are more difficult to obtain accurately.

The challenges include feature selection (invariants for lighting/motion), motion estimation (finding correspondence in images), active vision (controlling camera zoom/tilt), and ensuring real-time implementation.

It uses a modification function f′(p)=T(f(p)) to transform original gray values into a new standardized gray value, making the system robust against different illumination conditions.

Fusion can occur at the Feature level (often too simple), the Phoneme/Viseme Level (typically successful), or the Word Level (known as the decision level).

Early fusion requires a large amount of training data and often fails to generalize well if the different modes (like speech and lip movement) differ substantially in their information content or time scales.

In Multistream, audio and video are processed as separate, parallel streams through their own feature extraction and LDA before being fused during recognition, whereas Concatenation blends them into a single feature vector earlier in the process.

This method uses weights (λ) based on the Signal-to-Noise Ratio (SNR) of the audio; if the audio is noisy, the system automatically shifts higher weight to the visual data stream.

Increasing the training set significantly improves word accuracy across all models, with the Multi-Stream architecture reaching the highest accuracy (61.76%) compared to pure audio (60.71%).

The research concluded that the best features are Gray Values/LDA, the best fusion level is Phonetic, and the best fusion weights are SNR-based or Entropy-based.

EMG-ASR allows for speech recognition without acoustic information (measuring muscle potential instead), which enables "silent" communication in public spaces or meetings where privacy is a concern.

It refers to the difficulty of training a system to recognize a word when the electric signal for that word changes significantly across different recording sessions or users.

It is the component that combines and integrates all incoming uni-modal events into a single representation of the intention most likely expressed by the user.