4. Core Technologies of AI

• Systems learn from data instead of explicit rules

• Example:

  • email routing (spam, refunds, support)

• Advantage:

  • handles ambiguity better than rule-based systems

• Key idea:

  • infer patterns from examples → not manual programming

• Systems improve performance through experience (data)

• Identify patterns and structures in data

• Generalize to unseen inputs

Tom Mitchell definition

• Learning = improvement in task performance through experience

• Task (T) → what to solve (classification, prediction, recommendation)

• Experience (E) → training data

• Performance (P) → evaluation metric (accuracy, error, etc.)

• Define problem

• Collect & prepare data

• Train model

• Evaluate performance

• Improve iteratively (loop)

• Structured collection of:

  • labeled or unlabeled data

• Used for training & evaluation

• Features → input variables (data description)

• Labels → target outputs (what to predict)

• Trained on input–output pairs

• Learns mapping: X → Y

Tasks:

• Classification:

  • discrete labels (spam / not spam)

• Regression:

  • continuous values (house price, temperature)

• No labels provided

• Goal: find hidden structure

Key tasks:

• clustering (group similar data)

• dimensionality reduction (simplify data)

Applications:

• customer segmentation

• visualization

• anomaly detection / fraud detection

• Agent learns via interaction with environment

• Feedback = rewards / penalties

Core elements:

• State → current situation

• Actions → possible moves

• Transition → environment dynamics

• Reward → feedback signal

• Policy → strategy for decisions

Goal:

• maximize long-term reward

1. Problem Definition

2. Data Collection & Preparation

3. Model Selection

4. Training

5. Validation

6. Testing

• Define task clearly

• Define success metric

• Data sources:

  • databases, sensors, logs, web

• Steps:

  • cleaning

  • feature extraction

  • normalization

  • handling missing values

• Quality of data = critical factor

• Model = mapping from input → output

• Trade-off:

  • simple models → interpretable, less powerful

  • complex models → powerful, risk overfitting

Common models:

• Linear regression / logistic regression

• Decision trees

• Random forests / gradient boosting

• SVM

• Neural networks

• Model learns from data

• Adjusts parameters iteratively

• Goal: generalize, not memorize

• Evaluate on validation set

• Detect overfitting

• Tune model

• Final evaluation on unseen test data

• Measures real-world performance

• MAE → average error

• MSE → penalizes large errors more

• True positives

• True negatives

• False positives

• False negatives

• Precision → correctness of positive predictions

• Recall → completeness of detection

• F1-score → balance of precision & recall

• Model deployed in real system

• Continuous monitoring required

• Retraining if performance degrades

• Model too simple

• High bias

• Poor learning

• Model too complex

• Learns noise + details

• High variance

• Bias → error from simplicity

• Variance → sensitivity to data changes

• Goal:

  • low bias + low variance

• prevents overfitting

• penalizes complexity

• noise in data

• cannot be eliminated

⚖️ Fairness

• biased data → biased models

🧾 Interpretability

• black-box problem in complex models

🛡️ Robustness

• vulnerable to noisy/adversarial inputs

🌱 Sustainability

• high energy / computational cost

• Field of AI focused on:

  • understanding human language

  • generating human-like language

• Goal:

  • enable communication between humans and machines

• Applications:

  • chatbots

  • summarization

  • translation

  • search engines

  • information extraction

• Human language is:

  • ambiguous

  • context-dependent

  • highly variable

• Requires understanding at multiple levels:

  • structure

  • meaning

  • context

• Cannot be processed directly → must be encoded into machine-readable form

• Morphology

• Syntax

• Semantics

• Pragmatics

• Discourse Analysis

• Structure of words

• Breaks words into morphemes (prefixes, suffixes)

• Links word forms:

  • train / trainer / training

• Sentence structure and grammar

• Identifies:

  • subject

  • verb

  • object

• Uses parsing rules

• Meaning of words and sentences

• Resolves ambiguity based on context

• Example:

  • “agent” = person or AI system

• Meaning depends on context & intention

• Uses world knowledge

• Example:

  • “I have an early flight” → suggests need for alarm/ride

• Meaning across sentences / conversations

• Tracks:

  • references (“it”, “they”)

  • coherence over time

• Important for chatbots, summaries

• Tokenization

• Normalization

• Splits text into tokens:

  • words, subwords, punctuation

• Handles languages with no clear word boundaries (e.g. Chinese, Arabic)

• Standardizes text:

  • lowercasing

  • removing punctuation

  • expanding contractions

Word normalization:

• Stemming:

  • rule-based shortening

  • fast but imprecise

• Lemmatization:

  • dictionary + grammar-based

  • more accurate

  • returns correct base form

• Assigns grammatical role:

  • noun, verb, adjective

• Uses context to resolve ambiguity

• Modern approaches:

  • machine learning + statistical models

• Analyzes sentence structure

• Determines relationships between words

Types:

• Constituency parsing

  • phrase structure trees (NP, VP)

• Dependency parsing

  • word-to-word relationships

  • identifies “who does what to whom”

• Machines cannot process raw text

• Convert text → numerical vectors

• Counts word occurrences

• Ignores:

  • order

  • grammar

• Pros:

  • simple, fast

• Cons:

  • sparse, no semantics

• Measures word importance

• Combines:

  • term frequency (local importance)

  • inverse document frequency (global rarity)

• Highlights meaningful words

• Reduces impact of common words (“the”, “and”)

Distributional Hypothesis

• “Words are defined by their context”

• Similar contexts → similar meanings

• Words → dense vectors

• Capture semantic similarity

• Improve over BoW/TF-IDF

• Word2Vec

• GloVe

• fastText

• Predict-based model

• Learns meaning via context prediction

Architectures:

• Skip-gram:

  • predicts context from word

• CBOW:

  • predicts word from context

• Captures semantic relationships

• Enables analogies (king - man + woman ≈ queen)

• Uses global co-occurrence statistics

• Builds word relationships from entire corpus

• Captures broader semantic structure

• Complementary to Word2Vec

• Uses subword (character n-grams)

• Example: “robotics” → rob, bot, tic

• Advantages:

  • handles rare words

  • works with misspellings

  • supports new words (OOV problem solved)

• Same word → same vector

• No context sensitivity

• Transformer-based model

• Bidirectional context understanding

Training tasks:

• masked language modeling

• next sentence prediction

Key feature:

• word meaning depends on context

• Extremely large Transformer models

• Trained on massive datasets

• Use autoregressive prediction

Strengths:

• text generation

• reasoning

• translation

• summarization

Learning modes:

• zero-shot learning

• few-shot learning

Limitations:

• bias

• hallucinations

• high computational cost

1. Problem Definition

2. Preprocessing

3. Feature Extraction

4. Classification

5. Output Generation

6. Evaluation

• Define task (e.g., spam detection)

• tokenization

• normalization

• cleaning text

• TF-IDF

• word embeddings

• hybrid features

• models:

  • logistic regression

  • SVM

  • random forest

  • neural networks

• prediction:

  • spam / not spam

• confidence scores

• precision

• recall

• accuracy

• F1-score

• user feedback loop

• retraining with new data

• adapts to changing language patterns

• Field of AI that enables machines to:

  • interpret visual data (images, videos)

  • extract semantic meaning from pixels

• Goal:

  • transform raw visual input → structured understanding

• Key applications:

  • autonomous driving

  • face recognition (smartphones)

  • warehouse robotics

  • medical imaging

• Input: pixel data (images/videos)

• Output:

  • objects

  • positions

  • relationships

  • actions/events

• Key transformation:

  • low-level pixels → high-level meaning

• Image processing:

  • improves image quality (no interpretation)

• Computer vision:

  • interprets content and meaning

• Humans:

  • robust perception

  • context-aware

  • good with occlusion & ambiguity

• Machines:

  • require large labeled datasets

  • sensitive to noise and variation

  • limited generalization

• image classification

• object localization

• object detection

• image segmentation

• Assigns one label to whole image

• No object localization

• Examples:

  • disease detection in medical scans

  • crop monitoring

  • product tagging

• Detects:

  • object + bounding box

• Output:

  • class + (x, y, w, h)

• Use cases:

  • robotics

  • warehouse automation

• Detects multiple objects per image

• Outputs:

  • multiple labels + bounding boxes

• Applications:

  • self-driving cars

  • surveillance

  • industrial inspection

• Pixel-level classification

• Produces:

  • detailed scene map

• Use cases:

  • crowd tracking

  • robotics manipulation

• lighting, viewpoint, distance variation

• occlusion (objects hidden)

• deformation (changing shapes)

• noise (blur, low quality, compression)

• dataset bias and domain shift

• expensive labeled data (especially pixel-level)

1. Problem Definition

2. Image Acquisition & Preprocessing

3. Feature Extraction

4. Learning Models (CNNs)

• define:

  • task (classification, detection, etc.)

  • input/output format

• sources:

  • cameras, satellites, medical devices

• steps:

  • resizing

  • normalization (0–255 → scaled values)

  • noise reduction

  • batch processing

• goal:

  • convert pixels → meaningful patterns

Convolution

• detects:

  • edges, textures, shapes

• produces:

  • feature maps

Pooling

• reduces spatial size

• types:

  • max pooling (strongest signal)

  • average pooling (smoothed representation)

• exploit spatial structure of images

• layers:

  • convolution → pooling → fully connected

• build:

  • hierarchical features (edges → objects)

LeNet-5

• early CNN

• digit recognition

• alternating conv + pooling layers

GoogLeNet

• inception modules (multi-scale features)

• efficient deep architecture

• global average pooling

ResNet

• uses skip connections

• solves vanishing gradient problem

• enables very deep networks

• training loop:

  • forward pass → loss → backpropagation → update

• loss function:

  • cross-entropy (classification)

Overfitting prevention:

• data augmentation:

  • rotation, flipping, brightness changes

• regularization:

  • dropout

  • early stopping

  • weight decay

  • batch normalization

• high computational cost (GPU/TPU needed)

• real-time inference requirements

• dataset bias and domain shift

• adversarial attacks (manipulated inputs)

• need for robustness in real environments

• use pretrained models (e.g., ImageNet)

• fine-tune for new tasks

• reduces:

  • training time

  • data requirements

• move beyond recognition → generation

• can create:

  • images

  • videos

  • synthetic data

Applications:

• entertainment (animation, VFX)

• gaming & VR

• medicine (simulation)

• data augmentation

• combines:

  • language + vision

• example:

  • text → image generation (e.g., DALL·E-style systems)

• deepfakes & misinformation

• bias in generated outputs

• lack of control over content quality

• need for regulation and transparency