2 - Image Class

Training:

• Model learns from labeled data by adjusting weights.

• Computationally intensive, requires backpropagation.

Inference:

• Model makes predictions on new images using learned weights.

• Fast and less computationally demanding.

  
  

Nearest Neighbor:

• A simple algorithm that classifies data points based on the closest labeled examples in the dataset.

• Compares distances (e.g., Euclidean) between points, assigning the label of the nearest one.

• Commonly used in classification and regression tasks.

Manhatten Block Dist.

Problems with Nearest Neighbor:

• Sensitive to Noise: Outliers or mislabeled data points can heavily influence predictions.

• Computationally Expensive: Searching through all data points for each query can be slow, especially with large datasets.

• Curse of Dimensionality: Performance degrades as the number of features (dimensions) increases, making distance calculations less meaningful.

K-Nearest Neighbors (K-NN): Instead of using a single neighbor, consider the majority label among the K nearest neighbors to reduce the impact of outliers.

Hyperparameters:

• Hyperparameters are configuration settings used to control the learning process of a model, which are set before training and cannot be learned directly from the data.

Example in Nearest Neighbor:

• K (Number of Neighbors): The number of nearest neighbors to consider for classification or regression. Choosing the right value for K is crucial, as a small K may be sensitive to noise, while a large K may smooth out important patterns.

• Distance Metric: The choice of distance metric (e.g., Euclidean, Manhattan) used to determine the proximity of data points.

• Weighting Function: Whether to use uniform weights (all neighbors contribute equally) or distance-based weights (closer neighbors have more influence).

Cross Validation:

• A technique used to assess the performance of a model by partitioning the dataset into training and validation sets multiple times.

Role in Hyperparameter Optimization:

• Model Evaluation: Helps determine how different hyperparameter settings affect model performance by providing a more reliable estimate of its generalization ability.

• K-Fold Cross Validation: The dataset is divided into K subsets (folds). The model is trained on K-1 folds and validated on the remaining fold, repeating this process K times. The average performance across all folds is used to evaluate each hyperparameter setting.

• Grid Search/Random Search: Cross validation is often combined with these methods to systematically explore hyperparameter combinations and select the best-performing set based on validation scores.

Bei gleichmäßiger Abdeckung des Raums wächst die Anzahl der benötigten Trainingspunkte exponentiell mit der Dimension

Linearer Classifier:

• Ein linearer Classifier ist ein einfaches Modell, das Daten durch eine lineare Kombination von Eingangsmerkmalen klassifiziert. Er verwendet eine Entscheidungsgrenze, die durch eine lineare Funktion (z.B. eine Gerade oder eine hyperplane) definiert ist.

In einen Array gepackt.

1. Use a loss function to quantify how good a value of W is

2. Find a W that minimizes the loss function (optimization)

1. Vermeidung von Überanpassung (Overfitting): Regularisierung hilft, Modelle zu vereinfachen, indem sie die Komplexität einschränkt. Dadurch wird das Risiko verringert, dass das Modell die Trainingsdaten zu gut anpasst und nicht gut auf neue, unbekannte Daten generalisiert.

2. Stabilität und Robustheit: Durch das Hinzufügen von Regularisierungsparametern (z.B. L1 oder L2) wird das Modell robuster gegenüber kleinen Störungen in den Trainingsdaten, was zu stabileren Vorhersagen führt.

3. Kontrolle der Modellkomplexität: Regularisierung gibt dem Benutzer die Möglichkeit, das Gleichgewicht zwischen Bias und Varianz zu steuern, um ein besseres Modell für die gegebene Aufgabe zu finden.