Data analysis

with Image.open(image_files[0]) as im:

np.array(im)

• Images have as many features as they have pixels

• RGB: thre channels (red, blue, green)

• Grayscale: one channel

• There may be an additional transparancy channel

• Convert PIL-image to numpy array and use mean() and std()

mean = image.mean(axis=(0, 1))

std = image.std(axis=(0, 1))

• With large datasets it may be useful to create save points and compress the saved data

import dill as pkl

import gzip

• You can use pretrained CNN models provided by Pytorch, to downproject the data into a better suited feature space, e.g. SqueezeNet 1.1 model

weights = torchvision.models.SqueezeNet1_1_Weights.IMAGENET1K_V1

pretrained_model = torchvision.models.squeezenet1_1(weights=weights)

from sklearn.manifold import TSNE

tsne = TSNE(n_components=2, perplexity=30, random_state=1)

images_projected_tsne = tsne.fit_transform(images_projected_cnn)

• Scaling to range [-1, 1].

• Scaling to zero (0) mean and unit (1) variance

• Scaling to range [0, 1].

• You compute it for the training set and apply it also to the test set

• generate a Dataset from a subset of the original Dataset according to a list of indices.