Introduction
In the histogram analysis we used histograms to quantify the pixel-level information of the Seamagine dataset images and we saw that the performance of AD on pixel level metrics is dismal. The answer to this problem is to use a feature extraction method that can provide a hierarchical aggregation of the features of the image and project them into a lower-dimensional space - the manifold. Here we use a pretrained Convolutional Neural Network (CNN) to provide such features in the form of a 2048-dim vector that we then feed into a UMAP algorithm.ResNet-50
The ResNet-50 model is a deep convolutional neural network. It was pretrained in a discriminative way on a large dataset of natural images known as ImageNet that has classes. For each dataset image in our training dataset we obtain a 2048-dim vector at the global pooling layer.Visualizing the test and train datasets with UMAP
We tried three dimension-reducing algorithms: PCA, UMAP and t-SNE and based on the results we selected the UMAP algorithm for this task. UMAP forms clusters that construct a weighted k-nearest neighbors graph to model the high-dimensional data structure and then optimizes a low-dimensional representation of this graph to preserve topological features of the original data.