> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Optimizing Embedding Dimensions

> Hyperparameter optimization for dimensionality reduction in anomaly detection

## Introduction

As we discussed in the pretrained CNN section, the aim is to bring the high-dimensional vector data from the CNN in a lower $d$-dimensional space (a hyperparameter) and to use the $d$-dim features for the downstream tasks. To optimize the hyperparameter of what should be the best $d$-dim space we use kNN to classify the images using the accuracy of the classification as a metric for the adopted grid search optimization approach. Typically kNN is implemented inside a Qdrant vector database but the analysis uses an exact kNN algorithm without any dependency on the vector database.

## Visualizing Embeddings

<Info>
  Please note that in the plots a lexicographic mapping is used to map the labels (`PASS` and `FAIL`) to the colors.
</Info>

The UMAP algorithm reduces the 2048-dimensional ResNet-50 embeddings to 2D and 3D for visualization purposes. These visualizations show the separation between nominal (PASS) and anomalous (FAIL) samples in the embedded space.

## UMAP kNN Results

The kNN classifier is applied to the UMAP-reduced embeddings with various values of $k$ and embedding dimensions $d$. The results show:

* **Best Performance**: AU-ROC of approximately **0.97** achieved with UMAP dimensionality reduction paired with kNN using majority voting
* **Optimal $k$**: Values between 3-7 neighbors typically perform best
* **Optimal $d$**: Embedding dimensions between 10-50 provide good discrimination while maintaining computational efficiency

The high AU-ROC score indicates that the pretrained ResNet-50 features, when projected via UMAP and classified with kNN, can effectively distinguish between nominal and anomalous product seam images despite the visual similarity of the raw images.

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/products/applications/anomaly-detection/manufacturing/unsupervised-learning-models/pretrained/cnn/pretrained-dim-reduction.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>