> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Finetuned CNN
> Finetuning pretrained CNN on domain-specific seam images for improved feature extraction
## Introduction
The Seamagine dataset contains images labeled with various machine settings. The presence of such annotations can offer the following possibility: to use them to obtain a *finetuned* backbone network and subsequently to use the features produced by it to perform the downstream anomaly detection task.
We can contrast this approach to using a backbone that have been trained on ImageNet (and other datasets) that is representative of natural images. The ImageNet pretraining gave the network a strong ability to recognize general visual features like edges, textures, and shapes but the finetuning tailors the features to the product sideseam domain. We call this model *finetuned CNN* to distinguish it from the ImageNet *pretrained CNN* model.
It is important to highlight that in finetuning we do **not** use the `PASS` or `FAIL` anomaly detection labels. We only use the supervision afforded by all the machine settings and *bet* that with all $K=12$ machine settings we have a complete coverage of variability of our data distribution $p_{data}(\mathbf{x})$ via the various machine settings, to use the weights of the finetuned backbone network that has now learned seam-oriented features rather than general features. In more general terms this is a form known as *transfer learning*.
## Finetuning Architecture
The finetuning process uses the following architecture:
```
Pretrained ResNet-50 → Classification Head (K=12) → CrossEntropy Loss
↓ ↓
pooled features softmax
```
The backbone network is pretrained on ImageNet and finetuned on the Seamagine dataset. We then can use the finetuned embeddings generated by this architecture in the Anomaly Detection flow with UMAP and kNN.
We haven't tested the permutations of the finetuning protocol in terms of freezing the CNN layers or not, or the learning rate, or the number of epochs. The results presented here are the first iteration of the finetuning exercise where all layers are trained.
## Training Results
The results presented currently contain the loss and accuracy with respect to the number of epochs. The Area Under the Received Operating Characteristic (AUROC) curve will be added - this curve will reveal the tradeoff between False Positives and False Negatives that is essential in quantifying AD performance where we have an inherently imbalanced situation: a far larger number of nominal images compared to anomalous images.
We do not use the whole network (featurizer and classification head) in a downstream task to detect anomalies, and therefore AUROC results are nevertheless of limited value.
The finetuning exercise must be repeated such that the anomalous machine configurations (currently this is machine configuration C) are not part of the training dataset.
## Finetuned Embeddings
To replicate the finetuning you need to use the `06_finetuning` notebook. The embeddings from the finetuned model are shown below.
***
[Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/products/applications/anomaly-detection/manufacturing/unsupervised-learning-models/pretrained/cnn/finetuned-pretraining.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).