> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Training Pipeline

> Training infrastructure and model repository for anomaly detection models

## Introduction

The training pipeline was built around the PyTorch framework and includes the elements shown below.

<img src="https://mintcdn.com/aegeanaiinc/-u553YOuO9vAksef/products/applications/anomaly-detection/manufacturing/training-pipeline/images/overview.png?fit=max&auto=format&n=-u553YOuO9vAksef&q=85&s=8a69e2b49d01f97f336a15baa794b561" alt="Training / Serving Overview" width="1271" height="779" data-path="products/applications/anomaly-detection/manufacturing/training-pipeline/images/overview.png" />

Although the models developed in other sections of representation learning include training, this section outlines what generally needs to be kept in mind when training AD models across the different approaches.

## Model Repository

The trained model ends up in an S3 bucket and has a unique ID associated with the experiment ID of the training job. The experiment ID itself can track the exact version of the model development code based on the git commit.

## Training Considerations

### Data Pipeline Integration

The training pipeline consumes data from the data pipeline in the form of:

* Parquet files for efficient batch loading
* Transformed images with appropriate augmentations
* Train/validation splits for model evaluation

### Experiment Tracking

All training runs are tracked using ClearML, which captures:

* Hyperparameters and configuration
* Training metrics (loss, accuracy)
* Model artifacts and checkpoints
* Code version and git commit

### Hardware Requirements

Training anomaly detection models requires:

* GPU with sufficient VRAM for batch processing
* Fast storage for data loading
* Experiment tracking infrastructure

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/products/applications/anomaly-detection/manufacturing/training-pipeline/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>
