> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Principal Component Analysis (PCA)

> Dimensionality reduction using principal component analysis for visualization, noise filtering, and feature extraction.

<CardGroup cols={2}>
  <Card title="PCA Workshop" icon="book-open" href="/aiml-common/lectures/dim-reduction/pca/pca-workshop">
    Comprehensive PCA tutorial covering dimensionality reduction, noise filtering, and eigenfaces
  </Card>

  <Card title="3D PCA Visualization" icon="cube" href="/aiml-common/lectures/dim-reduction/pca/3d-pca">
    Interactive 3D visualizations with matplotlib and Plotly
  </Card>
</CardGroup>

## Introduction

Consider an artificial data set constructed by taking one of the off-line digits, represented by a 64 x 64 pixel grey-level image, and embedding it in a larger image of size 100 x 100 by padding with pixels having the value zero (corresponding to white pixels) in which the location and orientation of the digit is varied at random, as illustrated in the figure below.

<Frame caption="A synthetic data set obtained by taking one of the off-line digit images and creating multiple copies in each of which the digit has undergone a random displacement and rotation within some larger image field. The resulting images each have 100 x 100 = 10,000 pixels.">
  <img src="https://mintcdn.com/aegeanaiinc/jwQmIIeV-pEBcKt1/aiml-common/lectures/dim-reduction/pca/images/manifold-number-3.png?fit=max&auto=format&n=jwQmIIeV-pEBcKt1&q=85&s=d6b413b1a48589758d0de7b533b31bb7" alt="manifold-number-three" width="2817" height="508" data-path="aiml-common/lectures/dim-reduction/pca/images/manifold-number-3.png" />
</Frame>

Each of the resulting images is represented by a point in the 100 x 100 = 10,000-dimensional data space. However, across a data set of such images, there are only three degrees of freedom of variability, corresponding to the vertical and horizontal translations and the rotations. The data points will therefore live on a subspace of the data space whose intrinsic dimensionality is three.

For real digit image data, there will be a further degree of freedom arising from scaling. Moreover there will be multiple additional degrees of freedom associated with more complex deformations due to the variability in an individual's writing as well as the differences in writing styles between individuals.

## Geometric interpretation

Can we define PCA from a graphical point of view? This is shown in the next figure.

<Frame caption="PCA seeks a space of lower dimensionality (the magenta line) such that the orthogonal projection of the data points into the subspace maximizes the variance of the projected points (green dots) or equivalently minimizes the squared distances of the projection errors (blue lines).">
  <img src="https://mintcdn.com/aegeanaiinc/pdYPwYVo46FDdwsa/aiml-common/lectures/dim-reduction/pca/images/principal-subspace.png?fit=max&auto=format&n=pdYPwYVo46FDdwsa&q=85&s=d3fc814f2125e0272e70628d537315a9" alt="principal-subspace" width="1350" height="1325" data-path="aiml-common/lectures/dim-reduction/pca/images/principal-subspace.png" />
</Frame>

## Key concepts

PCA is fundamentally a dimensionality reduction algorithm, but it can also be useful as a tool for:

* **Visualization** - Project high-dimensional data to 2D or 3D for plotting
* **Noise filtering** - Reconstruct data using only the largest principal components
* **Feature extraction** - Discover the most important directions of variance
* **Data compression** - Represent data with fewer dimensions while preserving information

## Applications covered in this chapter

1. **Introducing PCA** - Principal axes and explained variance
2. **PCA as dimensionality reduction** - Projecting to lower dimensions
3. **PCA for visualization** - Hand-written digits example
4. **Choosing the number of components** - Explained variance ratio
5. **PCA as noise filtering** - Denoising images
6. **Eigenfaces** - Face recognition with PCA

## References

* [Python Data Science Handbook](http://shop.oreilly.com/product/0636920034919.do) by Jake VanderPlas
* [Scikit-Learn PCA Documentation](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/aiml-common/lectures/dim-reduction/pca/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>