> ## Documentation Index > Fetch the complete documentation index at: https://aegean.ai/llms.txt > Use this file to discover all available pages before exploring further. # Perception > Sensor models, image processing, CNNs, object detection, and segmentation. This chapter covers perception. It starts with how sensors form their measurements: camera models and calibration, and beam models for lidar and other range sensors. It then turns to classical image processing, convolutional networks, object detection, and pixel-level segmentation. > **[Camera Sensor Models](/aiml-common/lectures/sensor-models/cameras/index)** > > The pinhole camera model, lenses, intrinsics and extrinsics, and camera calibration, including Zhang's method. > **[Beam Sensor Models](/aiml-common/lectures/sensor-models/beam/index)** > > Probabilistic beam and likelihood-field models for sonar, lidar, and other range sensors. > **[Image Processing](/aiml-common/lectures/image-processing/index)** > > Linear filtering and convolution, blur filters, image derivatives, filtering in space and time, sampling without aliasing, and statistical image priors. > **[CNNs](/aiml-common/lectures/cnn/cnn-intro/index)** > > Convolutional neural networks for image classification, including layer types, architectures, and visualization techniques. > **[Object Detection](/aiml-common/lectures/scene-understanding/scene-understanding-intro/index)** > > Detecting and localizing objects in images using two-stage and single-stage deep learning detectors. > **[Object Segmentation](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/index)** > > Pixel-wise classification for semantic and instance segmentation using Mask RCNN and UNet. ## Image processing Classical image processing treats an image as a signal. Filtering, differentiation, and sampling are the operations that CNNs later learn and stack. > **[Convolution and Linear Filters](/aiml-common/lectures/image-processing/linear-filtering/index)** > > Linear translation-invariant systems, convolution, boundary handling, and template matching. > **[Smoothing an Image with Blur Filters](/aiml-common/lectures/image-processing/blur-filters/index)** > > Box, Gaussian, and binomial filters and their frequency responses. > **[Measuring Change with Image Derivatives](/aiml-common/lectures/image-processing/image-derivatives/index)** > > Discrete and Gaussian derivatives, the Laplacian, sharpening, and Retinex. > **[Filtering in Space and Time](/aiml-common/lectures/image-processing/temporal-filters/index)** > > A video as a space-time volume, velocity-tuned blur, and velocity-nulling filters. > **[Sampling an Image Without Aliasing](/aiml-common/lectures/image-processing/sampling-and-aliasing/index)** > > The sampling theorem, reconstruction, sampling lattices, and anti-aliasing. ### Image priors What natural images have in common, and how that knowledge becomes a prior for denoising, synthesis, and inference. > **[What Natural Images Have in Common](/aiml-common/lectures/image-processing/statistical-image-models/index)** > > The 1/f power law, heavy-tailed derivatives, and denoising with image priors. > **[Synthesizing Textures](/aiml-common/lectures/image-processing/textures/index)** > > Texture synthesis by histogram matching and by copying neighborhoods. > **[Markov Random Fields and Belief Propagation](/aiml-common/lectures/image-processing/graphical-models/index)** > > Markov random fields and belief propagation for denoising, segmentation, and stereo. ## Convolutional neural networks CNNs are the backbone of modern computer vision systems, enabling image classification, feature extraction, and visual understanding. > **[CNN introduction](/aiml-common/lectures/cnn/cnn-intro/index)** > > Introduction to convolutional neural networks. > **[CNN layers](/aiml-common/lectures/cnn/cnn-layers/index)** > > Understanding CNN layer types and operations. > **[CNN architectures](/aiml-common/lectures/cnn/cnn-example-architectures/index)** > > Example architectures: LeNet, AlexNet, VGG, ResNet. > **[Feature extraction: ResNet](/aiml-common/lectures/cnn/feature-extraction-resnet/index)** > > ResNet as a backbone for downstream vision tasks. ## Object detection Object detection covers scene understanding fundamentals, evaluation metrics, and the evolution from two-stage (RCNN family) to single-stage (YOLO family) detectors. > **[Scene understanding](/aiml-common/lectures/scene-understanding/scene-understanding-intro/index)** > > Detection vs classification, the detection pipeline, region proposals, FCNs, and the COCO dataset. > **[Detection metrics](/aiml-common/lectures/scene-understanding/object-detection/detection-metrics/index)** > > Precision, recall, mAP, and IoU for evaluating detectors. > **[From R-CNN to Faster R-CNN](/aiml-common/lectures/scene-understanding/object-detection/rcnn-introduction/index)** > > Video references that walk through R-CNN, Fast R-CNN, Faster R-CNN and mAP. > **[RCNN](/aiml-common/lectures/scene-understanding/object-detection/rcnn/index)** > > Region-based CNN: selective search, CNN features, SVM classification. > **[Fast RCNN](/aiml-common/lectures/scene-understanding/object-detection/fast-rcnn/index)** > > Shared convolutional features and ROI pooling for end-to-end training. > **[Faster RCNN](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/index)** > > Region Proposal Network enabling fully end-to-end two-stage detection. ### Faster RCNN from scratch (PyTorch) A six-notebook series building every Faster RCNN component from scratch in pure PyTorch, from COCO data loading through end-to-end training and inference. > **[01 · COCO dataloader](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/01_coco_dataloader/01_coco_dataloader)** > > Streaming COCO from Hugging Face, collation, and anchor target assignment. > **[02 · Backbone](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/02_backbone/02_backbone)** > > ResNet50 feature pyramid network (FPN) with lateral connections. > **[03 · RPN](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/03_rpn/03_rpn)** > > Region Proposal Network: anchor generation, objectness head, NMS. > **[04 · ROI head](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/04_roi_head/04_roi_head)** > > ROI Align, two-layer MLP head, and sibling classification and regression predictors. > **[05 · Training](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/05_training/05_training)** > > End-to-end training with AMP and gradient checkpointing on COCO streaming data. > **[06 · Inference](/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/pytorch/06_inference/06_inference)** > > Checkpoint loading, COCO validation inference, proposal and detection visualization. ### YOLO from scratch (PyTorch) A five-notebook series building YOLOv8-style single-stage detection in pure PyTorch, from data loading through inference and evaluation. > **[YOLO introduction](/aiml-common/lectures/scene-understanding/object-detection/yolo/introduction)** > > Single-stage detection design philosophy, anchor-free heads, and the YOLO architecture family. > **[01 · COCO dataloader](/aiml-common/lectures/scene-understanding/object-detection/yolo/pytorch/01_coco_dataloader/01_coco_dataloader)** > > Streaming COCO, grid target assignment, and mosaic augmentation. > **[02 · Backbone](/aiml-common/lectures/scene-understanding/object-detection/yolo/pytorch/02_backbone/02_backbone)** > > CSPDarknet backbone with C2f bottleneck blocks. > **[03 · Neck and head](/aiml-common/lectures/scene-understanding/object-detection/yolo/pytorch/03_neck_and_head/03_neck_and_head)** > > PANet feature pyramid neck and decoupled detection head. > **[04 · Loss and training](/aiml-common/lectures/scene-understanding/object-detection/yolo/pytorch/04_loss_and_training/04_loss_and_training)** > > Task-aligned assignment, distribution focal loss, and training loop. > **[05 · Inference and evaluation](/aiml-common/lectures/scene-understanding/object-detection/yolo/pytorch/05_inference_and_evaluation/05_inference_and_evaluation)** > > NMS post-processing, COCO mAP evaluation, and latency benchmarks. ## Object segmentation Instance and semantic segmentation extend object detection to produce pixel-level masks, enabling fine-grained scene understanding. > **[Mask RCNN](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/index)** > > Extending Faster RCNN with a mask head for instance segmentation. > **[Mask RCNN · TF demo](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/tf/demo/demo)** > > Running inference with the TensorFlow Mask RCNN implementation. > **[Mask RCNN · inspect data](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/tf/inspect_data/inspect_data)** > > Visualizing COCO data loading, augmentation, and anchor generation. > **[Mask RCNN · inspect model](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/tf/inspect_model/inspect_model)** > > Layer-by-layer inspection of model activations and outputs. > **[Mask RCNN · inspect weights](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/tf/inspect_weights/inspect_weights)** > > Visualizing learned filter weights and statistics. > **[Mask RCNN · PyTorch (Detectron2)](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/pytorch/maskrcnn_detectron/maskrcnn_detectron)** > > Detectron2 Mask RCNN training and evaluation workflow. > **[Mask RCNN · torchvision inference](/aiml-common/lectures/scene-understanding/semantic-segmentation/maskrcnn/pytorch/maskrcnn_torchvision_inference)** > > Running pretrained Mask RCNN inference with torchvision. > **[UNet](/aiml-common/lectures/scene-understanding/semantic-segmentation/unet/index)** > > Encoder-decoder architecture for semantic segmentation. *** Found an error on this page? [Report an issue](https://github.com/aegean-ai/eaia-feedback/issues/new?template=page-issue.yml\&page=/book/perception). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.