> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Faster RCNN

With [Faster RCNN](https://arxiv.org/abs/1506.01497), the 3rd generation in the family of region-based detectors, we are replacing the selective search algorithm that is considered computationally expensive, with a neural network called the Region Proposal Network (RPN) that as the name implies produces the proposals. This allows us to call the detector differentiable and therefore train it end-to-end  in a much more straightforward way.

<img src="https://mintcdn.com/aegeanaiinc/kzxVP5xXTKVusMDe/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/images/faster-rcnn.png?fit=max&auto=format&n=kzxVP5xXTKVusMDe&q=85&s=89300f3371694d54886a5aa5edabf4d3" alt="" width="732" height="532" data-path="aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/images/faster-rcnn.png" />

Faster RCNN Architecture - the RPN tells the Fast RCNN detector where to attend to.

Therefore, in this architecture there is one CNN network that not only produces a **global** feature map but also produces proposals from the feature map itself rather than the original image, using additional convolutional layers and a sliding window scheme detailed below. Since the RPN component is the key differentiator we limit the discussion to it.

## Region Proposal Network (RPN)

<img src="https://mintcdn.com/aegeanaiinc/hS9p_kPxnquN68XS/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/images/rpn.png?fit=max&auto=format&n=hS9p_kPxnquN68XS&q=85&s=45733f1b4a15668018be3586dc50370a" alt="" width="682" height="232" data-path="aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/images/rpn.png" />

Region Proposals as generated by the RPN network

The RPN produces proposals by sliding a window $n \times n$ over the feature map. At each sliding-window location, we simultaneously predict multiple region proposals, where the number of maximum possible proposals for each location is denoted as $k$. So the regression layer has $4k$ outputs encoding the coordinates of $k$ boxes, and the classification layer outputs $2k$ scores that represent the probability of the presence of an object or not an object for each proposal.

The $k$ proposals are parameterized relative to $k$ reference boxes, which we call **anchor boxes**. The size can be changed but by default we use 3 scales and 3 aspect ratios, yielding k = 9 anchors at each sliding position. For a feature map of a size $W × H$ (typically ∼2,400), there are $W \times H \times k$ anchors in total.

The RPN network produces a classification score i.e. how confident we are that there is an object for each of the anchor boxes as well as the regression on the anchor box coordinates.

<Note>
  See lecture notes for further RPN discussion
</Note>

**Key references**: (Ren et al., 2015; Zhou et al., 2014; Sermanet et al., 2013; Zhou et al., 2014; Redmon et al., 2015)

## References

* Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2015). *You only look once: Unified, real-time object detection*.
* Ren, S., He, K., Girshick, R., Sun, J. (2015). *Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks*.
* Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., et al. (2013). *OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks*.
* Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., Torralba, A. (2014). *Object detectors emerge in deep scene CNNs*.
* Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A. (2014). *Object detectors emerge in deep scene CNNs*.

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/aiml-common/lectures/scene-understanding/object-detection/faster-rcnn/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>