Faster RCNN

With Faster RCNN, the 3rd generation in the family of region-based detectors, we are replacing the selective search algorithm that is considered computationally expensive, with a neural network called the Region Proposal Network (RPN) that as the name implies produces the proposals. This allows us to call the detector differentiable and therefore train it end-to-end in a much more straightforward way. Faster RCNN Architecture - the RPN tells the Fast RCNN detector where to attend to. Therefore, in this architecture there is one CNN network that not only produces a global feature map but also produces proposals from the feature map itself rather than the original image, using additional convolutional layers and a sliding window scheme detailed below. Since the RPN component is the key differentiator we limit the discussion to it.

​

Region Proposal Network (RPN)

Region Proposals as generated by the RPN network The RPN produces proposals by sliding a window $n \times n$ over the feature map. At each sliding-window location, we simultaneously predict multiple region proposals, where the number of maximum possible proposals for each location is denoted as $k$. So the regression layer has $4k$ outputs encoding the coordinates of $k$ boxes, and the classification layer outputs $2k$ scores that represent the probability of the presence of an object or not an object for each proposal. The $k$ proposals are parameterized relative to $k$ reference boxes, which we call anchor boxes. The size can be changed but by default we use 3 scales and 3 aspect ratios, yielding k = 9 anchors at each sliding position. For a feature map of a size $W × H$ (typically ∼2,400), there are $W \times H \times k$ anchors in total. The RPN network produces a classification score i.e. how confident we are that there is an object for each of the anchor boxes as well as the regression on the anchor box coordinates. Key references: (Ren et al., 2015; Zhou et al., 2014; Sermanet et al., 2013; Zhou et al., 2014; Redmon et al., 2015)

​

References

• Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2015). You only look once: Unified, real-time object detection. arXiv [cs.CV].
• Ren, S., He, K., Girshick, R., Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv [cs.CV].
• Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., et al. (2013). OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks. arXiv [cs.CV].
• Zhou, B., Khosla, A., Lapedriza, À., Oliva, A., Torralba, A. (2014). Object detectors emerge in deep scene CNNs. International Conference on Learning Representations.
• Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A. (2014). Object detectors emerge in deep scene CNNs. arXiv [cs.CV].

---