Books
- THRUN - Probabilistic Robotics by Sebastian Thrun, Wolfram Burgard, and Dieter Fox, 2005. Required textbook - the foundational text for modern robotics.
- LYNCH - Modern Robotics: Mechanics, Planning and Control. Free to download. Oriented towards manipulation with foundational motion algebra.
- CORKE - Robotics, Vision and Control: Fundamental Algorithms in PYTHON by Peter Corke, 3rd edition, 2023. Hands-on complement to THRUN and LYNCH. See also the robotics-toolbox-python repository.
Learning Outcomes
After completing this course, students will be able to:- Design the various subsystems involved in robotic agents with egomotion
- Implement perception using sensor fusion (computer vision with LiDAR and other sensors)
- Implement planning algorithms for path planning and motion/trajectory planning
- Train robotic control policies in simulation and transfer them to reality
- Instruct robots using natural language
- Program robotic systems using the ROS2 framework
Planned Schedule
This course emphasizes mobile robots (not manipulation) and covers:- Part I: Robotic Perception (Lectures 1-5)
- Part II: Localization and Mapping (Lectures 6-7)
- Part III: Task, Global and Local Planning (Lectures 8-9)
- Part IV: Reinforcement Learning, Instruction Following (Lectures 10-12)
| Lecture | Topic | Description |
|---|---|---|
| 1 | Introduction | AI and robotics systems perspective with autonomous vehicles focus. Prerequisites review. |
| 2 | Perception with DNNs | Multi-modal sensing, maximum likelihood optimization, fully connected and convolutional architectures. |
| 3 | Object Detection | MaskRCNN, YOLO, transfer learning with pretrained feature extractors. |
| 4 | Segmentation | Semantic and instance segmentation for complex scene planning. |
| 5 | State Estimation | Recursive estimation, Dynamic Bayesian Networks, Kalman filters. Reading: THRUN Chapters 2-3 |
| 6 | Localization | Egomotion, velocity/odometry models, pose estimation. Reading: THRUN Chapters 5, 7; CORKE Chapter 4 |
| 7 | SLAM | Simultaneous Localization and Mapping, Visual SLAM with monocular cameras. Reading: THRUN Chapters 9-10; CORKE Chapter 6 |
| 8 | Global Planning | Optimal planning under uncertainty, A*, D*, RRT*, PRM algorithms. |
| 9 | MDPs and POMDPs | Sequential decisions, reward signals, Bellman equations. |
| 10 | Deep RL | Model-free methods for robotic control policies. |
| 11 | Instruction Following | Vision Language Models for human-robot collaboration. |
| 12 | VLA Models | End-to-end Vision-Language-Action models for instruction parsing, perception, and action. |
| 13 | Review | Final exam preparation |