> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Recycling Robot Example

> Applying Bellman optimality to a recycling robot MDP.

## Problem statement

<img src="https://mintcdn.com/aegeanaiinc/UzutG6ztfQfeNkZI/aiml-common/lectures/mdp/mdp-workshop/recycling-robot/images/recycling-robot-fsm.png?fit=max&auto=format&n=UzutG6ztfQfeNkZI&q=85&s=edfb2c5a584296b2274eae433fbee689" alt="recycling-robot-fsm" width="649" height="831" data-path="aiml-common/lectures/mdp/mdp-workshop/recycling-robot/images/recycling-robot-fsm.png" />

*Finite State Machine of a a recycling robot and MDP dynamics LUT*

## Solution

<img src="https://mintcdn.com/aegeanaiinc/eGY_daH3zPg8UjMM/aiml-common/lectures/mdp/mdp-workshop/recycling-robot/images/solution.png?fit=max&auto=format&n=eGY_daH3zPg8UjMM&q=85&s=148af6e9be41be095ebb97cc08562d6b" alt="solution" width="689" height="477" data-path="aiml-common/lectures/mdp/mdp-workshop/recycling-robot/images/solution.png" />

**Key references**: (Bency et al., 2019; Schmidhuber, 2015; Marco et al., 2017; Hamrick et al., 2017)

## References

* Bency, M., Qureshi, A., Yip, M. (2019). *Neural Path Planning: Fixed Time, Near-Optimal Path Generation via Oracle Imitation*.
* Hamrick, J., Ballard, A., Pascanu, R., Vinyals, O., Heess, N., et al. (2017). *Metacontrol for Adaptive Imagination-Based Optimization*.
* Marco, A., Berkenkamp, F., Hennig, P., Schoellig, A., Krause, A., et al. (2017). *Virtual vs. Real: Trading Off Simulations and Physical Experiments in Reinforcement Learning with Bayesian Optimization*.
* Schmidhuber, J. (2015). *On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models*.

***

<Callout icon="pen-to-square" iconType="regular">
  [Edit this page on GitHub](https://github.com/aegean-ai/eaia/edit/main/src/aiml-common/lectures/mdp/mdp-workshop/recycling-robot/index.mdx) or [file an issue](https://github.com/aegean-ai/eaia/issues/new/choose).
</Callout>
