Skip to main content
Realistic simulation environments are essential for training and evaluating embodied AI agents before deployment on physical hardware. This section introduces Gazebo Sim, the standard open-source simulator for ROS 2 robotics, with a focus on authoring rich indoor environments and using large language models to generate simulation assets from natural language descriptions. The running example throughout is the TurtleBot3 behavior demos project: a ROS 2 Jazzy robot navigating AWS RoboMaker environments using Nav2, behavior trees, and AI-based perception.

Gazebo worlds and SDF/Xacro

World file anatomy, the SDF + Xacro format, and a walkthrough of the AWS RoboMaker Small House — 68 residential furniture and appliance models.

LLM-driven world authoring

Using large language models to generate SDF/Xacro world files and Gazebo models from natural language descriptions. Prompt patterns, iterative refinement, and literature review.

​
Why simulation matters

Training and evaluating embodied AI agents in simulation offers several key advantages:
  • Safety — no risk of hardware damage during early exploration
  • Scale — run hundreds of parallel environments that would be impossible to instrument physically
  • Reproducibility — fixed random seeds and deterministic physics for fair comparison
  • Domain diversity — vary lighting, furniture layout, object placement, and sensor noise programmatically
The central challenge is the sim-to-real gap: models that work well in simulation often degrade when deployed on physical hardware. High-fidelity asset authoring — realistic geometry, physics, and sensor responses — closes this gap.

​
Simulation stack

The tutorials use the following stack:
ComponentRole
Gazebo Sim (Harmonic)Physics simulation and 3D visualization
ROS 2 JazzyRobot middleware — topics, services, actions
Nav2Autonomous navigation stack
AWS RoboMaker worldsOpen-source indoor environment assets
Docker ComposeReproducible multi-container setup
Edit this page on GitHub or file an issue.