Skip to main content
Course notebooks execute inside Docker containers using papermill for reproducible runs. A central registry (notebook-database.yml) tracks which notebooks exist, their execution environments, and run history.

​
Execute a single notebook

From the eng-ai-agents repository root: 
make execute-notebook NOTEBOOK=transfer-learning/transfer_learning_tutorial.ipynb
This will:
  1. Look up the notebook’s environment in notebook-database.yml
  2. Start the appropriate Docker container via docker compose
  3. Install notebook dependencies (make install-notebooks)
  4. Execute the notebook with papermill, extracting plots and artifacts
  5. Log results to W&B (if WANDB_API_KEY is set in .env)
  6. Update the registry with execution date and duration

​
Execute all notebooks

Run every registered notebook in sequence: 
make execute-all-notebooks
This iterates over all entries in notebook-database.yml, executing each in its registered Docker environment. Notebooks marked as colab are skipped with a printed Colab URL instead.

​
Make targets

CommandDescription
make execute-notebook NOTEBOOK=<path>Execute a single notebook in its registered Docker environment
make execute-all-notebooksExecute all notebooks from the registry
make install-notebooksInstall notebook extras and register the Jupyter kernel

​
Notebook registry

The notebook-database.yml file is the source of truth for all course notebooks. Each entry specifies: 
- source: aiml-common/lectures/regression/linear-regression/sgd_sinusoidal_dataset.ipynb
  notebook: regression/linear-regression/sgd_sinusoidal_dataset.ipynb
  code_cells: 10
  environment: torch.dev.gpu
  description: SGD with polynomial regression
  last_executed: 2026-02-08
  duration_seconds: 30.5
FieldPurpose
sourcePath to the notebook in the eaia documentation repository
notebookPath relative to notebooks/ in eng-ai-agents
code_cellsNumber of code cells (for validation)
environmentDocker Compose service to use (torch.dev.gpu, torch.dev.cpu, colab)
descriptionShort description of what the notebook covers
last_executedDate of most recent execution
duration_secondsRuntime of last execution

​
Colab notebooks

Some notebooks require Google Colab (e.g., for Colab-specific APIs or free GPU access). When you run make execute-notebook on a Colab notebook, the pipeline prints the Colab URL instead of executing locally: 
═══════════════════════════════════════════════════════════════
  This notebook requires Google Colab
═══════════════════════════════════════════════════════════════

  Open in Colab:
   https://colab.research.google.com/github/pantelis/eng-ai-agents/blob/main/notebooks/<path>

​
Output artifacts

After execution, notebooks produce:
  • Executed notebook — <stem>-executed.ipynb with all cell outputs populated
  • PNG images — plots extracted from cell outputs into an output/images/ directory
  • Plotly HTML — interactive charts saved as HTML files
  • W&B logs — metrics and images uploaded to the experiment dashboard
Edit this page on GitHub or file an issue.