> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# The ontology

> How ontology terms name conditions in the Physical AI kDB, how ingest updates it one paper at a time, and the commands your program provides.

The same words name different things in the same library. LIBERO-90 is an evaluation suite in the LIBERO benchmark. It is also the set of 90 tasks the LIBERO paper uses as pretraining data. A kDB that tags results and training data with words cannot tell the two apart, so a question about one returns the other. To prevent this, conditions, dataset properties and model classes name ontology terms.

Each term is one page under `ontology/<ontology>/`, with fields modelled on [W3C SKOS](https://www.w3.org/TR/skos-reference/): `pref_label`, `alt_labels` for the other ways papers write it, `broader` for the more general term, and `close_match` or `related_match` for the same idea in an established vocabulary, such as the [SOMA](https://github.com/ease-crc/soma) activity ontology used in robotics. Each term also has an `iri`, and that IRI is what a row stores. The body is a short scope note, cited like any other claim.

Ontologies are layered. A general ontology describes ideas that hold in any field. A specialized one refines it for one kind of data or work by pointing the `broader` of its terms at the general ones. The seed has this shape for datasets: `vla-datasets` extends `datasets`, so teleoperation is a kind of human-collected data and cross-embodiment pretraining is a kind of pretraining. A specialized ontology refines the one it extends and never redefines its terms. It may not reuse a label that ontology already has. Questions follow the layers, so a question about pretraining also finds models trained with cross-embodiment pretraining.

Run `okf-seed kdb` once, after `kdb lake init` and before your first ingest, with the ten papers of [What you are given](https://aegean.ai/aiml-common/projects/nlp/physical-ai-kdb/inputs.md) in `kdb/papers/`. When a paper writes "put the bowl on the plate" or "pick-place", your agent links the result to the pick-and-place term through its `alt_labels`. It reads the paper's own context to choose the right term when a word could name more than one.

When the library needs a term the seed lacks, add it in an ontology of your own that extends the seed ontology it belongs to. Write a seed spec in the same form as the course's specs, with a cited scope note and a `broader` link, and run it with `okf-seed kdb --spec <file>`. `okf-seed` finds the quote in your copy of the paper, highlights it and writes the page through the `SeedOntology` action, so the page is logged like every other change. Well-chosen additions earn credit. This spec adds one task term under robot manipulation:

```yaml theme={null}
domain: team-7-robotics
title: Team 7 robotics terms
description: Robotics terms that the library needs and the seed ontology lacks.
base_iri: https://kdb.aegean.ai/vla/team-7/robotics/
extends: [robotics]
papers:
  calvin: '2112.03227'
terms:
  - slug: object-lifting
    kind: tasks
    pref_label: Object lifting
    alt_labels: [lift, lift the block]
    broader: [robotics/tasks/manipulation]
    scope:
      - text: In CALVIN, a lift task succeeds once the object is grasped and raised at least 5 cm.
        quote: {paper: calvin, page: 9, text: "The object has to be grasped from the table surface and lifted at least 5 cm high."}
```

`okf-seed` does not check that your labels are new. Run `okf-validate` afterwards: a label that an existing term already uses is a `project.term-label` error. The seed ontology already lists "open drawer" as another name for articulated object manipulation, for example. Fix the spec and run `okf-seed` again, and the action rewrites the page.

The ontology makes some questions answerable. "Show every pick-and-place result" is a query for every `results` row whose `task` is the pick-and-place term or a term narrower than it, grouped by benchmark, suite, embodiment, number of trials and perturbation. Results in different groups are not compared, so a clean result is never set against a perturbed one. "Which VLA models were trained on the same datasets?" is a query over the `models` rows whose `model_class` is the vision-language-action model term, joined with their `trained_on` rows and grouped by dataset, with the role each model gave the dataset. Every link in the answer is a row with its own evidence, so it can be checked like any other claim.

## Incremental updates

Your agent must provide one command, `ingest <pdf>`, that adds a single paper to an existing kDB and lake. An incremental update changes only the rows the new paper affects. It writes them through one or more actions, and the log row of each action names every row it inserted, updated or deleted, with the values before and after. A row the new paper does not affect is not written at all. Because `UpsertObjects` is a patch, adding an alias to an existing model takes a row with only the model's IRI and its `aliases`, and the rest of the row stays as it was. A list column is replaced as a whole, so send the full list of aliases, the old ones and the new one. Deleting rows and writing them again, or rebuilding the whole kDB to add one paper, does not count as an incremental update.

Pages change only when the paper adds a term or another type. Every other page must stay byte-identical, including its frontmatter and its `generated` stamp.

You can test this yourself at any time. Hold one of the 30 papers out, build the kDB from the other 29 and commit. Note the lake's snapshot:

```bash theme={null}
uv run python -c "from kdb_tools.lake import Lake, load_config; print(Lake.open(load_config()).snapshot())"
```

Ingest the paper you held out, commit again, note the new snapshot, and compare the two, with your own snapshot numbers in place of 18 and 34:

```bash theme={null}
uv run okf-diff kdb --lake-config kdb.toml --snapshots 18 34
uv run okf-diff kdb --git HEAD~1 HEAD
```

The first command lists, for each table and for the media table, the rows added, changed and removed between the two snapshots, each with the ids of the actions that made the change. A change that no logged action made is a `diff.unlogged` error. The second command lists the pages and files that changed between two commits of the kDB. It reports a page that was rewritten with no change to its content, and a page whose `generated` stamp does not match its change.

## The command line

Your agent is used through one command-line program with at least these commands. The starter's `kdb` command has a place for each of them. The rows that `ingest` writes, `history`, `--show` and `--save` are yours to write.

1. `ingest <pdf>` adds one paper to the kDB, as described above. It registers the PDF with `RegisterMedia`, writes the rows with `UpsertObjects`, highlights the evidence and registers the annotated copy.
2. `ask "<question>"` answers from the lake and the kDB alone. Each sentence of the answer cites the rows and the evidence entries it rests on. For each citation, the program prints the paper, the page and the quoted sentence or table value. `--show` also displays the cited region of the page with its highlight, inline in terminals that support images and as a PNG file otherwise. When the lake does not contain the answer, the program says so.
3. `history` lists earlier questions with their answers and citations, and `history --paper <arxiv-id>` lists only those whose answers cite that paper.
4. `ask --save` files the answer back as an `analyses` row, with the question, the answer, the rows it drew on in `drew_on`, and copies of the evidence it cites. The row must pass `okf-validate` and `okf-ground`. If it does not, nothing is saved. Later questions can then use it, so the kDB grows from the questions you ask, as Karpathy's LLM Wiki proposes.

Keep the question history in a SQLite file next to the kDB. Store the question, the answer, every citation, the model, the token counts, the git commit of the kDB and the snapshot of the lake at the time, and the full conversation. The commit and the snapshot record which version an answer used. When a later ingest changes a row, you can see which earlier answers relied on the old version.

***

Found an error on this page? [Report an issue](https://github.com/aegean-ai/eaia-feedback/issues/new?template=page-issue.yml\&page=/aiml-common/projects/nlp/physical-ai-kdb/ontology).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.