> ## Documentation Index > Fetch the complete documentation index at: https://aegean.ai/llms.txt > Use this file to discover all available pages before exploring further. # The kDB format > How a Physical AI kDB is laid out as an OKF bundle and a DuckLake lake, with its classes and tables, evidence, actions, worked examples and rules. Your kDB has two parts. The kDB itself holds types: the ontology terms, and the classes and properties that say what a paper, a model or a result is. It is a bundle in Google's [Open Knowledge Format (OKF) v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md): a directory of markdown files with YAML frontmatter, kept in a git repository. Read the spec. It is one short file. The lake holds the instances. Every paper, model, dataset, training link, result, contested claim, computation and analysis is a row in a table of a DuckLake catalog, and every PDF is registered in its media table. A page in the kDB describes a kind of thing. A row in the lake describes one thing, so if a page would only make sense for one paper or one model, it belongs in the lake as a row. Your project folder looks like this after `kdb lake init` and `okf-seed`: ```text theme={null} kdb.toml # the model server, and the [lake] table that kdb lake init adds compose.yml # RustFS, for when you keep the lake's data in S3 storage kdb/ # the kDB: an OKF bundle in git index.md # lists the pages log.md # one line for every action that changed a page ontology/ core/ # shared properties, such as title, aliases and evidence vla/ # the VLA classes and properties robotics/ # Term pages, one directory per ontology datasets/ vla-datasets/ papers/ # the original PDFs, unmodified annotated/ # your highlighted copies, with the same file names as papers/ lake/ catalog.ducklake # the catalog: tables, snapshots and the action log data/ # the data files and the media files ``` The lake is local by default: a catalog file and a data folder next to `kdb.toml`. You can keep its data in RustFS instead, an S3-compatible store that runs in Docker on your own machine from the `compose.yml` that `kdb lake init` writes. Decide before your first `lake init`. `kdb.toml.example` in the starter shows how. Either way the lake runs on your machine, and you never need a cloud credential. Run every command from your project folder, because `kdb` finds `kdb.toml` in the current folder or a folder above it. `okf-validate` and `okf-ground` look for it in the folder of the bundle you name or a folder above that one. ## Classes and tables `kdb lake init` binds each VLA class to its own table. Every table also has the columns `iri`, `type` (the class IRI) and `team`, which are set for you. | Class | Table | Columns | | - | - | - | | Paper | `papers` | `title`, `arxiv_id`, `version`, `media` | | Model | `models` | `title`, `aliases`, `description`, `model_class`, `evidence` | | Dataset | `datasets` | `title`, `aliases`, `description`, `collection`, `embodiments`, `tasks`, `modalities`, `evidence` | | Trained on | `trained_on` | `model`, `dataset`, `role`, `evidence` | | Result | `results` | `title`, `model`, `task`, `benchmark`, `suite`, `embodiment`, `perturbation`, `trials`, `trials_reported`, `value`, `metric`, `evidence` | | Contested claim | `contested` | `claim`, `disputed_by`, `evidence` | | Computation | `computations` | `title`, `code`, `inputs`, `value`, `evidence` | | Analysis | `analyses` | `title`, `question`, `answer`, `drew_on`, `evidence` | The Class pages under `ontology/vla/classes/` map each column to a Property page under `ontology/vla/properties/` or `ontology/core/properties/`, which gives its datatype and whether it is required. A paper's `media` is the sha256 of its PDF. The condition columns of a result, a model's `model_class`, a link's `role`, and a dataset's `collection`, `embodiments`, `tasks` and `modalities` hold the IRIs of Term pages. The `model` and `dataset` columns hold the IRIs of rows. Every row's IRI starts with `https://data.aegean.ai/vla//`, where `` is the id you gave `kdb lake init`: your team id, or your own id if you work alone. Derive the rest of the IRI from what the row is, such as the arXiv id, the model's name, or the paper, table, row and column of a result. Ingesting the same paper again then finds the same rows instead of making new ones. ## Evidence Each row cites its evidence in its `evidence` column, as a list of `MediaRef`s. A `MediaRef` names a registered file by its sha256 and points into it. A sentence gives the `page`, the `bbox` and the `quote`. A table value gives the `page`, the `bbox` and the `value`. A cell of a spreadsheet gives the `sheet`, the `cell` and the `value`. A `bbox` is `[x0, y0, x1, y1]` in PDF points, with the origin at the bottom-left corner of the page. That is the space PDF highlights use. Pages are counted from 1. Your agent highlights every quote and value in `annotated/.pdf` with `kdb_tools.annotate.highlight`, which keeps the highlights already in the file, including the ones `okf-seed` wrote. You then register the annotated copy with `RegisterMedia`, giving `annotates` the sha256 of the original paper. `okf-ground` looks for each highlight in the registered copies that annotate the cited PDF, and a highlight in any one of them passes. Register the copy again each time you add highlights. ## Actions Every change to the kDB and the lake is an action: a typed operation that checks its parameters before it changes anything, applies its edits as one transaction, and writes a row to the action log in the lake. You, your agent and the course tools all use the same actions. | Action | What it does | | - | - | | `RegisterMedia` | adds a file to the media table and stores its bytes; registering the same bytes again changes nothing | | `UpsertObjects` | inserts rows into a class's table and updates rows that changed | | `DeleteObjects` | removes rows, with a reason | | `SeedOntology` | writes an ontology's pages; `okf-seed` runs it | | `RevertAction` | undoes an earlier action of yours, while nothing later has changed what it touched | The type actions (`ProposeClass`, `AddProperty`, `BindTable` and the rest) change Class and Property pages. `kdb lake init` uses them to set up the VLA classes, and you do not need them otherwise. `UpsertObjects` takes a class IRI and a list of rows. An update is a patch: a column you leave out keeps its stored value, and a column you set to null is cleared. Call `dry_run` first. It changes nothing and returns the plan, with the rows it would insert and update. Then `submit` the plan. A submit fails, and changes nothing, when something the plan read has changed since the dry run. In Python: ```python theme={null} from kdb_tools.actions.objects import UpsertObjects plan = UpsertObjects().dry_run(ctx, params) # params is one of the examples below, as a dict UpsertObjects().submit(ctx, params, plan) ``` `ctx` is the context the starter's `kdbagent.lake.open_context` gives every command: the kDB, the lake, your team and who is acting. An action that would be refused raises `kdb_tools.actions.Failed` from `dry_run` or `submit`, and its `messages` list every reason. A page changed by hand fails `okf-validate` with `action.unlogged`, even when the change is only whitespace. That includes `index.md` and `log.md`. You never edit a page yourself. The actions that change pages keep the index and the log up to date. ## Examples The examples below are the `params` of real `UpsertObjects` actions, written as YAML. The quotes, values and bboxes are from the papers, and every row passes `okf-validate` and `okf-ground`. A paper is a row of its own, and so is a dataset. The sha256 is that of the arXiv version the library links. Before these rows were written, the three PDFs they cite were registered with `RegisterMedia`, the quotes and values were highlighted, and each annotated copy was registered. ```yaml theme={null} class_iri: https://kdb.aegean.ai/vla/classes/paper rows: - iri: https://data.aegean.ai/vla/team-7/papers/2310.08864 title: "Open X-Embodiment: Robotic Learning Datasets and RT-X Models" arxiv_id: "2310.08864" version: v9 media: 13f16aff5afdee583dc25b3c570e9bb0cf6baefb59b539928d919588cd8d9246 ``` ```yaml theme={null} class_iri: https://kdb.aegean.ai/vla/classes/dataset rows: - iri: https://data.aegean.ai/vla/team-7/datasets/open-x-embodiment title: Open X-Embodiment aliases: [OXE, Open X-Embodiment Dataset] description: More than a million real robot trajectories pooled from 60 existing datasets. collection: [https://kdb.aegean.ai/vla-datasets/collection/cross-embodiment-pooling] evidence: - sha256: 13f16aff5afdee583dc25b3c570e9bb0cf6baefb59b539928d919588cd8d9246 page: 3 quote: "The Open X-Embodiment Dataset contains 1M+ real robot trajectories spanning 22 robot embodiments" bbox: [313.2, 268.32, 558.0, 290.23] - sha256: 13f16aff5afdee583dc25b3c570e9bb0cf6baefb59b539928d919588cd8d9246 page: 3 quote: "The dataset was constructed by pooling 60 existing robot datasets from 34 robotic research labs around the world" bbox: [313.2, 232.45, 558.0, 266.32] ``` A model is a row of `models`, and each dataset it was trained on is a row of `trained_on` with the role the dataset played and the sentence that says so. One action can write several rows of the same class. ```yaml theme={null} class_iri: https://kdb.aegean.ai/vla/classes/model rows: - iri: https://data.aegean.ai/vla/team-7/models/octo title: Octo description: An open-source transformer policy for robot manipulation. evidence: - sha256: 73bff297cfafe523319162124e6b7f96919c0930e0a380f307255c4c7464ac93 page: 1 quote: "Octo is a transformer-based policy pretrained on 800k diverse robot episodes from the Open X-Embodiment dataset [67]." bbox: [48.96, 262.47, 563.25, 281.4] - iri: https://data.aegean.ai/vla/team-7/models/openvla-oft title: OpenVLA-OFT model_class: https://kdb.aegean.ai/robotics/models/vision-language-action-model ``` ```yaml theme={null} class_iri: https://kdb.aegean.ai/vla/classes/trained-on rows: - iri: https://data.aegean.ai/vla/team-7/trained-on/octo/open-x-embodiment model: https://data.aegean.ai/vla/team-7/models/octo dataset: https://data.aegean.ai/vla/team-7/datasets/open-x-embodiment role: https://kdb.aegean.ai/vla-datasets/roles/cross-embodiment-pretraining evidence: - sha256: 73bff297cfafe523319162124e6b7f96919c0930e0a380f307255c4c7464ac93 page: 1 quote: "Octo is a transformer-based policy pretrained on 800k diverse robot episodes from the Open X-Embodiment dataset [67]." bbox: [48.96, 262.47, 563.25, 281.4] ``` A result states the conditions under which its number holds, as Term IRIs, and cites the table cell and the sentence that gives the number of trials. Table I of the OpenVLA-OFT paper has two rows labeled "OpenVLA-OFT (ours)", one for each set of policy inputs, and 94.5 appears three times on its page. The `bbox` names the cell this result means. ```yaml theme={null} class_iri: https://kdb.aegean.ai/vla/classes/result rows: - iri: https://data.aegean.ai/vla/team-7/results/2502.19645/table-1/openvla-oft-wrist-proprio/libero-long title: OpenVLA-OFT on LIBERO-Long, with wrist camera and proprioception model: https://data.aegean.ai/vla/team-7/models/openvla-oft task: https://kdb.aegean.ai/robotics/tasks/long-horizon-manipulation benchmark: https://kdb.aegean.ai/robotics/benchmarks/libero suite: https://kdb.aegean.ai/robotics/benchmarks/libero-long embodiment: https://kdb.aegean.ai/robotics/embodiments/franka-panda trials: 500 trials_reported: true value: 94.5 metric: success rate (%) evidence: - sha256: b860aa1206b6cfb0ce8be177f961379dd6a133d52cc74ac346636e0f4952a596 page: 6 value: "94.5" bbox: [448.52, 365.44, 462.01, 373.15] - sha256: b860aa1206b6cfb0ce8be177f961379dd6a133d52cc74ac346636e0f4952a596 page: 6 quote: "OpenVLA results from this work are averaged over 500 trials for each task suite (10 tasks × 50 episodes)." bbox: [48.96, 694.14, 401.11, 702.11] ``` Two numbers contest each other only when they were measured under the same conditions. A newer paper that reruns a model under perturbed camera views reports a different result under a different `perturbation`, in a row of its own. ## The rules 1. A quoted sentence must appear word for word on its cited page. When a sentence runs onto the next page, cite the part on each page as its own `MediaRef`. A quote must also stay within one column. Words taken from two columns are not a sentence of the paper. Every `bbox` must match a highlight in the registered annotated copy of that PDF. 2. Each paper, model and dataset has exactly one row. List its other names under `aliases`. No two rows of `papers`, `models` and `datasets` may share a title or an alias, compared after normalizing, and the check runs across all three tables (`project.entity`). Give a paper its full title, so that it does not take the name of the model or dataset it presents. You lose points if you merge two different things into one row or leave one thing split across two rows. Benchmarks, suites and robots are Terms of the ontology, not rows. 3. A result always states its conditions. A `results` row gives the `task`, the `benchmark`, the task `suite` and the `embodiment`, and the `perturbation` when there is one, each as the IRI of a Term of the `robotics` ontology or one that extends it (`project.result`, `project.condition`). It gives the number of `trials`. Many tables do not say how many trials stand behind a number. In that case, set `trials_reported` to false and leave `trials` empty rather than guess. 4. A dataset lists in `collection` how its data was collected, as terms of the `datasets` ontology or one that extends it. Its `embodiments`, `tasks` and `modalities`, when given, are Term IRIs too (`project.dataset`). A model's `model_class`, when given, is a Term IRI. Each `trained_on` row names a `models` row, a `datasets` row, a `role` term of the `datasets` family and at least one `MediaRef` (`project.trained-on`). 5. A number you compute from other numbers, such as an average gain across benchmarks, is a `computations` row. Its `code` is a short Python script that reads only the values named in `inputs`, so anyone can run it again. Each input is `#`, such as `https://data.aegean.ai/vla/team-7/results/openvla-libero-long#value`. `okf-validate` checks that `inputs` is not empty, that each named row exists in your lake, and that its table has the column (`project.computation`). 6. When two papers disagree, keep both rows and both sources, and write a `contested` row. Its `claim` names the older statement as `#evidence/`, where the index counts the entries of that row's `evidence` list from 0. `disputed_by` lists the statements that dispute it in the same form, at least one, and never the claim itself. `okf-validate` checks that each named row exists and has that evidence entry (`project.contested`). Do not overwrite a claim. 7. Every row IRI starts with your lake's `iri_base`, and any class or property you add has an IRI under `https://kdb.aegean.ai/vla//` (`project.iri`). 8. Every change goes through an action. A page that no logged action wrote, or that differs from what the last action wrote, fails with `action.unlogged`. 9. A `Term` page lives under `ontology//` and has a `pref_label`. Its `broader` terms are in the same ontology or in one it extends. They must never loop back to it. Each ontology has a page, `ontology//ontology.md`, whose `extends` lists the ontologies it specializes. Ontologies never extend each other in a loop. No two terms share a label within an ontology and the ontologies it extends. Unrelated ontologies may share a label. *** Found an error on this page? [Report an issue](https://github.com/aegean-ai/eaia-feedback/issues/new?template=page-issue.yml\&page=/aiml-common/projects/nlp/physical-ai-kdb/kdb-format). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.