Skip to main content
  1. The library: 30 papers on VLA models, plus one more paper marked “ingest last”. The library section lists them. Save each PDF as kdb/papers/<arxiv-id>.pdf, for example kdb/papers/2306.03310.pdf, because the seed ontology and our grading refer to the papers by those names.
  2. The project repository, pantelis/physical-ai-kdb. It installs four of the command-line tools that we also run at grading time.
    • okf-validate checks that your kDB is a valid OKF v0.2 bundle and follows the rules of the kDB format. With your lake it also checks the rows, and it reports every page that changed outside a logged action.
    • okf-ground checks the evidence of every Term page and every row. A quoted sentence must be on its cited page and inside its bbox, and a table value must be inside its bbox. Every bbox must match a highlight in the annotated copy of the PDF. Every file that a row cites must be registered in your lake, and its stored bytes must still match its sha256.
    • okf-diff compares two versions of your kDB pages, as two directories or two git commits. With --snapshots it compares two snapshots of your lake instead.
    • okf-seed writes the seed ontology into your kDB (item 5).
  3. A starter, in starter/ of the same repository. It is the kdb command, with lake init, ingest, ask, export and history, and two Pydantic AI agents. Through kdb-tools it also installs three more commands. okf-export writes your lake into a folder with a manifest of hashes, and okf-restore checks every hash before it restores one. okf-mcp serves the checks and actions over MCP. Agent design says what it does and what is left to you.
  4. The same checks as a Python package, kdb_tools, so your agent can call them as functions instead of parsing command output. validate() and ground() return a report with one finding per problem. kdb_tools.locate.find_quote(page, text) returns the bbox of every place a quote appears on a PDF page. It uses the same matching as okf-ground, so your agent and the checker agree on where a quote is. The package also holds the actions that write your lake, in kdb_tools.actions.
  5. A seed ontology in three parts. robotics covers tasks (such as pick-and-place, pushing and articulated object manipulation), benchmarks and their suites (LIBERO and its suites, LIBERO-Plus, CALVIN, SIMPLER), robots (such as the Franka Emika Panda and the WidowX 250), the seven LIBERO-Plus perturbation types and the vision-language-action model class. datasets covers how data is collected and what a model uses it for, for any kind of dataset. vla-datasets specializes datasets for robot learning data. Teleoperation is a kind of human-collected data, cross-embodiment pretraining is a kind of pretraining, and so on. okf-seed kdb writes one Term page per term under ontology/<ontology>/ and one page per ontology that says what it extends. Each term is defined by a sentence it quotes from a library paper and highlights in your copy of that paper. The ontology section explains how to use it.
  6. The VLA classes. kdb lake init writes them into your kDB as Class and Property pages, with the core ontology they build on, and binds each class to a table in your lake. The kDB format lists them.
okf-seed kdb quotes ten of the papers, so it needs these files in kdb/papers/ before it runs: LIBERO-Plus is the paper you ingest last. The seed quotes it for the perturbation types, so its PDF is in kdb/papers/ from the start, but no row may cite it before Submission 2.

​
The library

The library has 30 papers on vision-language-action (VLA) models, plus one paper to ingest last. Each paper links to its Semantic Scholar page and to one version of its PDF on arXiv. Download that version and no other. Several of these papers are still being revised, and a later version moves sentences and table values to other pages. The checks compare your sources with our copies of these versions. A correct source taken from another version therefore fails okf-ground. The papers report on the same benchmarks (LIBERO, SIMPLER, CALVIN), compare the same models (RT-2, Octo, OpenVLA, π0) and train on the same datasets (Open X-Embodiment, BridgeData V2, DROID). Because of that overlap, the kDB can answer questions that span several papers. Ingest last. Its PDF is in kdb/papers/ from the start, because the seed ontology quotes it, but no row may cite it before Submission 2.
Found an error on this page? Report an issue.