Skip to content

CLI reference

Every command yidam carries, grouped as yidam --help-all groups them. This page is the map; yidam <command> --help is the detail. A gate holds the two surfaces equal: a command the binary offers and this page omits fails cli_reference.rs.

yidam --help is a shorter listing: the fourteen commands a session usually needs. Everything on this page is on --help-all.

So the roster below is the roster you have. It says nothing about the prose. A description here, or in a long help, is written by hand. It can fall behind the behaviour it describes, which is what #873 was.

Three conventions run through the whole surface.

A * means the command rewrites files in the repository it is run against. That was previously visible only in each command's long help, where you had to already suspect it to go looking. That is the wrong way round for a tool people point at a checkout they only meant to inspect.

--format json is available on most commands and emits the machine-readable report contract of RFC-0016. text is the default and is byte-stable. The editor surface and CI both consume the JSON, so the prose is free to stay prose. Commands below that take no options at all are marked (no flags).

--root <DIR> names the corpus to read. Every command that reads one takes it, including the generators that refresh a README block. Without it yidam finds a repository with git rev-parse --show-toplevel from wherever the process started, which makes the working directory load-bearing. It takes the corpus directory or any directory inside one. The root field of every JSON report says which one was resolved.

A named --root must hold a corpus. A directory with no .yidam/ is refused, not reported empty. The working directory is not held to that. Run a report from nowhere and it still prints what it found. Naming a directory is an assertion; being somewhere is not.

The commands that refuse it are the * ones that do more than refresh a README block. They author a commit, write a build artifact, or name a repository to derive. Naming a corpus to report on is one promise. Being run inside the one you are about to write to is another. It is not a promise a reader should be able to make by passing a path. yidam <command> --help is the answer for any single command.

Some commands need a build carrying the matching cargo feature. Those are marked, and Installation has the table. yidam --version prints the feature list of the binary in your hand.


Checks and gates

Read-only, and they exit nonzero on a problem — which is what makes them usable as CI steps.

CommandWhat it answers
doctorIs this setup sound? One verdict per check, and a remedy where something is owed. --strict makes warnings fail. --only <id> reports only that check. See Troubleshooting
graph-checkThe graph gate: orphans, broken links, missing labels
lintCorpus quality checks against the baseline ratchet
index-verifyDoes an embedding provider reproduce this index's contract? --remote reads the contract from the vector index [index.remote] declares
samudaya-auditInspect and validate samudaya/ seed files (no flags)
derive checkDoes each argument under [derive] paths hold to the corpus it cites? See below

lint carries the most flags of any command, because it is the one with a ratchet:

FlagEffect
--commitsAlso check the git log against the commit vocabulary
--range <RANGE>Restrict --commits to a revision range, e.g. main..HEAD
--explainPrint each check's rationale beside its findings
--warnReport findings but always exit 0
--blessRewrite .yidam/lint-baseline.yml from this run instead of gating on it
--init-baselineWrite the baseline only if absent, then exit — safe to run unconditionally

lint answers did this change make the corpus less clean? — not is the corpus clean? The baseline is what makes that the question. — see Configuration.

derive

A memo, dossier or finding carries a claim out of the repository. agent-conduct.md gives that claim three rules, and derive check holds it to them (RFC-0045). It reads every artifact under [derive] paths:

claim: The dam's release schedule governs discharge below it.
reach: attributed
cites:
  - node: reach/tailwater
    span: Discharge tracks the release schedule
answers:
  - node: reach/tailwater
    refusal: The record does not say the dam caused the 2019 avulsion.
    answer: This memo claims the schedule, not the avulsion.

Each span must sit in one paragraph of the node it names. Resolution, drift and a declared tag: are checked as lint checks a node's own local citation.

The tier is computed. It is the weakest standing beneath the spans. public admits [verified] only, attributed admits [inference] and stronger, and internal admits anything. An artifact citing nothing is [open].

A declared refusal is owed an answer. A node declares one under refuses:. When an artifact cites that paragraph, it names the refusal under answers: and says why the claim survives it. Quoting the refusal itself owes nothing.

FindingSeverity
derive-unresolved, derive-span-drift, derive-span-crosses-paragraph, derive-tag-drifterror
derive-beyond-reach, derive-no-reacherror
derive-unanswered-refusal, derive-stale-answererror
derive-refusal-candidateinfo

The candidate is a cited paragraph that reads like a refusal and declares none. It is a question for a person and never fails the check. No [derive] paths means nothing is read, and the check passes.

The practice

Six commands, and none is a gate. Each exits zero however much is owed, however far a corpus has drifted, and however a contribution reads.

CommandWhat it answers
dueWhat is due? Four clocks read together — index staleness, catalog TTL, unanswered questions, phases in flight. --strict exits nonzero on a due clock
cycleWhere is this repository in its loop, and what is the next act? Owed, in flight, blocked, next — composed from due, phases, lint and graph-check. --strict exits nonzero on owed or blocked
hook <event>The loop's Claude Code hooks. session-start prints the brief; stop says what a turn moved. Always exits zero
kuten *What is this corpus's practice aimed at? Writes the declaration into AGENTS.md. kuten check reads the history against it; kuten adopt <name> declares one
practice *What has this corpus practiced? Writes a sampled series of its conduct into PRACTICE.md, each point read against the kuten that point's own tree held. Regenerated, never authored
score <range>How did this session's work read? One row per declared criterion, with its evidence

cycle gives the loop an object

Four surfaces each answered a quarter of where am I, and what next. due says what is owed, phases says what is in flight, lint and graph-check say what is blocked. A person opening a session runs none of them, or one.

cycle is those four read together, in four headed halves:

HalfWhat it reads
oweddue's five clocks, through due's own reader
in flightthe unsettled inquiry refs yidam phases lists — active, never a settled phase or a standing position
blockedwhat lint's baseline ratchet and graph-check would fail on today
nextone act per half, each carrying the finding, clock or declaration behind it

It composes rather than computes. Every number is read from the command that already owns it. So this report and its sources cannot come to disagree. That is what status did: it reported 26 active phases against a true count of 1.

Blocked means what a gate would fail on, not what the corpus was forgiven. Take a corpus carrying two hundred blessed error-severity findings. It is not blocked by them: it agreed to them, and yidam lint is green over it.

What appears here is what lint itself gates on. Three things do. A finding introduced against the baseline; an entry past its expiry; a stale entry the run no longer reproduces.

The order under next is the command's, not a kuten's. No kuten profile declares a phase order. The phases slot is a set of four type names and a commit-share band. Nothing reads it as a sequence.

What a kuten does declare is named as what it is. Two things are: the phase types a phase opened now would be named from. And the kind of question the practice presses toward.

The ranking itself is blocked, owed, in flight, practice. It is an argument the command makes, and it states it at the bottom of every report.

A repository holding no kuten gets the other three halves and is told that one needs a declaration. Holding none is a supported state.

Like due, it exits zero however much is owed. --strict exits nonzero when anything is owed or blocked. That is the union of the two signals it composes. A scheduled job wants one signal off the whole loop. It should not go quiet because the half that fired was the other one. It drafts no commit.

A round remembers the last one

cycle --record writes this round to .yidam/loop/<branch>.json. That is local state, and the directory ignores itself. cycle --since-last reports what arrived and what left since then. A branch with no record is told so, which is not the same as nothing having moved.

cycle --brief is the report cut to a token budget, 500 by default. --budget changes it.

The hooks hand a session the loop

yidam hook is what .claude/settings.json runs, and a person rarely runs it by hand. The scaffold installs it, and yidam migrate scaffold adds it to an older repository.

EventWhat it does
session-startPrints cycle --brief into the session's context and records the round. It runs after a compaction too
stopRecords the round a turn ended on. When something moved, it tells the person what

A hook never fails the session. It exits zero on every path. Outside a corpus, or on any failure, it prints nothing.

A quiet turn costs one git status. The round records a fingerprint of the checkout. A stop that matches it reads nothing else.

A kuten declares what the work is for

A repository declares what it is about — in its ontology, its classes, its central question. It had no way to say what its work is for. A kuten is that declaration.

It is vendored with the prelude, and it carries a revision. A decision record in .yidam/decisions/kuten.yml adopts it. yidam kuten writes it into AGENTS.md, so an agent meets it at session start.

A kuten narrows the loop and may not widen the model. Five things it may not do:

  • add a commit verb — it may declare a subset of the closed list and gloss it;
  • add or alter a claim standing;
  • contradict Articles I–VI;
  • change the graph encoding;
  • loosen a gate. It proposes no severity, and a profile that tries is refused.

A repository holding no kuten is a supported state. It reports as one.

Adopting one in a corpus that already exists

The bootstrap writes the decision record at genesis. A repository older than the layer has none. Re-vendor the prelude first, then adopt:

YIDAM_REF=v0.4.0 mise run yidam-vendor-update
yidam kuten adopt inquiry

adopt copies the revision from the vendored profile. Typing it by hand is the error the revision exists to survive.

It also adds the AGENTS.md section when that file has none. The scaffold carrying that section is deleted at genesis, so repositories predating the layer never received it.

Where there is no AGENTS.md at all, it writes none. Conjuring one reaches past what was asked. It says so instead, and prints the section to paste. doctor's kuten-read check warns until a file carries the block.

It refuses when a record already exists. Changing a kuten is a decide: commit carrying a superseding record.

kuten check asks a question and never fails

It reads the declaration and the repository's own history. Then it says where they disagree — you declared inquiry and have settled no phase in two hundred commits.

That is a question for a person, not a defect. due's argument applies verbatim: a corpus that has drifted is owed a look. So the command writes nothing and exits zero. Anything that refuses arrives through the policy layer, visible as an override.

Vintage is never reported as divergence. Every consumer reads the kuten and the prelude this repository vendored, not the current ones. A repository whose vendored GRAPH.md has no phase verb has not stopped running phases. It never could, and the check says so.

A comparison across revisions is annotated, never silently made. The vendored profile may have moved past the revision the decision record names. The report says so, and the numbers stay readable.

It also counts what the artifact cites. A repository that declares [object] paths gets a coupling section. It counts the relative links between the corpus and the artifact, in both directions. It says how many nodes, decisions and catalog entries the artifact cites. The JSON lists the uncited ones by id. They are not findings, because no rule says a node must be cited. A dead link from the artifact is a broken-object-link warning in yidam lint.

score reads a contribution, not a repository

The genesis rubric scores a repository's birth. It fires once. After that nothing said whether a session's work was any good. The gates said only whether it broke something, which is a floor and not a standard.

yidam score <range> reads one range against declared criteria. It reports one row each: a number, and the commits or nodes that number came from. There is no overall score.

A single number over a range names somebody's session. So there is none. Each row is a reading a person can disagree with by looking at its evidence.

The range is required, as diff's is. A default range answers about a scope nobody chose.

The criteria come from the kuten's rubric slot. Where a repository holds no kuten, the template's own criteria are used. The report says which, every time.

It also says why. The report names one of four states:

  • the criteria are the corpus's own;
  • the repository holds no kuten;
  • the range ends before the adoption the repository now holds;
  • a kuten is declared and no profile is vendored for it.

The last is a warning, the same one kuten check and doctor give.

--brief adds four questions a person answers. Nothing scores them.

It writes nothing and exits zero however it reads. A kuten binds nobody. A score that gated would be a gate decided by one. It refuses exactly once. A range spanning a kuten revision has no single set of criteria. It names both revisions and stops.

due is not doctor, and the difference is the point

doctor answers is this setup sound now. It is read under suspicion, and it exits nonzero on what is wrong. due answers is it time, and it is read on a cadence. That might be a cron job, a weekly ritual, or the start of a session.

A corpus with three expired sources is not unhealthy. It is owed. Nothing about it is broken, no traversal will lie, and the gate is green. Fold that into doctor's warnings and it reads as a problem. The repository is doing exactly what it is meant to do. The reader would learn to skip the line. So the two reports are separate. due has its own verdicts: due, ok, undeclared, declined, unmeasurable. It exits zero unless you pass --strict to ask for a signal.

Every interval is declared, and a clock nobody set never comes due

A clock is an age and an interval. The age is measured. The interval is always something the corpus said about itself, never a number in the binary. The reasoning is escalate_after's.

ClockMeasuresIntervalUnit
indexCorpus files changed since the index was built[due] index_afterfiles
catalogHow long since a source record was retrieved[catalog] ttl_days, or an entry's owndays
questionsHow long a question has gone unanswered[due] questions_aftercorpus commits
phasesHow long a bounded inquiry has been in flight[due] phases_afterdays
supersededHow long a node has cited a source that changed since it was last committed[due] superseded_afterdays

The catalog clock reads the interval where it already lived rather than restating it under [due]. A source's TTL is a statement about the source. Two places to set one number means one of them is wrong.

A clock with no interval reports what it measured and is never due. Its remedy is the key that would set it. That is the state of every repository that has not opted in. It is the design, not a degraded mode.

A corpus may also decline a clock outright, under [due.declined]. That is a different thing from never setting one. The clock reads declined, still prints what it measured, and prints the decision record behind it. The record is required, so a decline is something a corpus wrote down and can be asked about.

Three of the five count days and two do not, which is deliberate. How long a question has gone unanswered is a fact about the repository, so its clock is HEAD. A corpus that has not committed has not ignored anything. A source's TTL, a phase's time in flight and a source's new version are facts about the world. The world does not stop moving because nobody committed.

What discharges a clock

due reports; it does not act. Only one of the five clocks is discharged by something propose can draft:

ClockWhat discharges it
indexyidam index-build. A build, not a commit
catalogyidam propose, which already drafts an open: against each expired source
questionsA person. Deciding a question is answered is a resolution event, and Article V confines those to a sangha
phasesA person. Settling a phase or abandoning it is not a mechanical consequence of a finding
supersededA person re-reading the source, and a commit to each node it lists. Any commit touching the node counts

Each clock names its own remedy in the report. The distinction is visible where it matters, not only here.

README blocks

Each of these rewrites its own <!-- REGEN: yidam <name> --> block in the repository's README. That is their purpose, and it is why every one carries a *.

CommandBlock content
regen *Refresh every REGEN block in one pass. --check reports staleness and writes nothing; --commit authors the regen: commit for exactly the files that changed
status *Repository overview: nodes, open questions, catalog, index freshness, genesis
open-questions *Unresolved questions, newest first
corpus-index *Every corpus node by class, with label and link count
catalog-audit *Which catalog sources the corpus cites, and which it does not
index-status *Whether the vector index is present, and how stale against the corpus
agents-index *The domain agents in .yidam/agents/ (no flags)
skills-index *The domain skills in .yidam/skills/, each with its status: (no flags)
crates-index *The domain-computer crates in crates/, each with the capability that runs it (no flags)
packages-index *The domain-computer packages in packages/, each with the capability that runs it (no flags)
bundle-status *Freshness of .yidam/bundle.yiz against the corpus it was built from (no flags)
gates *The gate table: every run: step of every job in .github/workflows/ci.yml, in order (no flags)
routes *The reading routes in AGENTS.md, by occasion, from the vendored routes.yml (no flags)

Four more generators are filed by what they report, not by the fact that they generate. They are vault-status, decisions-log, kuten and practice. yidam regen runs all sixteen.

gates reads the workflow and nothing else. A derived repository's README used to say in a sentence which checks CI runs. The sentence could not follow the workflow. The table can. A job that only runs once a layer exists says which path switches it on. A step that reports without gating says so.

In a derived repository a stale REGEN block is a failing build. Run mise run regen before committing; yidam regen --check is what CI runs.

A block naming a generator that does not exist is a failing build too. No generator answers for it, so yidam regen never writes it and it keeps whatever it holds. A typo in a marker name would otherwise turn a generated block into a hand-maintained one, silently. yidam regen --check names the file, the command and every generator there is. Correct the name, or delete the block.

Only yidam commands are judged. A block another program refreshes is left alone, and so is one a page only shows inside a code fence.

The yidam prefix is reserved. A block marked yidam <name> belongs to yidam, and yidam regen rewrites it whatever wrote it. Mark your own generator's blocks with your own program's name: <!-- REGEN: my-tool <name> -->.

A gated block holds only what every checkout of the commit agrees on. The tree is all they share. So a block may not report anything read from outside it.

Phase counts broke that rule. They read branch refs, and which refs a checkout holds depends on how it was fetched. For one commit there was then no count that passed everywhere. CI and a fetched machine wanted different numbers. Pushing a phase branch also reddened this gate on every open pull request. The counts moved to yidam phases and yidam status --format json, which are reports rather than committed content.

Build artifacts broke the same rule, and for two releases nobody noticed. yidam status reported whether .yidam/index/ existed. index-status and bundle-status reported what those artifacts held. None of it is in a commit. So the gate passed or failed on what a machine had built. CI has neither artifact and never saw it. The three blocks now hold a pointer. The measurements stay in the commands: index-status, bundle-status, doctor and due.

Depth is the other half. genesis reads the repository's first commit, which a shallow clone does not have. Git names the boundary commit a root instead. yidam regen refuses in a shallow clone rather than writing that date. Check out with fetch-depth: 0, or run git fetch --unshallow.

These are the commands to be careful with against a checkout you only mean to read. yidam status sounds read-only and is not — it rewrites the README block. yidam doctor is the read-only overview.

The corpus and its history

CommandWhat it does
graphThe corpus graph: nodes, resolved edges, and the classes that license them
neighbors <node>One node's neighbourhood — the traversal serve --mcp performs. --depth
retrieve <words>Semantic search over corpus nodes — the retrieval serve --mcp performs. --k, --class, --corpora, --where
query <query>A typed path over the resolved graph
pack <query>A query's full answer filled to a token budget, with an account of what did not fit
estimate <query>What a query would cost before you run it
count [query] *How many nodes a query matches. With no query, refreshes every <!-- REGEN: yidam count <query> --> block in the tracked markdown set — the form regen runs (RFC-0043)
diff <range>Node and edge changes between two git refs
check-diff [range]What a code diff names that the ontology does not (RFC-0021). The ontology includes every installed dependency's classes. Defaults to the merge-base with main — this branch's work. An unmodelled-concept row may carry a nearest declared name: one whose --separated word shares a root of at least five characters with a word of the type's name, the two words' lengths within three. A word that ends three or more of the report's type names and appears nowhere else in them, such as error in FetchError, is the code's suffix and is not compared. More shared words win, then the longer root, then the name. It is a lead to check, not a match — no model reads either name
log [range]Commit history classified as testimony or pipeline work. --epistemic, --operational
phasesActive inquiry phases — ma/*, rigpa/* and phase/* branches. State is active, interrupted, settled, rewritten or position. Source says which evidence decided it
replayCorpus health reconstructed across the repository's whole history. --every
cohort <repo>…What a set of derived repositories says about the prelude they inherited: per norm, how many kept it. --paths
decisions-log *Decision records in .yidam/decisions/, newest first. Writes the <!-- REGEN: yidam decisions-log --> block where that directory has a README carrying one (no flags)
sanghaElectors, positions, and settled resolutions
vocabularyThe closed commit vocabulary. --check <subject> tests a subject line before the commit exists
rename <old> <new> *Rename a node or a catalog entry (catalog/old), rewriting every edge into it. --dry-run
migrate <sub> *Change an ontology and every instance that adopted it, as one event. --dry-run
propose *Draft findings as proposed epistemic commits on a propose/<head> branch
run [step] *Invoke the stale capabilities declared in .yidam/capabilities.toml, in dependency order, and commit what each produced with a receipt. Named with a step, runs that step and everything it declares it comes after; with nothing, the whole manifest. --dry-run plans and writes nothing
phase <sub> *start opens a phase and snapshots what it begins from. run invokes its plan, recording each step. settle checks it produced outputs and drafts the merge — a person runs it
cluster <sub> *The same run on Argo Workflows, one pod per act. workflow generates the manifest. network-policy writes the egress fence. pin bundles the branch tip into a vault. step runs one capability against a bundle. land is the one command that moves a ref. admit says whether a run is owed. survey, ask and gather run a gather. status reports recent runs
gather <name> *Ask pinned peers the question in .yidam/gathers/<name>.toml. Their answers land as one cited ? node on a propose/* branch. --dry-run asks and writes nothing
dispatch <question> *Run the agent electors named by --seat, each as its seat declares. Each position lands on a propose/elector/* branch with a receipt. --dry-run runs them and writes nothing

Retrieval, and the corpora it can reach

retrieve takes words, not a query expression. Reach for it before you know the corpus well enough to write one. It is the same call serve --mcp answers, dispatched by tool name. What it prints is what an agent sees.

Without a vector index it degrades to keyword search and says so. That arm ranks by BM25 over the nodes it scanned. With one it ranks by cosine similarity.

--where keeps only the rows whose declared date or number properties satisfy a predicate. The predicate is the text inside a query step's [...]. It is the same grammar, the same check and the same comparison. So the two commands below return the same tenures, and the second returns them ranked:

yidam query 'tenure[began<=1893, ended>?1893]'
yidam retrieve 'courthouse' --where 'began<=1893, ended>?1893'

The predicate runs before the k cut. Without --class it narrows to the classes that declare each property with an ordered type. A predicate that cannot be asked is rejected with query's code. Over a local index built before typed property columns it is where-unindexed; run yidam embed && yidam index-build. A remote index carries no properties and refuses it.

--corpora reaches the other corpora sharing the vector index [index.remote] names. This corpus is always searched, and the named ones are added to it. Every result says which corpus it came from: a yidam://<corpus>/node/<class>/<name> identifier, and a corpus field beside it. A row from another corpus cannot be opened. Its text is all you get. That is why a row the index had to cut says so.

yidam retrieve 'school funding' --corpora ohio-budget,allen-county

A corpus is named either by a nickname this repository declares or by its genesis hash:

[index.remote.corpora]
ohio-budget = "3f2a9c4d1b70"

The nickname is this repository's own. Nothing outside this file knows it, which is why a result carries the hash. yidam index-push --dry-run lists the corpora an index holds, with a row count each. It writes nothing, and it is where the hashes come from.

Only a shared remote index can answer a span. A local .yidam/index/ is one corpus by construction. So a --corpora against one is not an error. It is not silently honoured either. The local index answers, and the report reads scope local. That is what tells you your request did not take effect.

The query language

query, pack and estimate share one language. A query is a class anchor and a sequence of typed hops:

yidam query 'reach -measured-by-> gage'
yidam query 'concept~"hydropeaking" <-exhibits- reach'

Whitespace around a hop is required. -rel-> and <-rel- are single tokens, and that is what lets a hyphenated relationship name be unambiguous. ~"…" is a similarity anchor. It opens on --anchor-k entry nodes (default 1). An anchor is a starting point, not an answer.

Predicates

A step may be filtered by […], comma-separated. Commas are and; there is no or.

OperatorHolds whenDeclared types
=the value equals the operand, compared at the precision the operand was written — so observed_on=2026-08 matches every day in that monthany
!=it does not. On a list, of every element — or claim_tag: [open, verified] would satisfy claim_tag!=openany
~contiguous, case-insensitive substring of the value's serialized textany
< <= > >=the date, or the number, orders that waydate and number only
?written as prop? — the node carries no value for itany

Ordering is date and number only, and asking for it elsewhere is refused rather than answered. Comparing a string would compare text, ranking 10 before 9 and saying nothing about it. yidam query 'reach[regulated<9]' comes back unordered-property with that reason.

A number compares exactly, and = on one is numeric. reach[length_km>9] returns the reach measuring 10, and length_km=7.0 matches a stored 7, so <, = and > are trichotomous. There is no precision rule: 7 is a point where 1893 is an interval. The operand is a bare number. The unit is declared once on the class, as unit: km, and never written in a value or a query.

A comparison runs at the precision the two sides share. began<1900-06-01 holds for a began: 1893: 1893 is before 1900 whatever day it fell on. This is deliberately not ='s rule. Ordering at the query's precision would silently drop every node the corpus knows only to the year.

? after an operator means an absent property satisfies it too. The standing rule is that an absent property never matches, for any operator including !=. ? is how one predicate opts out of it, where a reader can see it.

That is what makes the interval question expressible. The filter is a conjunction, and "ended after 1893 or never ended" is not:

# every tenure open at some point in 1893 — the line of office-holders
yidam query 'tenure[began<=1893, ended>?1893] -of-office-> office'

# questions nobody has closed
yidam query 'question[closed?]'

prop? and prop>?D are one character apart and mean different things. The first tests for absence and takes no operand. The second is a comparison an absent property also satisfies.

FlagApplies toEffect
--selectquery, estimateFields to project: node, class, label, description, body, properties.<name>
--limitquery, estimateBounds the projection, not the traversal — the reported count is always the full one (default 50). With --paths, the number of paths printed (default 5)
--budgetpack, estimateApproximate token budget, 1 token ≈ 4 chars. pack is unbudgeted by default, because a default budget would silently truncate the first pack anybody builds
--at <ref>queryAnswer as of a commit, reconstructed from git objects; the working tree is never touched
--between <a..b>queryAnswer at every corpus-touching commit in the range, as a series
--acrossqueryQuery installed dependencies too. Every result says whose corpus it came from, and no hop crosses a corpus boundary
--paths <from> <to>queryPrint the typed paths between two nodes, shortest first. Each one is a query that returns <to> when run. Takes no query
--max-hopsquery --pathsThe longest path searched (default 4). A cut-off search names the nearest length that would connect

migrate subcommands

A class definition cannot be corrected in place once the class contract gates. Editing it puts every instance in violation until each is fixed by hand. These do both halves together and write a record of what they touched.

SubcommandChanges
classRename a class: its definition, its directory, and every edge that named it
propertyRename a declared property on a class and on every instance carrying it
retypeChange a declared property's type; requotes an instance value where only the quoting differs, refuses the rest
valueRename one value of a declared values: set, on the declaration and on every instance holding it
edgePoint a declared relationship at a different class, at both ends
referencesLift every reference written inside an evidence tag into the node's references: field
findingsLift every paragraph an earlier propose spliced into prose into a yidam: record
routesPut the yidam routes block into AGENTS.md, in place of its whole-file reading list
scaffoldMark the part of ci.yml and CLAUDE.md that a re-vendor updates
locationsRewrite every catalog location of kind doi or pmc as kind: identifier

retype converts an instance value where the two types differ only in how the value is written. A number is an unquoted YAML number. So retyping to number unquotes "24", and retyping to string quotes a bare 24. Each is a repair property-type already names. A value holding no number — about 24, ~24 — is refused, and the refusal says what it tried. A retype into string on a property that declares values: is held to the set. An instance outside it is reported and blocks. Widening the set or fixing the value is the author's call.

value renames one item of a closed set. The set is closed, so neither half can go first. It rewrites the item on the declaration and the value on every instance holding it, in one event. The quoting follows what was written. A value outside the set is refused. So is a rename onto a value the set already holds, which would merge two.

references, findings and locations migrate data rather than the ontology over it.

A detail reading [verified — #362] names an issue no consumer can follow. The migration writes issue/362 to the node. Where the detail was the reference and nothing else, the tag then collapses to [verified].

Where the reference sits inside a sentence, the entry is written and the sentence is left exactly as written. Excising #42 from #42 (agglomeration) would leave (agglomeration).

Nothing but an issue number is written unless the thing it names is on disk. So Auglaize/Mercer and 6a/6b resolve to nothing and stay prose. Nothing is written that reference-not-in-the-grammar would report. It is idempotent; run it with --dry-run first.

Applying it to the two corpora that write most of these tags left yidam lint reporting what it reported before. One claim-tag-malformed finding went away per collapsed tag. That is 535 → 399 in one corpus and 534 → 459 in the other.

findings is the one-time lift that propose left behind. Before the record, a finding was an English sentence appended to description:. It was found again by grepping the prose for its own opening words. The paragraph's check and the finding's words become a record under the node's yidam: key, and the paragraph goes.

A paragraph reworded past recognition stays prose. The reconstruction is exact — the marker, the check in brackets, and the fixed framing sentence — or it does not happen. A sentence an author rewrote is theirs, and guessing where the tool's words ended would edit somebody's argument. The report counts those rather than listing them as work to do.

Two numbers move for a corpus that runs it. The [open] claim counts drop, because a carried question stops being counted among the claims the corpus makes. open-questions does not change: the record is still an open question on the node. That asymmetry is the thing the paragraph could not express. It is idempotent; run it with --dry-run first.

locations moves each catalog location of kind doi or pmc onto kind: identifier. The value moves under its scheme unchanged:

- kind: doi                      - kind: identifier
  value: 10.1167/tvst.8.5.14  →    value: doi:10.1167/tvst.8.5.14

A value that already carries its scheme keeps it rather than gaining a second. The quoting follows what was written. Run it with --dry-run first:

yidam migrate --dry-run locations   # what it would rewrite, and what it leaves
yidam migrate locations

A value that is a URL is left alone and reported. https://doi.org/10.1/x holds a DOI. Taking a link apart to find one is a reading of it, and that reading is yours. Rewrite it as kind: url, or as kind: identifier with the id the link holds. A block-scalar value or a flow mapping is reported too. So is a rewrite that does not read back as written. Each leftover is still a catalog-location-malformed finding, and the report lists it.

propose is deliberately small

Three acts only. open records a finding's question against the node it is about. withdraw deletes a node this corpus declared over-collected via [propose] withdraw_uncited_after. close retires a question this command opened whose finding is gone.

Nothing merges itself and nothing synthesizes — no edge is drawn, no claim is re-tagged, no node is authored. It writes git objects and one ref. The working tree, the index and HEAD are untouched, so it is safe to run mid-edit. RFC-0020 has the argument for why the surface is this small.

gather cites its peers and merges nothing

A gather asks every installed tonpa peer one question. You write it in .yidam/gathers/<name>.toml:

question = "What peak discharge has each gage recorded?"
query    = "gage[state=VA]"   # one step, in this corpus's own classes
answer   = "peak_cfs"         # the property whose value is the answer
key      = "station"          # optional: answers under one key are compared
lands_as = "question"         # the local class the question node is written as

[peers.usgs]
classes    = { gage = "station" }
properties = { station = "usgs_id", peak_cfs = "flood_peak", state = "state" }

Each [peers.<name>] says which of that peer's classes and properties mean yours. Nothing is matched by spelling. A class or property the correspondence leaves out makes that peer unaligned, and it is not asked.

yidam gather <name> translates the query per peer and runs it against the pinned bundle. Every answer becomes a cites: entry with the package, node, the bundle's commit, and a verbatim span. It lands as one ? … node on propose/gather/<name>/<head>, as an open: commit. Peers that give different answers under one key get a second ? node citing each answer.

Nothing is imported, no edge is drawn, and no claim is merged. Deciding between answers is yours. Every peer is reported: answered, empty, unaligned, refused, missing or undeclared. A path dependency is refused, because a working tree has no commit to cite. A repeat over unchanged peers writes nothing and reports unchanged. Each asked peer gets a v2 receipt at .yidam/runs/gather/<name>/<peer>.yml, in the first commit.

dispatch runs an agent seat and proposes what it wrote

An agent seat declares what runs it in .yidam/sangha/dispatch/<name>.toml, on its own ma/<name> branch:

model   = "claude-opus-4-8"          # must equal the row's Model
version = "1"                        # must equal the row's Version
run     = ["sh", "elector.sh"]       # the argv
reads   = ["elector.sh", ".yidam/corpus/**"]
config  = ["elector.sh"]             # the files hashed into Config

yidam dispatch <question> --seat <name>… reads the declaration, the registry row and the corpus at each seat's tip. A row must already record the model, version and config hash. A seat whose row does not is refused, and the report gives the hash to record. A human seat is refused.

The elector runs in a scratch tree with $YIDAM_SEAT, $YIDAM_QUESTION, $YIDAM_POSITION, $YIDAM_MODEL, $YIDAM_MODEL_VERSION and $YIDAM_CONFIG set. It may write .yidam/sangha/positions/<name>-<question>.md and its commitments file, and nothing else.

The position and a receipt land as one open: or revise: commit on propose/elector/<name>/<question>/<tip>. The seat does not move until you take the commit: git merge --ff-only <proposal>, or git cherry-pick -S <proposal> if the row binds a Key. A repeat over an unchanged tip reports unchanged. --dry-run runs the elector and writes no branch.

The report derives independence: over the seats that ran and never counts positions. The command exits non-zero if any seat is refused or fails.

cohort reports about the prelude, not about a corpus

replay and kuten check answer about a corpus. Neither can see that a rule lost everywhere. That is the question docs/post-genesis-measurement.md had to ask by hand.

yidam cohort ../repo-a ../repo-b ../repo-c

The rows are norms and the columns are repositories. A rule every derivation fails is one finding about the rule, not five findings about five corpora.

Read-only: nothing is checked out and nothing is written. It is safe against repositories somebody is working in, and it exits zero however much lost.

Two controls, both printed rather than divided out. A repository whose vendored prelude does not state a rule is reported as vintage, never as having broken it. Members are ordered by authored commit count, the count beside every row. No rate is normalised by age.

A rule a repository never had the chance to break is unmeasurable, not kept. Every share is quoted over the repositories the question could be asked of. Norms that held are reported too: a negative result about coverage is the only durable record that coverage was checked.

Members are lettered rather than named. Most derived repositories are private, and this report is written to be pasted somewhere. --paths opts back in.

One section is a measurement with no verdict per member: accretion. It fits the two candidates RFC-0028 named for inquiry's retired classes bands. Each is replayed over every member's history. A candidate any member leaves as it ages is printed as rejected. The trailing window is read over the last 25, 50 and 100 authored commits. RFC-0028 names no window, so the candidate is rejected only if it fails at all three.

Each member also carries upstream: how many commits cite an issue on the template's tracker. It is a count. No norm is scored on it.

run is a plan, and only an operational commit advances your branch

.yidam/capabilities.toml declares what may run. Per entry: a kind, the argv to invoke, the globs it reads and writes, and the commit verb it authors.

The kind is one of the three capability types the domain computer is built from — calculator, connector, featurizer. All three are declarable and only a calculator is invoked. A plan holding one of the other two is refused before any of it runs. The refusal names the step, its kind, and why that kind is not invoked. A connector reaches a network and a credential path. Feature engineering is declarable and not yet built.

[capability.travel-tier]
kind   = "calculator"
run    = ["sh", ".yidam/capabilities/travel-tier.sh"]
reads  = [".yidam/corpus/**", ".yidam/capabilities/**"]
writes = [".yidam/computed/**"]
verb   = "compute"

yidam run travel-tier checks the declared reads out of HEAD into a scratch directory. It invokes the step there, with $YIDAM_IN, $YIDAM_OUT, $YIDAM_STEP and $YIDAM_INPUT_COMMIT set. A step whose reads admit a corpus node gets a fifth, $YIDAM_GRAPH. It names a file holding the corpus as yidam parsed it. Classes, labels, properties, and every link with its target already resolved. So a calculator writes no parser of its own. A list arrives one row per item, keyed <key>[<i>]. A record in a list adds one row per field, keyed <key>[<i>].<field>.

That file is sliced to what the step reads. Its digest is part of the input state. Changing how yidam resolves a link therefore makes every calculator's step stale. A step that reads no corpus node is handed no $YIDAM_GRAPH. The name is removed from the environment rather than left empty, so a script can test for it. What the step wrote lands as one commit on the current branch. A receipt lands with it, at .yidam/runs/<step>.yml, naming the input commit and every digest.

Both declarations are load-bearing. The step sees what reads resolves to and nothing else. It cannot quietly depend on a file it did not name. A step that writes outside writes is refused, with nothing committed.

The verb decides the destination. An operational verb advances the current branch. An epistemic verb — establish, revise, resolve — loads and runs just the same. But its commit lands on propose/<head>, and the branch does not move. A person merges that branch to accept it, or deletes it to reject it. So a run may open a question and may not decide one.

There is no route field and no policy key. A corpus that could write that permission for itself could license its own runs to author establish: on its baseline.

It writes git objects and refs: the current branch, propose/<head>, or both when a plan holds both kinds of verb. The working tree and the index are untouched, so it is safe to run mid-edit. It therefore leaves your checkout behind HEAD. The report says so, and names the path-scoped git restore that syncs it. A re-run whose inputs have not moved writes no commit at all.

A manifest is a plan. An entry's after names the steps it waits for. yidam run travel-tier runs that step and everything it comes after, dependencies first. yidam run alone runs the whole manifest. An after that forms a cycle is refused with the cycle named, and nothing runs. A step may not come after an epistemic one. That step's output is on propose/<head>, not in the tree a dependent is checked out from. So every epistemic step runs last, at the head the branch steps leave.

A step whose input state matches its committed receipt is skipped, not invoked. The report names each skipped step and why. A step that reads what the repository does not hold declares ageing_days. It is re-run on that cadence even when its inputs have not moved. On the branch, that re-run lands a commit even when the answer is the same. The commit records the check. An epistemic re-run that reproduces its pending proposal lands nothing. The proposal already holds that answer.

--dry-run resolves the plan and reports what is stale. It invokes nothing and writes nothing: no commit, no ref, no receipt.

RFC-0026 has the argument for what a run may author. Its §6 has the argument for freshness.

The second arm: a calculator that is a typed function

run takes a second shape. A table naming one script declares a typed calculator (RFC-0042).

[capability.class-of]
kind   = "calculator"
run    = { gluon = ".yidam/capabilities/class-of.glu", calls = 2000000 }
reads  = [".yidam/corpus/**", ".yidam/capabilities/**"]
writes = [".yidam/computed/**"]
verb   = "compute"

The script's entry point is declared Corpus -> Computed. It is applied to the resolved corpus in this process. There is no scratch tree and no child process, so none of the five variables is set. A calculator that cannot perform an effect needs no sandbox to stand in.

Corpus is a record of three fields. nodes is every corpus node its reads cover, with each node's properties, its links and where each link resolves. classes is the ontology. signals is what the calculators before it committed. It is read back from the .yidam/computed/ files its reads cover. One row per node, in the shape the entry point returns.

That third field is what lets a pipeline's second stage be typed. A calculator whose input is another capability's answer declares that file under reads, and is handed it. One that declares no computed path is handed an empty signals. So is one in a corpus where nothing has been computed, and a script cannot tell those apart.

A row's node is a reference — gage/canyon-outlet — and so is a node's id. A script that wants both matches one against the other. The signals are not attached to the nodes, and the reason is the declaration. A second stage may read a computed file and no corpus path. Its signals are then about nodes that are not in nodes. Attaching them would mean the typed arm needed a wider declaration than the shell arm to compute the same rule.

Computed is a record of two fields and both are required. signals is the table below, one row per node. summary is what the calculator counted about the run as a whole. It is written under a top-level summary:, not against any node. A calculator that summarizes nothing writes summary = [] and the key is left out of the document. Gluon records are exact, so a script that omits either field is refused as not a calculator.

It must still declare its own script under reads. A declaration that does not is refused by name, because the script's digest belongs in the input state. That digest goes in the receipt too. So editing a typed calculator makes its step stale, exactly as it does for the shell arm.

calls caps how many calls the script may make. Omitted, the binary's default applies. A calculator that does not finish is refused rather than warned about.

The arm is outside the default build. The downloaded binary parses this declaration and plans it. It then declines the step by name, and says which feature would run it. A corpus that wants the downloaded binary to run it writes the sequence form.

A calculator writes into .yidam/computed/, which is committed. Nothing read it until #1028. A corpus could compute how far each assertion may travel, and commit the answer. It had no way to ask a search for the nodes that may leave the repository.

A file there is read as a signal table when it says it is one. Saying so is a top-level format_version: 1 and a signals: list. Every row of that list names a node.

format_version: 1
signals:
  - node: gage/canyon-outlet
    travels_as: open
    downgraded: true

Every other key in a row is a signal about that node. yidam embed attaches it to that node's embedding record as a signals object. The object is absent where there is none. A corpus that computes nothing writes the records it wrote before.

A signal is deliberately not folded into the embedded text. A tier a query can filter on is worth more than one a sentence embedding is nudged by. Concatenating it would also move every vector already built. Index columns are a separate decision and are not made here.

Rows are keyed in the reference grammar and in nothing else. That is gage/canyon-outlet, node/gage/canyon-outlet, or the absolute yidam://<corpus>/node/<path> form naming this corpus. A revision pin is refused rather than ignored. A signal computed against a past commit is not a signal about the node as it stands. A signal name is repository-wide. Two files claiming one name is refused with both named, rather than resolved in favour of either.

A file with no format_version is listed and not read. A calculator whose output is a report rather than a table stays legal. yidam doctor's computed check reports what is there, and what no capability declares it writes. It also reports whether the answer still stands. That includes the window right after a run, when the files are at HEAD and not in your checkout. yidam status --format json carries computed_files, computed_signals, computed_nodes and computed_unowned.

A phase carries what it began from, and says so after an interruption

PHASES.md specifies a phase as a branch, a declared input state, a body of agent work and a --no-ff merge. Nothing in a repository held any of it. yidam phases read a phase's state off its ref's namespace, so active meant a ref matching a glob. A phase whose run died halfway through looked exactly like one opened this morning.

yidam phase start outcome-axis --type Investigation
git switch phase/outcome-axis
yidam phase run
yidam phase settle

start opens phase/<name> and commits the snapshot to .yidam/phases/<name>.yml. The snapshot is the baseline commit, the digests of .yidam/capabilities.toml and .yidam/config.toml, and the kuten revision. The declared --type is checked against the phase types the vendored kuten declares. A repository holding no kuten records the type and validates it against nothing. There is no default list, and inventing one would put a phase-type roster in this binary (RFC-0028 §3).

The record is committed rather than written to the working tree, and that is what makes it useful. yidam phases reads remote-tracking refs. A record living only on disk would answer for the one branch you are standing on. The commit's verb is scaffold: — operational, so the pipeline advanced and no understanding changed.

run resolves the manifest's plan and writes it to the record before the first step is invoked. Each step's completion is committed as it happens. That order is the whole design. A record that learned its plan on the way through could say which steps ran, and never which did not. A step already recorded is not re-entered, so a run killed partway completes rather than restarts.

The plan holds only the steps this binary can invoke. A kind = "connector" is refused by design, so it is left out of the plan. So is any step that runs after a refused one. The report names each under refused, with its reason. If no step can run, nothing is committed. The commits move the branch and not your checkout. The run prints the git restore line that takes them, whether it finished or stopped.

Until every step of the resolved plan is recorded, the phase reads interrupted. That is true in yidam phases, in yidam status and in yidam due's phase clock. The state has no ref shape — only the record knows it.

settle refreshes the REGEN blocks, checks the phase produced outputs and drafts the merge subject. Without a flag it commits nothing. phase: is an epistemic verb. due states who decides a phase is done: a person. Merging a phase, or abandoning it, is not a mechanical consequence of a finding. So settle validates and hands back the commands to run.

settle --merge writes them, once you have decided. The regen: commit and the --no-ff merge are authored by yidam phase, with you as committer. Then it switches to the baseline and deletes the phase branch. --summary "<text>" replaces the drafted tally in the subject. Nothing is pushed.

It refuses anything it would have to resolve. The baseline must be an ancestor of the phase. The checkout must be the commit, apart from the blocks settle staged. The baseline must not be checked out in another worktree. Each refusal says Nothing was merged. It never merges a propose/* branch. RFC-0026 §2 and PHASES.md state the licence.

The REGEN step is regen --check, then regen, then regen --check again. A stale block is refreshed and staged. The printed sequence then opens with the regen: commit, because git merge --no-ff refuses a dirty index. A block still stale after the refresh, or one no generator writes, is not ready, with the remedy regen --check prints. Nothing is refreshed from a checkout that differs from HEAD. A block generated from that tree would not hold against the commit. settle names the files and the sync instead.

settle also asks one question of the template. Did the phase add files outside the laid-out directories, and cite no issue on the template's tracker? Then it drafts a from:derived-repo report and prints the gh issue create line. Nothing is filed. The laid-out directories are the sections of guidelines/directories.md.

Like run and propose, all three write git objects and one ref, except settle --merge. They touch neither the working tree nor the index, so they are safe mid-edit. They therefore leave your checkout behind the branch. git reset --hard HEAD or a path-scoped git restore syncs it.

A phase opened by hand is not broken. git switch -c phase/<name> is PHASES.md's own documented flow, and every phase in every repository predates this record. Those rows are still listed, and their state is still read from the ref. The table says how many rows are an inference. RFC-0028 §3 ranks the two sources rather than replacing one with the other. The ref answers what is this ref; the record answers what happened in this run. Only the ref can say whether a phase has landed on the baseline.

A run on a cluster is the same run, and the ref write is a credential

cluster-runs.md is the deployment guide. This is the command surface.

yidam cluster workflow --cron "0 6 * * *" > yidam.cronworkflow.yml
yidam cluster network-policy --executor 10.0.0.1/32 > deploy/corpus.netpol.yml
yidam cluster admit --remote git@host:corpus.git
yidam cluster pin   --remote git@host:corpus.git --vault-url file:///var/yidam/vault --out pin.json
yidam cluster step  travel-tier --bundle <digest> --vault-url file:///var/yidam/vault --out step.json
yidam cluster land  --step-output @step.json --remote git@host:corpus.git --vault-url file:///var/yidam/vault
yidam cluster survey units --bundle <digest> --vault-url file:///var/yidam/vault --out survey.json
yidam cluster ask   --ask '<one of survey.asks>' --vault-url file:///var/yidam/vault --out ask.json
yidam cluster gather units --bundle <digest> --asked '[…ask records…]' --vault-url file:///var/yidam/vault --out step.json
yidam cluster status --remote git@host:corpus.git

workflow reads .yidam/capabilities.toml and [cluster] in .yidam/config.toml, and prints an Argo Workflow. With --cron it prints a CronWorkflow that admits itself. With --on-push it also prints an Argo Events EventSource and Sensor. A push to the branch then submits the run, which admits first. It runs in a checkout and writes nothing. Every flag it takes overrides one [cluster] key. --image names the container image every pod runs.

network-policy prints the NetworkPolicies that limit what each kind of pod reaches. Every pod reaches DNS and the API server. remote pods reach the remote. vault pods reach the vault and nothing else. internet pods reach anywhere but the remote. A policy matches addresses, so each place is a CIDR list from [cluster.egress]. Each flag replaces its key, and each may be repeated:

FlagReplacesWhat it is
--executor <CIDR>[cluster.egress] executorThe API server's endpoint addresses. The generator refuses without them
--remote-cidr <CIDR>[cluster.egress] remoteWhere the git remote is
--vault-cidr <CIDR>[cluster.egress] vaultWhere an s3:// vault's endpoint is
--sts-cidr <CIDR>[cluster.egress] stsWhere STS is, for pods that assume a role

--vault-url overrides the vault's url, as it does for workflow. Apply the output with the corpus's overlay. cluster-runs.md says what each address should be.

admit clones the branch and reads due's clocks, the manifest's staleness and the count of open propose/* branches. Its record says admitted either way, and it exits zero either way. A corpus with nothing owed is not a failure. The generated workflow submits no step when admitted is false.

pin bundles refs/heads/<branch> and puts the bundle in the vault. The record names the sha and the bundle's digest. Everything downstream reads the pin, never the remote.

step fetches a bundle, clones it into scratch and invokes one capability there. It is run's loop body with the ref write removed. What it builds is a commit in a second bundle, and its record carries {sha, class, verb, receipt, bundle}. The command has no --remote and no --branch. That is the point of it, and a test holds it. Its --image is the reference the pod started from. The receipt records it only when it pins a digest.

land reads a step record and verifies the bundle. It reads the class off the commit, not off the record. Then it pushes the commit to main or to propose/<input>, with --force-with-lease. If the branch moved, it rebuilds the commit on the new tip. That holds only when nothing the step reads moved. Otherwise it refuses. Its last act is a fresh pin, which the next step reads. With --git-auth github-app it mints an installation token from --github-app-key and pushes over HTTPS. The token goes to git's environment and nowhere else. --github-app-id names the App. --github-api is GitHub's REST API, https://api.github.com by default. For GitHub Enterprise Server, pass https://<host>/api/v3.

survey, ask and gather run a gather, one pod per peer. None takes --remote. survey plans at the pin and translates each peer's query there. Its asks list feeds the fan-out. ask takes the peer's bundle from the vault, or fetches its lock url when the vault lacks it. It checks the bytes against the lock, runs the query, and always puts a record in the vault. Like step, it records --image in the peer's receipt only when it pins a digest. gather holds every record to the plan. A peer with no record is refused, never dropped. It builds the tree a local yidam gather would, and land pushes it to propose/gather/*. The lander refuses a gather commit that is not open:, or that touches other files. The generated workflow adds these four tasks per file in .yidam/gathers/.

The records are the pod contract, versioned by format_version. Under --format json each sits under a record key in the report envelope.

status reports what the recent runs did. It reads the records Argo keeps on each Workflow, through kubectl or from --workflows <file>. For each run it gives the admission and its reason. It gives each step's outcome: landed, proposed, refused with the lander's reason, or not reached. It gives each gather peer's outcome. With --remote, or [cluster] remote, it reads the branch tip and says whether each proposal is still open. A run whose records are gone is unknown, with the reason. It never reads a log, and it commits nothing. Under --format json it is a report, not a pod record. --limit sets how many runs it shows, newest first; the default is 5. --namespace overrides [cluster] namespace, and --context picks the kubeconfig context. --corpus is the corpus's name in the cluster's objects. It defaults to the root directory's name.

Every pod that reads the vault takes the same vault flags:

FlagWhat it is
--vault <NAME>The vault's name, which decides which credential variables are read. Default default
--vault-url <URL>file:///path or s3://bucket/prefix
--vault-region <REGION>The S3 region, for an s3:// url
--vault-endpoint <URL>An S3-compatible endpoint, for a store that is not AWS
--vault-path-styleAddress the bucket in the path rather than the host

The generated workflow writes these from the config, so a person rarely types them.

When land refuses, it still writes a record to --out, holding refused and the reason. The pod still fails. That record is what status shows.

Index and embeddings

CommandWhat it does
embed *Extract embedding text from corpus instances to .yidam/embeddings/. --no-catalog, --dry-run
index-build *Build the vector index as Arrow IPC (corpus.arrow), which retrieve and the web shell read. --model. Needs vector-read, which is in the default set
index-pushMirror .yidam/index/ into the vector bucket [index.remote] declares. --dry-run, --create. Needs vector-read, which is in the default set

embed walks .yidam/catalog/ by default. In a real derived corpus the catalog was 51.3% of the indexable text, against the corpus's 41.9%. Leaving it out had been a scope decision nobody made on purpose.

embed --dry-run composes every record and writes none of them — not the record, not .yidam/embeddings/, not the directory. It reports what a push would carry: the composed text distribution, split by node and source. It names the largest row against the 40 KB per-vector ceiling. It names the largest filterable half against the 2 KB one. Any row whose text would be cut is listed. Run it before declaring a remote index. It is also how RFC-0033 §8.3 was measured, over corpora nothing was written into.

index-push is a mirror. It writes every row the local index holds, and deletes the ones this corpus no longer has. A node removed from the corpus stops being findable. It never touches a record that is not this corpus's.

Several corpora may share one index. A push reports the ones it found there — shares: <corpus> (<n> row(s)) — from the listing it already reads. With --dry-run that is a read-only way to ask an index who is in it. It is how a [index.remote.corpora] table gets written.

One index holds one vector space. An index carries a single embedding contract, and every push overwrites it. So a push into an index built with different embedding settings is refused. Two spaces in one index would let a spanning query rank one corpus's rows against the other's. The scores would be in the range a correct ranking has. Where the contract cannot be read at all — a write-only credential, a throttle — the push says so and proceeds.

The index carries typed properties, and the push does not forward them. index-build writes a properties column: a JSON object of each row's declared date and number values. A row with none holds null, and meta.json lists the columns. That is what retrieve --where reads. The push leaves the column behind, deliberately. A remote index refuses a where rather than answering it from metadata that only sometimes fits (RFC-0041).

Artifacts

Bytes a corpus rests on or produces are large, derived, or licensed — a fetched source, a built index. Git is the wrong place for all three. A vault holds them; the repository holds the record of which bytes. RFC-0023 states the constraint: a vault stores bytes, git stores the record of them, . So every pointer into a vault is a committed file. Losing a vault costs no knowledge claim, only the time to re-fetch.

CommandWhat it does
vault listEvery store this repository declares — its audience, what it holds, how many artifacts route to it
vault put <path> *Hash a file into the local cache; prints the content address on stdout
vault get <sha256> *From the cache, else from the vault. --out also writes a named copy
vault path <sha256>Where the artifact sits locally, or exit nonzero — so … || fetch works
vault verifyRe-hash every cached artifact; exits nonzero if any is not what it claims
vault push *Upload what the corpus names and the vaults lack. --dry-run prints the exact string that would be signed; --artifact and --vault narrow; --index/--embeddings/--bundle send what this repository computed instead, and --commit authors the index: or bundle: commit for the lock entry it wrote
vault pull *Fetch what the corpus names and the cache lacks; --vault narrows; --index/--embeddings/--bundle fetch and unpack what .yidam/index.lock records
vault statusWhere each named artifact goes and where it is, grouped by store. --remote asks each vault — one HEAD per record, never a bucket listing
vault gcReport cached artifacts no committed file names; --yes deletes them
vault materializeHardlink cached artifacts into .yidam/vault/<slug>/ under names a person can open; --entry narrows
vault-status *Writes the <!-- REGEN: yidam vault-status --> block in README.md. Committed files only — never the cache, never the network. yidam vault status, a hyphen apart, is the read-only report; yidam regen runs this one with the rest
catalog-fetch [entry] *Follow a catalog entry's declared address, cache the bytes under their digest, record them in artifacts:, and commit it as refresh:. --location narrows to one address; --bind name=value fills a url_template slot; --archive adds the nearest Wayback capture of each identifier whose pack allows it; --dry-run resolves and writes nothing. Exits nonzero if a publisher refused a location
catalog-extract [entry] *Take each reading an artifact lacks — a PDF's text, a source pack's extract, or a zip member its location lists under members: — file it in the cache, record it under the artifact in readings:, and commit it as extract:. --dry-run reports and reads nothing
catalog-reconcile [entry] *Rewrite a drifted used-by list to the citations, which are authoritative, and commit it as reconcile:. --dry-run reports and writes nothing
source checkHold every source pack and every prelude_sources pin to the pack format, offline, and run each transform over its fixtures. Exits nonzero on an error
source listPrint each enabled pack, its schemes, and what its transport needs: YIDAM_CONTACT and any auth variables, each marked set or not. Never prints a value
source search <pack> <query>Ask the pack's [search] endpoint and print the identifiers it answers with. --limit caps them; --offline answers from the pack's fixtures. Writes nothing
source add <scheme:id>… *Resolve each identifier through its pack and commit a draft catalog entry, obtained: false, as catalog:. --fetch then runs catalog-fetch on it; --offline answers from fixtures; --dry-run writes nothing

Following an address

A catalog entry has always carried a machine-shaped address. The gate has always checked its syntax: catalog-location-malformed fails a url_template with no {…} slot. Until #720 nothing dereferenced one. url_template appeared four times in the CLI — twice in the type, twice in that check.

catalog-fetch is the path between them. Four steps: address → bytes → cache → record → commit.

$ yidam catalog-fetch local-registry --location 0
local-registry
  location 0 — /repo/sources/registry-2026.csv
    sha256:9f2c… (118 bytes)
    push route: sources (s3://newsroom-sources/yidam)
  a41f3c8 refresh: local-registry from sources/registry-2026.csv

The four location kinds are not equally followable, and the difference is reported rather than hidden:

KindWhat a fetch does
fileReads it, relative to the repository root. No network, and no feature gate — this works in every build
urlGETs it
url_templateGETs it once every slot is bound. A partly-bound template is refused, naming the slots left
addressNothing. A records office is a real way to hold a source and not an endpoint; it is passed over in silence beside a location that can be followed, and explained when named by --location

A template is refused rather than half-filled. A URL still carrying a literal {year} is one a server answers with something. That something would then be committed as a digest this corpus obtained.

$ yidam catalog-fetch local-registry --location 1
local-registry
  location 1 skipped — `https://registry.example/…?year={year}` has a slot nothing bound:
    `{year}`. Supply it — for example `--bind year=…`

Bindings are not inferred from the corpus. A gage node carrying properties.parameter: "00060" looks like it could fill a slot of that name. It might. That has not been checked against more than one corpus, so nothing guesses.

What lint holds a catalog entry to

Two checks read a catalog entry's shape. Both are Warn.

catalog-type-unknown reports a type: outside the closed set. The set is paper, dataset, api, database, statute, report, standard, document and other. A word outside it is reported, never mapped. The finding asks which of the four newer types it is. primary gets one more clause: it is a standing, not a form. A primary source can be a statute, a dataset or a report. Only someone who read it can choose, so no migration maps it.

catalog-location-malformed reports a location with no kind or no value. It also reports a kind outside url, url_template, address, file and identifier. Several locations need a description each. A value must match its kind:

KindRefused
urlA value that does not start http:// or https://, or that carries a # fragment
url_templateA value with no {…} slot
identifierA value that is not scheme:local-id, has no local id, or is a URL

Once the corpus enables a source pack, an identifier is also held to the packs. Its scheme must be one an enabled pack declares, and its local id must match that scheme's pattern. When a template pack declares the scheme, the finding names the pack to pin. A corpus that enables no pack is held to the shape alone. See Source packs.

A fragment is never sent to the server, so catalog-fetch downloads the whole file. The finding names that file. To read one file inside a zip, list it under members:. See Read a file inside a zip.

kind: doi and kind: pmc were written before identifier existed. The finding for either names yidam migrate locations, which rewrites it. See migrate subcommands.

Where a fetch stops

At the record, and at the local cache. Two boundaries, each closing a shortcut.

It never authors a node. That is establish:, which is epistemic. RFC-0026 confines those to a person. A run authors operational commits directly. Every epistemic commit goes to a proposal branch. refresh: and reconcile: are both operational, so both land on your current branch. The commit writer checks its own subject against classify_commit rather than trusting its callers.

It never uploads. Bytes land in the machine-wide cache. Sending them is vault push, which consults redistributable. That is a decision a person makes after looking at what arrived.

The record carries the digest, the size, the media type the server declared, the date, and the location. On a first capture it omits two fields deliberately. vault: is left out because routing is .yidam/config.toml's decision, and freezing it per record would make that routing dead. redistributable: is left out because it is a licensing fact, and no HTTP 200 establishes one.

On a re-capture of a location the entry already holds a record for, both are carried forward from that record. They are your statements, not the command's. Carrying one reproduces a decision you made about that source. Inventing one would assert a licence on the strength of a response code.

Only the latest record for that location is consulted. Change your answer and every edition after it inherits the new one. A record that says nothing carries nothing forward.

A re-capture with new bytes is a new version of the source. Every node citing the entry was read against an earlier one. The report and the refresh: commit both name them, and yidam due keeps naming each one until a commit touches it:

$ yidam catalog-fetch local-registry --location 0
local-registry
  location 0 — /repo/sources/registry-2026.csv
    sha256:4be1… (131 bytes)
    push route: sources (s3://newsroom-sources/yidam)
  c07d2e9 refresh: local-registry from sources/registry-2026.csv
  a new version — 1 node(s) cite an earlier one, and `yidam due` holds them until each is re-read:
    .yidam/corpus/record/hydrant-count.yml

"Of the source" means the same location and the same resource:. A location serving one file per state, or an API asked at several endpoints, is many sources. Record which one each artifact is with resource:. A new state's file is then not read as a new version of the others.

The refresh: commit body names each field it carried and the digest it came from. So a licence appearing on a new record is reviewable as continued rather than established.

Without this, one licensing decision had to be re-entered by hand per edition. vault push refused every re-capture of a source you had already cleared. A vault: none hold-back lapsed the same way, routing new bytes to a store you kept the previous edition out of.

Re-running is free. A fetch that finds bytes the entry already records writes nothing and commits nothing. That is what makes it safe to put on a ttl_days clock. A source that changed appends a second record beside the first rather than replacing it. Overwriting would delete the provenance of every claim resting on the older bytes.

Source packs

A source pack names a family of sources by identifier (RFC-0048). A catalog location kind: identifier, value: doi:10.1167/tvst.8.5.14 names the scheme doi, and a pack declares how that scheme resolves to an address. Source packs is the task page: pinning, writing, every manifest key, and the transform contract. A pack is a directory:

<pack>/
  pack.toml          # schemes, resolution, transport, defaults, fixtures
  entry.md           # the body a new catalog entry starts from
  transforms/*.glu   # optional
  fixtures/          # recorded responses
[pack]
name    = "scholarly"
version = "0.1.0"

[scheme.doi]
pattern = '^10\.\d{4,9}/\S+$'
type    = "paper"
resolve = { template = "https://api.crossref.org/works/{id}", media = "application/json" }

[transport]
contact      = "required"
min_interval = "100ms"

[fixtures]
"doi:10.1167/tvst.8.5.14" = "crossref-tvst.json"

A template slot is {id} or a named group in the pattern. [fixtures] maps an identifier to a file under fixtures/, and every file there must be named. A manifest refuses a key it does not know.

A corpus writes its own packs in .yidam/sources/. It vendors others by pinning them in .yidam/decisions/proposals.yml:

prelude_sources:
  - scholarly@^0.1
  - us-oh-legislature@^0.3 from github.com/<owner>/<corpus>@<40-hex commit>

mise run yidam-vendor-update copies each pinned pack into .yidam/.vendor/sources/. A pin without from names a pack the template ships. One with from names a pack another corpus wrote, at that commit. The copy records the pin and its origin under [vendored].

A range is ^X.Y, ~X.Y.Z, =X.Y.Z or a bare X.Y, with cargo's bounds. A commit holds one version of a pack, so a from range is checked, never resolved. The update refuses a pin the pack's version does not satisfy and copies nothing. An authored pack is read before a vendored pack of the same name.

source check is the CI gate for an authored pack. It reads no network, and exits nonzero on an error. --format json emits passed, packs and pack_findings; text is the default.

A scheme may name two transforms, describe and extract, as .glu files under transforms/. They run in the calculator arm's closed prelude, over the response the scheme resolves to. The binary parses that response as JSON, XML or CSV, by the scheme's media. A describe returns a draft entry. An extract returns a reading of the artifact, which catalog-extract records. source check admits each transform and runs it over the scheme's fixtures. A then that reads describe.<key> must be answered by at least one fixture.

Transforms need the source-transforms feature. A build without it checks the rest of the pack and says it did not check the transforms. catalog-extract reports each reading it could not take as skipped.

A pack may declare one search endpoint. source search asks it:

[search]
scheme   = "doi"
template = "https://api.crossref.org/works?query={query}&rows={limit}"
media    = "application/json"
items    = "message/items/*"
id       = "DOI"
title    = "title/0"
fixtures = { "retinal imaging" = "crossref-search.json" }

Each field matching items is one result, and * matches any one name. id and title are paths below it, named the way a transform names them. A search is read by path, so it needs no transform. JSON is read in every build. XML and CSV need source-transforms.

source add writes a draft entry for each identifier and commits each as catalog:. It follows the scheme's then, and adds each identifier found as another location. Where the build runs transforms, describe fills the title, type, date and description. The entry is obtained: false, with the pack's ttl_days. Its body is the pack's entry.md, or a template of eight prompts. source add never writes a node and never writes obtained: true. It refuses an identifier the catalog already holds. If one identifier fails, it writes nothing.

source search and source add ask a publisher the way catalog-fetch does, under the pack's [transport], described below. A pack missing its contact or a credential is not asked. --offline answers from the pack's recorded fixtures and asks nothing.

Fetching an identifier

catalog-fetch resolves a kind: identifier location through the enabled pack that declares its scheme. It then fetches the address as that pack's [transport] says:

[transport]
contact      = "required"
min_interval = "250ms"
auth = [
  { env = "COURTLISTENER_TOKEN", header = "Authorization", prefix = "Token " },
  { env = "BLS_KEY", query = "registrationkey" },
]
archive = "wayback"
  • Contact. Every request sends User-Agent: yidam-catalog/<version>, plus (+$YIDAM_CONTACT) when that variable is set. A pack with contact = "required" is skipped without it, before any request.
  • Auth. Each credential is read from the named variable and sent in the header or query parameter the pack names. A bare name is refused, since it says nothing of where to send it. No credential reaches the report, the entry or the commit. A credentialed request follows redirects only on its own host.
  • Spacing. min_interval holds between two requests to one host, across one run.
  • Atomic write. The body streams into a .partial file and is renamed once complete.
  • Refusals. A non-success status is recorded under refused on the entry, and the run exits nonzero. It is not retried. The entry's other locations are still fetched and recorded.

An identifier in a scheme no enabled pack declares is passed over, as before packs existed.

What the host will not do. It sends no browser User-Agent and takes no step around a 403. A pack for a publisher that refuses automated clients says blocked = "<why>". Its identifiers are refused with that reason. Record a manual export as a kind: file location instead. The same holds for a publisher whose TLS chain does not verify: no pack can turn verification off.

Archive. wayback:<timestamp>/<url> resolves to the capture's id_ form, the original bytes. catalog-fetch --archive looks up the nearest successful capture of each identifier whose pack says archive = "wayback". It appends that capture to the entry's location: list as a wayback: identifier. Its description carries the capture date, and it lands in the same refresh: commit. It reads the Wayback availability API and never asks the archive to capture anything. It needs an enabled pack declaring the wayback scheme.

Reading a PDF

A quotation of a PDF is compared with a text reading of the PDF. Lint never extracts text itself. Run catalog-extract once, on the machine that holds the PDF:

  1. Run yidam vault pull if the cache lacks the PDF.
  2. Run yidam catalog-extract <entry>.
  3. Run yidam vault push to share the reading with the other machines.

The command records the reading's digest, media type and extractor under the PDF's record, in readings:. A record that has a reading keeps it. To take a new reading, delete that reading and run it again. An entry written before readings: holds its reading under text:, which is still read.

A scanned page has no text layer. The command refuses a reading with no letters or digits. It reports that PDF as skipped.

Reconciling a used-by list

catalog-audit reports the disagreement between an entry's hand-maintained used-by and the nodes that actually cite it. It reconciles none of it. catalog-reconcile is the other half.

It can be mechanical because one of the two lists cannot be wrong. catalog-used-by-drift says so of itself: the citations are authoritative — they cannot drift from the corpus, and a hand-maintained list can. So this is a substitution, not a merge.

An entry declaring no used-by is left alone. Absence is not drift, and the gate stays silent about it. Writing one would be a command deciding an entry should make a claim it never made.

An explicit used-by: [] is a different state, and both the gate and this command read it as one. An empty sequence is something an author typed — it asserts that nothing cites the entry. When nodes do cite it, that assertion is false, so catalog-used-by-drift reports it and catalog-reconcile substitutes the citations for it. An entry genuinely cited by nothing agrees with its own empty list and produces no finding.

Every artifact is named by the SHA-256 of its bytes, in lowercase hex. The cache is machine-wide — $XDG_CACHE_HOME/yidam/vault, or YIDAM_VAULT_CACHE — so two repositories citing the same source store it once. It is deliberately not partitioned by vault. A cache hit answers do I have these bytes, never may I send them.

list, get, push, pull and status read .yidam/config.toml and need a repository. put, path and verify touch only the cache and work anywhere.

[vault.default]
url      = "file:///mnt/archive/yidam"
audience = "Who can read this store, and why that is acceptable."

audience is required and nothing can check it — it is .yidam/publishable's argument applied to a store. What is enforced is that somebody wrote one.

A lone vault may be called anything and, if it declares no holds, takes everything. What a name other than default gives up is the ambient AWS_* fallback below.

Naming vaults, and routing between them

A repository's own index and a licensed PDF it obtained have different readerships, and one store cannot express both. Each vault declares what it holds:

[vault.default]
url      = "s3://corpus-artifacts/yidam"
audience = "Anyone who can read this corpus. Derived output only."
holds    = ["index", "embeddings", "bundle"]

[vault.sources]
url      = "s3://licensed-sources/yidam"
audience = "The sangha. Documents obtained under a licence to read, not to host."
holds    = ["catalog"]

The kinds are catalog, index, embeddings and bundle. Only catalog artifacts exist today. The others are named so a corpus can declare their routes before #417 and #418 start writing them. That avoids reorganising storage on the release that does.

one vault, no holdsit holds everything
two or moreevery one declares holds — a vault claiming nothing would be a catch-all, and routing by default is how a licensed document reaches the public store
a kind claimed by tworefused, naming both
a kind claimed by nonerefused when something of that kind needs a route, naming the kind
a holds entry that is not a kindrefused — a typo claims nothing
a record's own vault:overrides the route its kind would take

A kind nobody claims is refused at the artifact rather than when the config is read. The alternative would make the list of kinds a compatibility surface. Adding one in a later release would turn every multi-vault config red for a kind those corpora have none of.

--vault <name> on push, pull and status is a narrowing flag. It is useful where one store is reachable from a runner and another is not. It never re-routes. An artifact routed to sources is not pushed to default because somebody typed a flag. Moving an artifact between stores is an edit to its record, in a commit. That is how every other assertion the repository makes is done. Stores are opened lazily, one per vault that has work. A vault whose credentials are absent does not block a push to one whose are present.

S3-compatible stores

[vault.default]
url        = "s3://corpus-artifacts/yidam"
region     = "us-east-1"                    # signing scope; MinIO wants one too
endpoint   = "https://s3.example.net"       # omit for AWS
path_style = true                           # defaults true when an endpoint is set
audience   = "Anyone who can read this corpus."

Credentials come from the environment only — .yidam/config.toml is committed and must never carry one:

VariableFor
YIDAM_VAULT_<NAME>_ACCESS_KEY_IDthat vault, always
YIDAM_VAULT_<NAME>_SECRET_ACCESS_KEYthat vault, always
YIDAM_VAULT_<NAME>_SESSION_TOKENtemporary credentials
YIDAM_VAULT_<NAME>_ROLE_ARN / YIDAM_VAULT_<NAME>_WEB_IDENTITY_TOKEN_FILEthat vault, assuming a role by web identity
YIDAM_VAULT_<NAME>_ROLE_SESSION_NAMEoptional, with a role
AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEYthe vault named default, and no other
AWS_ROLE_ARN / AWS_WEB_IDENTITY_TOKEN_FILEthe vault named default, and no other
YIDAM_INDEX_ACCESS_KEY_ID / YIDAM_INDEX_SECRET_ACCESS_KEY / YIDAM_INDEX_SESSION_TOKENthe remote vector index [index.remote] declares
YIDAM_INDEX_ROLE_ARN / YIDAM_INDEX_WEB_IDENTITY_TOKEN_FILE / YIDAM_INDEX_ROLE_SESSION_NAMEthe remote index, assuming a role by web identity

A vault's own variables are tried first: its role, then its keys. Setting both is refused. Then, for default only, the AWS_* keys, then AWS_ROLE_ARN.

The remote index follows the same order under YIDAM_INDEX_. It may fall back to AWS_*, as default does. A corpus declares at most one remote index, so there is no second store to confuse it with.

A role is exchanged with STS's AssumeRoleWithWebIdentity, as IRSA on EKS sets it up. The token file is read on each exchange, and credentials are renewed before they expire. STS is AWS_ENDPOINT_URL_STS when set, else the regional endpoint for AWS_REGION. The file is a path the platform projects, never a key, so it counts as the environment.

The AWS_* asymmetry is deliberate. An ordinary AWS environment is plausibly already configured for the store a repository publishes its own output to. A second vault exists because its readership differs. That is the only reason to declare one. Letting it inherit whatever happens to be exported is the failure the boundary was drawn to prevent.

A single PUT caps at 5 GiB, and multipart upload is not built. Over the cap the upload is refused with a message that says so. It does not fail at the server as EntityTooLarge.

The artifacts this repository computes

.yidam/index/ is built by yidam index-build, and nothing keeps it in git. Building one is minutes of embedding, so the vault carries it from the machine that built it to every other:

yidam vault push --index --commit   # on a machine that has one; commits .yidam/index.lock
yidam vault pull --index            # anywhere else; unpacks into .yidam/index/

--embeddings and --bundle do the same for .yidam/embeddings/ and .yidam/bundle.yiz.

These flags are an either/or, not an addition. vault push alone sends what the catalog names; vault push --index sends the index and nothing else. An index is hundreds of megabytes. --index quietly also uploading a corpus of papers would be a surprise in the direction nobody wants.

An index is a directory and a vault stores one object. So a directory is packed into a single archive and hashed as a whole. A corpus.arrow from one build beside a meta.json from another is a corrupt index that nothing would notice. The archive is deterministic (sorted entries, zeroed mtimes), so pushing an unchanged index does nothing. A .yiz is already one object and is stored verbatim.

.yidam/index.lock

Committed, and the only thing that travels through git:

format_version = 1

[index]
sha256 = "9f2c8e…"
bytes  = 41943040
vault  = "default"

It names the store as well as the hash. A pull reads the store from there, not by re-deriving it from holds. A holds edit made after the push would otherwise send the pull to somewhere the bytes are not. That is a mutable ref wearing a lock file's clothes — the one thing this design exists to avoid.

What push --index refuses

An index is not a file that happens to sit in .yidam/index/. It is a re-encoding of everything walked to build it, and each row carries the node's text verbatim. So the question is not whether the index may leave but whether everything it was derived from may:

ArtifactDerived from
index, embeddings.yidam/corpus, .yidam/catalog
bundlethose, plus .yidam/skills, .yidam/decisions

A path .yidam/private-paths declares private that intersects one of those — in either direction — refuses the push, naming the path. This is the rule sadhana/github/workflows/release.yml already applies to a bundle, for the reason it gives: the artifact outlives the access. A declared directory holding only a README.md or a .gitkeep is intent rather than material and does not refuse.

Unlike a catalog artifact, a repository's own output is pushed by default. There is no redistributable to set, because there is no third party whose licence it could be.

Reclaiming space, and opening a file

yidam vault gc reports cached artifacts that no committed file names. The live set is exactly computable, because every pointer into a vault is a committed file. It deletes nothing until --yes.

Read the list before you pass it. The cache is machine-wide and shared by every yidam repository on this machine. An artifact another one names looks exactly like an orphan from here. Usually deleting one costs a re-fetch. The exception is an artifact recorded vault: none, which is in a cache and nowhere else by decision. For that one the cache is the only copy.

yidam vault materialize hardlinks cached artifacts to .yidam/vault/<entry slug>/<slug>.<ext>. Content addressing is right for storage and useless for opening. Nobody wants to hand a colleague 9f2c8e… with no extension. The extension comes from the record's media_type, and an unlisted type becomes .bin rather than a guess. A hardlink shares the bytes; a copy is the fallback when the cache is on another filesystem.

It refuses to write until .yidam/vault/ is ignored. It asks git check-ignore rather than grepping .gitignore. A repository ignoring it some other way is not reported as broken. A licensed document in a tracked path is the leak push refuses, arriving through git add -A instead.

What push refuses

vault push is the first egress channel yidam itself opens. An artifact must clear two independent checks, and neither implies the other:

  • .yidam/private-paths — about this repository. An artifact whose record sits under a declared path is never uploaded, whatever its licence says. Same rule the release workflow applies to a bundle, for the reason it gives: the artifact outlives the access.
  • redistributable — about the source. A catalog artifact is not pushed unless its record says redistributable: true. A default of "upload unless told otherwise" would make the first push anybody runs a redistribution nobody chose. A catalog is full of papers.

Both checks are policy, not code in this binary. disclose/record covers an artifact the catalog names, disclose/derived one this repository computed. What they say is unchanged. A repository may state its own rule in .yidam/policy/, and yidam policy check names it if it has.

Refusals are grouped by the store they were headed for, each under that store's own audience. The reader learns what they were about to publish to, and with several vaults, which boundary held. --artifact and --vault narrow what is sent and never bypass either check. A digest the corpus does not record is refused, because it carries no redistributable and no path to check.

yidam doctor warns when two vaults resolve to the same credentials. That is legal, since one account can own two buckets. It is also what a half-finished isolation setup looks like. The two are indistinguishable from outside, so it reports the shape and lets the reader decide.

The rules this repository writes about itself

A gate's refusals are rules. A rule compiled into this binary is one the corpus it governs cannot argue with. RFC-0024 makes the rule a committed file instead: git stores the rule, and yidam evaluates it. The rules are Rego, evaluated in-process — there is no daemon, no sidecar, and no network.

CommandWhat it does
policy checkCompile every rule; report each decision and whether it is inherited or this repository's own. Exits nonzero if a rule names a builtin this build does not carry
policy eval --decision <name>Ask one decision about one situation. Reads the input as JSON from --input <file> or stdin; --explain names the rule that fired
policy gate <name>Ask a decision about this repository and exit nonzero if it refuses. The form a workflow calls
policy testRun every test_* rule in every *_test.rego

check, eval and test need no repository. The default policy is compiled in. Somebody working out why a push was refused can ask without a checkout. gate is the exception. It builds its input from the working tree, and there is nothing to answer about without one.

--format json emits the report contract on all four. It did not until #893 — the fields arrived with no envelope around them, against RFC-0024's own stated dependency on RFC-0001. If you read policy test's passed, it is now passing. This contract reserves passed for a gate verdict, which is a boolean.

The decisions

The first family is disclosure — what this repository may let leave.

DecisionAsks
disclose/at_restMay this material sit in this repository, given whether it is public?
disclose/recordMay these bytes be uploaded, given what their catalog record says?
disclose/derivedMay this computed artifact — an index, an embedding set, a bundle — be uploaded, given what it was built from?

disclose/record and disclose/derived are separate because their evidence is. A record-bearing artifact is judged by what its record says. A computed one has no record, so it is judged by what it encodes. Neither consults whether the repository is private, and at_rest does — the artifact outlives the access.

Routing is not a policy decision. Which vault an artifact goes to is vault list's question. Whether it may go at all is this one. The two are kept apart because they fail differently. A route is edited casually by somebody reorganising storage. A licence is not something that edit is allowed to undo.

Overriding a rule

Write .yidam/policy/<name>.rego declaring the same package. That rule then decides — including by being more permissive than the default. RFC-0024 records the argument for and against that.

An override is never silent. policy check names it and the file it came from. policy test runs the inherited cases against your rule, and reports which expectations it no longer meets:

$ yidam policy test
  ok       test_no_overlap_permits
  changed  test_a_private_path_beats_a_licence
  changed  test_silence_is_not_a_licence

10 passed, 0 failed, 5 changed by an override

Those are not failures; a repository is entitled to decide. They are the list to read before concluding the override says what you meant. policy check compares text. Whether a rule is more permissive than the one it replaced is a question about every possible input. Nothing here claims to have answered it. The changed list is the closest thing to an answer.

A repository's own *_test.rego failing is a failure, and it exits nonzero.

An override is also reported by yidam lint as policy-override at Info. It therefore reaches the JSON report and the editor. yidam doctor reports it too, and is where a policy that does not compile is caught, as a fail. Neither gates: the repository decided. What neither permits is deciding quietly.

Export

CommandWhat it does
export *Export the domain model. --format, --out, --list, --force, --prefix, --allow-breaking, --with-corpus
bundle *Alias for export --format bundle, kept for compatibility (no flags)
schema *Emit JSON Schema for the corpus shapes into .yidam/schemas/. A file it did not write is left alone, and the run refuses; --force replaces it. --settings prints the editor yaml.schemas mapping instead

yidam export --list reports each format and its implementation status in your build. That is the reliable answer, because two of the nine are feature-gated:

FormatProducesNeeds
bundle.yidam/bundle.yiz — corpus, ontology, skills, decisions, indexdefault
webFeeds for the browser shell. --webllm-model names the chat panel's modeldefault
graphmlGraphML for graph toolsdefault
llmsA flattened corpus for a context window, on stdout unless --out names a file. --token-budgetdefault
rdfTurtle and JSON-LD. --rdf-format picks oneexport-graph
sqliteSQLite + sqlite-vecexport-sqlite
warehousewarehouse/: the ontology as a BigQuery contract (contract.json and migrations under ddl/bigquery/), plus one load script per table for each feed under .yidam/warehouse/feeds/ (feeds/<feed>/<table>.sql). The first export writes 0001_init.sql. Each later one diffs against the contract.json it left. An additive change writes the next numbered migration, and no change writes none. A breaking change (a removed, renamed or retyped column, a removed value, a tightened or newly required column) is refused; --allow-breaking writes it to ddl/bigquery/breaking/<n>.sql for review. A feed that does not fit the contract is refused, and classes no feed maps are listed. A file edited since the last export is refused; --force overwrites it, except a migration, whose edit it records and keeps. A file it did not write is left alone. --with-corpus adds the feed corpus: each table's rows as feeds/corpus/<table>.ndjson with a BigQuery schema file, its .sql, and a load.sh that stages and merges them. A row's id is the node's reference. A node the contract cannot place is refused, naming it. Without the flag, no instance data is written. See RFC-0049default
textqltextql/: the same contract as Markdown and CSV for a TextQL Ontology folder. It writes a README.md navigation table, warehouse.md (provenance columns, feeds, date precision), one classes/<class>.md per class and one vocab/<class>__<property>.csv per value set. It writes no ANA.md; it prints the row to add there, under --prefix (default: the output folder's name). The ownership rules are warehouse's, and --force works the same waydefault

A directory that is not a corpus is refused, not exported empty. export, bundle and schema check for .yidam/ first. schema --settings is the exception: it prints a compiled-in mapping and reads no corpus.

Serving the domain computer

CommandWhat it does
serve --mcpMCP over stdio — the agent surface. See Connecting an agent
serve --mcp --httpThe same server over HTTP, for a client that takes a URL rather than spawning a process. --bind (loopback by default; an address off loopback with no token prints a startup warning naming every tool it serves), --port, --allow-origin, --token-file (or YIDAM_SERVE_TOKEN) to require a bearer token. --bundle serves a .yiz read-only. /healthz and /readyz need no token
serve --lspLSP over stdio — the editor surface. See Editor setup
recordWhat serve --mcp was asked, read from the record [serve] record keeps. Reads only, and exits 0
record --foldCommits the record's new lines into .yidam/consumption.json as one refresh: commit. Commits nothing when nothing is new

--root <DIR> matters most here, and serve is where it started (#421). A client configures a command line, not a working directory. Without the flag the corpus depends on wherever the client happened to spawn the process. That is what Connecting an agent used to document a sh -c 'cd … && exec …' workaround for. A directory that is not in a corpus is refused, not served empty.

Both transports are in the default build, and so is vector-read. It upgrades MCP's retrieve from keyword to semantic search and adds nothing else. A build without it still serves every other tool. It says degraded on the calls where the difference shows.

serve --mcp writes nothing unless the corpus says to. With [serve] record = true in .yidam/config.toml it appends one line per tools/call to .yidam/record/calls.jsonl. The line names the tool, a digest of its arguments, the row count and the latency. It also names whether retrieval was degraded, and the corpus commit it answered from. Never the query text. .yidam/record/ must be gitignored and the server refuses to start until it is. See Configuration.

yidam record reads that file. It prints the calls per tool, then one sentence per question the record answers. Those are the empty answers, degraded retrieval, act-tier calls, and tools never called. It also names the corpus nodes no recorded retrieve returned. A missing file is reported as nothing was recorded. That is not the same as nothing was asked, and the command says which one it found. It never gates: the file is gitignored, so a fresh clone or CI runner does not have it.

Measuring the corpus

CommandWhat it does
benchThe committed goal set: anchored traversal against flat retrieval. --budget, --scaling

--scaling measures the arms that are functions of N, over generated corpora rather than this repository's own. It needs no index. The flat arm is constant in N, and is excluded by argument rather than by omission.

Deriving and maintaining a repository

CommandWhat it does
init *Write the smallest runnable corpus into the repository you are standing in. --class (repeatable)
clone <target> *Copy the template into a new directory and git init it. Target must not exist
overlay <target> *Add yidam infrastructure to an existing git repo. --backfill, --backfill-ref
backfill *Write a decision record for each epistemic commit in history. --since
tonpa <sub> *Manage bundle dependencies in .yidam/tonpa/. Needs tonpa (a default)

init is the one of these that runs anywhere. clone and overlay copy the template out of the yidam checkout the shell is standing in. In your own project neither of them does anything. That was the whole of #1035.

init reads no template. It writes a class file per --class, one example node in each, and a catalog entry. --class defaults to concept and observation. The tree it leaves passes graph-check and lint as written. Every placeholder says what to replace it with. Each node carries an [open] claim, so the first open-questions answers with something.

Two things it deliberately does not do. It does not run git init. A corpus is a git history with a gate over it. Creating one where you are standing is a larger thing to do unasked. init refuses and names the one line instead. It does not commit either. The first commit's subject is a genesis: naming your domain. export and bundle read it back later, and only you know the domain.

It refuses a tree that already holds a .yidam/, the way clone and overlay do. To add a class to a corpus that exists, write its .ont.yml beside the others. first-corpus-by-hand.md explains the shape init writes.

clone copies everything except the top-level paths NOT_INHERITED names in cmd/clone.rs. Two of them are docs/ and examples/. The documentation here describes yidam itself. An example is a whole foreign corpus that a new repository should not be born holding. The rest is machinery this repository runs against itself. Also excluded: its installer, release and render scripts, its packaging and its .github/. So are its plugin manifest, its editor launch configuration and its demo shell. A derived repository can neither run nor update any of it (#807). overlay refuses a target that already has a .yidam/. It adds yidam/, sadhana/, BOOTSTRAP.md, mise.yidam.toml and the .yidam.toml pin, without touching the repository's own content. It also adds samudaya/, but only when this checkout holds seeds of its own. Each subtree it copies is what git tracks under it. That is the answer clone already gives for the whole template. A file this checkout does not track is not part of yidam. Build output, install prefixes and editor caches therefore have no route into a repository that already holds your work. Both commands need a git checkout of yidam as their source. Neither falls back to walking the directory when it cannot read a tracked set.

backfill's classification is heuristic — it reads leading verbs. It does not extract corpus nodes, and the records it writes are a starting point for a person, not testimony.

tonpa subcommands

SubcommandEffect
addAdd a dependency, fetch it, and update the lock file. ../sibling@<sha> pins a path dependency to a commit
installInstall everything declared in .yidam/tonpa.toml
removeRemove a dependency and delete its installed files
listInstalled packages with node counts and model info
statusPer-dependency: installed / missing / stale
verifyCheck installed bundle hashes against the lock file
updateRe-fetch one or all dependencies to the latest bundle. A pinned path moves to the sibling's HEAD

Sharing a derivation covers publishing a .yiz and consuming one, and what a cross-corpus citation is and is not.