Graph data model¶

The Postgres tables that hold the graph side of Indra, and the rules that keep them honest.


Status	Proposed. Scope, conventions, entities, and links are settled with Nacho (2026-09-03); supersession was reduced to two transaction pointers on every graph row on 2026-09-07 (Nacho, after a two-sided comparison with a transaction log as source of truth; Transactions, Alternatives); claims and transactions are in review; the functions layer is unwritten; the ingestion tables are a stub until their owner is assigned. Relationships are leaving the claim model for one of their own (#18, 2026-09-03); this design assumes claims are values on entities until that proposal lands. Decisions to log on ratification are listed at the end; open questions: #8, #12, #17; #10 closes with the reduction
Created	2026-09-03
Updated	2026-09-08
Context	PRD gate 5 (store and applier first, D15); domain.md (concepts, the ten invariants); architecture.md (projections, changed-source reconciliation, module decomposition); interfaces.md (B2, B4, B5, the claim-type registry with the relation attribute schemas); conventions.md (the code conventions the tables inherit). History only: relationships proposal (D16 to D21, being revisited in #18). Diagrams: Excalidraw board `a8d17080-29dd-47c2-98d7-b1fcaee9c0ab`; the ERDs here are inline Mermaid so a schema change shows in a review diff, and the board gets the snapshot at milestone close

Summary¶

Five tables hold the graph: entities, partial_entities, claims, entity_links, and transactions. Every graph row carries two pointers, the transaction that made it true (created_by_tx_id) and the transaction that made it false (superseded_by_tx_id, null while the row is live). Those two columns are the whole supersession model and the whole audit record. Plain Postgres, no graph database and no storage abstraction (per Nacho, 2026-09-02). Every row is org-scoped. Nothing is deleted: a row leaves the live graph when its second pointer is filled, and the one mutable column, entity status, changes only through a logged transaction. Type is data: entity_type and claim_type are validated against the shared registry, so a new entity type or claim type is a registry version bump, not a migration. A thin functions module enforces the invariants in one place; the database enforces the ones a constraint can express.

Scope¶

In. The five tables, their constraints and indexes, and the functions that write and read them. This is the storage half of module 4 (architecture, Module decomposition). It is designed ahead of PRD gate 5 on purpose: D15 defers the resolution block behind the M1 report because matching's parameters come from M1, and nothing in this data model is a parameter. Building the store first is the block's own internal order.

Out of the graph tables. Sources, events, event memberships, and summaries are ingestion's tables and follow the concepts in domain.md. The conventions below bind them, and they get their own document (ingestion tables, a stub until ingestion's owner writes it), but the graph design does not depend on their schema: partial entities reference an event and a stage run by id only, with no foreign key. Also out: the applier (routing the three B3 outcomes, promotion, corrections), the bootstrap importer, the review API, and the registry package. The schema leaves room for all of them, and the bootstrap and manual cases shape it directly: a partial entity created by an import or by a person has no event and no stage run, so those references are nullable and its claims carry origin = manual or structural.

The model at a glance¶

erDiagram
  entities ||--o{ entity_links : "target of"
  partial_entities ||--o{ entity_links : "source of"
  partial_entities ||--|{ claims : "carries"
  transactions ||--o{ entity_links : "created_by / superseded_by"

  entities {
    uuid id PK
    text entity_type
    text status "canonical | unresolved; mutable"
    uuid created_by_tx_id FK
    uuid status_tx_id FK "the transaction that last set status"
  }
  partial_entities {
    uuid id PK
    uuid event_id "nullable, no FK"
    uuid stage_run_id "nullable, no FK"
    text entity_type
    uuid created_by_tx_id FK
    uuid superseded_by_tx_id FK "nullable"
  }
  claims {
    uuid id PK
    uuid partial_entity_id FK
    text claim_type
    text value "raw, never rewritten"
    text normalized_value "nullable; rebuild-only rewrite"
    text normalizer_version
    text origin
    uuid created_by_tx_id FK
    uuid superseded_by_tx_id FK "nullable"
  }
  entity_links {
    uuid id PK
    uuid entity_id FK
    uuid partial_entity_id FK
    text kind "member | evidence | excluded"
    uuid created_by_tx_id FK
    uuid superseded_by_tx_id FK "nullable"
  }
  transactions {
    uuid id PK
    bigint seq "identity; the order, per org"
    text kind
    text actor_kind "system | human"
    uuid actor_id "no FK"
    text note "nullable; a person's reason"
  }

Every table also carries organization_id and created_at from OrgScopedBase (indra/models/base.py), omitted above. The two transaction pointers are drawn on every graph table because they are the model; only the entity_links edge to transactions is drawn, the other three are identical. Column-level detail is in the documents listed below.

Conventions¶

Names. Plural snake_case tables, the repo's existing choice (jobs, stage_runs). _id for every reference column, _at for timestamps. kind names a closed set the code switches on (entity_links.kind; transactions.kind is the one exception, an unchecked label for people, Transactions); *_type names a registry-validated vocabulary (entity_type, claim_type). Checked 2026-09-03 against the PostgreSQL keyword list: no planned name is reserved.
Concept to column mapping. domain.md names concepts; this doc names columns and carries the mapping where they differ. On the graph tables, domain.md's superseded_by is the column superseded_by_tx_id; its status: active | superseded is the predicate superseded_by_tx_id IS NULL, not a column; its superseded_reason is the superseding transaction: its actor, the rows it wrote, and its note (a person retiring a member link with a note is the wrong-merge reason). Ingestion's sources have no transactions and keep superseded_by_id until their owner writes the ingestion tables. A partial entity's kind (D22) is the column entity_type: the naming rule reserves kind for closed sets the code switches on, and entity kinds are a registry vocabulary.
Constraint names. Alembic's recommended naming_convention on Base.metadata (Alembic, The Importance of Naming Constraints), so every index and constraint has a deterministic name a later migration can drop.
Supersession is two pointers. Every graph row carries created_by_tx_id, the transaction that made it true, and superseded_by_tx_id, the transaction that made it false, null while the row is live. Live rows are WHERE superseded_by_tx_id IS NULL; the graph as of transaction T is rows whose creating transaction has transactions.seq at or below T's and whose superseding transaction, if any, has seq above it (timestamps do not order transactions, seq does; Transactions); a row's history is its two transactions. To change your mind you fill the second pointer on the old row and insert a new row, both under the same transaction; a row is never made live twice. superseded_at is the superseding transaction's created_at, the reason is the transaction itself, and the successor row is the one the same transaction inserted, so none of those are columns. Reduced from a five-column pattern on 2026-09-07 (Transactions, Alternatives).
Closed vocabularies are text plus a check constraint, never native enums (conventions.md, rule 6). Registry vocabularies (entity_type, claim_type) get no check constraint at all: the functions layer validates them against the registry, so adding a type is not DDL.
Ids are caller-derived (uuid5 over stable inputs), never generated by the database. A replay after a crash reproduces the same ids and every write is an idempotent no-op; a version bump changes the inputs and mints new ids. The same id with different content is an error, never an overwrite.
Org scoping is structural. Cross-table references are composite foreign keys on (organization_id, id), so a link across organizations fails at the database, and the functions layer fixes organization_id once per call, so no query can omit it.

Which layer enforces each invariant¶

Invariant (domain.md)	Database	Functions layer	Caller
1 Never destroyed	`BEFORE DELETE` trigger on all five tables; `BEFORE UPDATE` rejects everything except filling a null `superseded_by_tx_id` once, `entities.status` with `status_tx_id`, and the rebuild-only normalization columns on claims (Claims)	no delete function exists; every write happens inside a transaction
2 At least one hard claim per partial		rejects a partial with no hard claim by registry type; whether a claim that fails normalization counts is deferred (Claims)	extractor enforces first
3 Identity from active members only		the search and entity-view queries join live member links only (`superseded_by_tx_id IS NULL`), through partial indexes that hold only live rows; `excluded` links never enter either
4 One active member per partial	partial unique index	maps the unique violation to an error
5 No-match is not an error		search returns a tagged `hits \| no_match \| error` value	promotion legality
6 Every merge records what drove it	`created_by_tx_id` on every row and its index; the by-claim record follows #21	the transaction's `actor_id` is the matching run whose stored output holds the full resolution item (Transactions)	the pipeline retains `stage_runs.output` as long as the graph
7 Sequential per organization	jobs index (`0001_initial.py`); per-org advisory lock inside every write transaction
8 Multi-tenant	composite `(organization_id, id)` foreign keys	`organization_id` fixed per call
9 Relationship usability derived			pending the relationship model (#18): relations leave the claims table, and this row is rewritten with that model
10 New evidence never re-ingests the source			ingestion's storage of the source payload; every graph row derives from stored events and claims and adds nothing to it. Whether bytes are copied or referenced is open (#7)

Documents¶

Covers	File	Status
Scope and conventions	this document	settled 2026-09-03
Entities, partial entities, entity links; indexes and query shapes	entities-and-links.md	settled 2026-09-03; link justification deferred to #21; `ontology_version` pending
Claims on entities, normalization, reconciliation	claims-and-normalization.md	in review; relationships out per #18
Transactions; the two pointers on every row; corrections as new transactions	transactions.md	in review
The functions layer: write transactions, reads, rules, search outcome	data-model/functions.md	unwritten
Ingestion tables: sources, events, event memberships, summaries	ingestion.md	stub; owned by the B1 producer, assigned at milestone 1 (D15)

Decisions this design logs on ratification¶

Written into decisions.md when Nacho ratifies the corresponding document (next free entry D29 at the time of writing); until then this list is the proposal.

Graph state is plain Postgres tables behind a functions module. No graph database, no backend abstraction; a port with adapters was designed, reviewed, and dropped on 2026-09-02 as insurance nobody had asked for. (Nacho, 2026-09-02.)
Type is registry data. One entities table with entity_type, not a table per type; adding or reshaping the ontology is a registry bump plus, for reshapes, a logged re-typing through re-extraction. (Pending.)
entity_links stays a first-class table, reaffirming D2: evidence to several candidates, supersession history, and per-link provenance do not fit a column. (Nacho, 2026-09-03.)
The store owns claim normalization at write and query time, as normalized_value and normalizer_version on the claim row, rewritten only by the rebuild command; extraction stops emitting normalized_value in B2. (Nacho, 2026-09-03, on the two columns; the claims document pending as a whole; a B2 change under interfaces.md rule 2.)
History lives in the graph rows as two transaction pointers, created_by_tx_id and superseded_by_tx_id, and nowhere else: no status, timestamp, reason, or successor columns, no effects table, no status-history table. The tables stay the source of truth; transactions is metadata. Chosen over a transaction log as source of truth with mutable projection tables, which would have superseded D1. (Nacho, 2026-09-07, "for now"; Transactions, Alternatives.)
A transaction stores no effect and no inputs: what it did is the rows that point at it, why is the stage run or job it names as actor, and a person's reason is a note. kind is an unchecked label for people. (Nacho, 2026-09-07.)
There is no undo primitive. A correction is a new transaction that retires and creates rows like any other, and the two pointers record both the mistake and the fix. This rewords D12 and PRD R3 ("one-step undo") without changing their acceptance criterion, and O3's undo rate becomes a correction rate. (Nacho, 2026-09-07.)
entity_links.kind gains excluded: the live negative judgement "this partial is not this entity", written by corrections and judge verdicts, honoured by matching's candidate search and enforced by the applier. Adds a kind to domain.md's EntityLink and a note to B3/B4 in interfaces.md. (Nacho, 2026-09-07.)
entity_links carries no claim column. driving_claim_id is removed: matching is expected to score all of a partial's claims, so one driving claim misdescribes the decision, and how a link records its claims is #21. Removes the field from domain.md's EntityLink, PRD S1 and S6, contracts B3 and B5, and the architecture's reconciliation paragraph at ratification. (Nacho, 2026-09-08.)
The table is transactions, not actions, and the pointers are *_tx_id: the word names the atomicity and the as-of semantics the table exists for, following Datomic. Not a database transaction; Transactions disambiguates. D12's superseding entry and PRD R2, R3, O3 adopt the word at ratification. (Nacho, 2026-09-08.)

Alternatives considered¶

A backend-agnostic port with Postgres, in-memory, and later Neo4j adapters. Designed and reviewed 2026-09-01. Dropped: two adapters and a conformance suite bought insurance against a backend change nobody had asked for, and no maintained library offers such a port over Postgres (graphiti's driver is a Cypher transport with no Postgres backend; Apache AGE is unavailable on RDS, Aurora, Neon, and Supabase).
A table per entity type. Rejected: entities have no type-specific columns (an entity is a view over its links, D1), so the tables would be identical, and entity_links.entity_id would need a polymorphic target with no real foreign key.
A generic graph API (add_node, add_edge). Rejected: it pushes the invariants into every caller.
A member entity_id column on partial_entities instead of entity_links. Rejected; see entities and links, Why a table and not a column.
The transaction log as source of truth, graph tables as mutable projections. Weighed against the two-pointer model on 2026-09-07 with one advocate brief per side; the comparison is in Transactions, Alternatives. Not chosen for now: it stores every change twice (JSON in the log, rows in the tables) with nothing in the database keeping them equal, needs a replay engine and upcasters, and reopens D1 and invariant 1. Nacho keeps it as the fallback if the two-pointer model turns out hard to build on.

Open questions¶

GitHub issues labeled open-question, per the writing-docs skill; the thread is the record and this list only points.

Generated status columns, #10: closed by the 2026-09-07 reduction, there is no status column to generate; liveness is superseded_by_tx_id IS NULL. Issue to be closed with a pointer here.
ontology_version on partial_entities, #12: a stamp so a future ontology reshape can find the rows typed under the old version. Recommended in entities and links.
Where derived domain claims are created, #8: extraction persistence (recommended, origin = derived) or the store.
Dynamic or AI-suggested claim types, #17: whether extraction may emit types the registry does not know, and how a suggestion becomes a registered type. Raised by Nacho 2026-09-03 as an idea, not a question anyone waits on.
Relationships as a separate model, #18: the storage shape for relationships and their own claims, owned by Agustin Benvenuto; extraction passes by Agustin Galarza. Supersedes D16 when its proposal is ratified. This design proceeds with claims as values on entities.
Claim types bound to one entity kind, or shared across kinds, #19: whether name is one type with per-kind bindings or two types, and how a normalizer bump is scoped by kind. Raised by Nacho 2026-09-03; decides registry v0's binding table.
How an entity link records the claims that justified it, #21: one driving claim, a link_claims junction, or an array; decided with matching at milestone 5. Interim: created_by_tx_id on every row and no claim column on the link; the matching run's stored output holds the claims. Rewords invariant 6 when decided.