Claims and normalization¶

From the graph data model: the claims table, its normalized form, the search read, and how claims are written and reconciled.


Status	Proposed, in review with Nacho. Assumes claims are values only, on entities only: relationships move to a model of their own, pending the proposal tracked in #18, and this document is adapted when it lands. Settled: the origin set and derived claims; normalization as two columns on the claim row; supersession as the two transaction pointers (2026-09-07). Deferred by Nacho: facets, search keys beyond exact, the normalization-failure policy. Items marked recommended are pending his answer
Created	2026-09-03
Updated	2026-09-08
Context	domain.md: Claim and its classification, invariants 1, 2, 3. architecture.md: Projections, Changed-source reconciliation. interfaces.md: B2, B4, the claim-type registry. D11, D22. Issues #8, #17, #18

Summary¶

One table holds every claim on a partial entity: a hard claim with a normalizable value, or a soft claim with free text. The raw value is stored as emitted and never rewritten. Beside it sit normalized_value and normalizer_version: the canonical form the store computed at write time, rewritten only by the rebuild command when a normalizer changes. Search normalizes its input with the same code and matches on normalized_value, so the two sides never drift. A claim leaves the live graph by supersession, never deletion, and re-extraction reconciles claim by claim: unchanged claims keep their row, stale ones are superseded, new ones are stored.

erDiagram
  partial_entities ||--|{ claims : "carries"
  claims |o--o| claims : "derived from"

  claims {
    uuid id PK "from B2, minted per run"
    uuid partial_entity_id FK "the partial the claim is about"
    text claim_type "registry-validated, no check"
    text value "raw, never rewritten"
    text normalized_value "nullable; rewritten by rebuild only"
    text normalizer_version "nullable; version that produced it"
    text snippet "nullable; required when extracted"
    text origin "extracted | structural | derived | manual"
    uuid derived_from_claim_id FK "nullable; iff origin = derived"
    uuid created_by_tx_id FK "the transaction that made it true"
    uuid superseded_by_tx_id FK "nullable; the transaction that made it false"
  }

organization_id and created_at come from OrgScopedBase and are omitted.

claims¶

Column	Meaning
`id`	The id extraction emits in B2, derived per run over the run key, the partial's ordinal, and the claim's ordinal. A claim kept through reconciliation keeps its first id; the new run's id for it is discarded
`partial_entity_id`	The partial the claim is about. Composite foreign key with `organization_id`
`claim_type`	A key from the registry, a versioned package, not a table (interfaces.md, the claim-type registry). No check constraint: adding a type is a registry bump. Whether extraction may suggest types the registry lacks is #17. The functions layer also checks the type is bound to the partial's `entity_type` (D22); whether a type may bind to several kinds is #19
`value`	The raw value as the producer emitted it. Not null. Never rewritten
`normalized_value`, `normalizer_version`	The canonical form of a hard claim (email lowercased and de-tagged, registrable domain, ISO code, alias surface form) and the registry version of the normalizer that produced it. Written at insert; rewritten only by the rebuild command. Null for soft claims, and for a hard claim whose value failed normalization
`snippet`	The source text the claim rests on. Nullable at the table; the functions layer requires it when `origin = 'extracted'`, because imports and manual entries have no text to quote
`origin`	Who produced the claim (below)
`derived_from_claim_id`	The claim this one was computed from; set iff `origin = 'derived'`. Superseding the parent supersedes the child in the same transaction
`created_by_tx_id`, `superseded_by_tx_id`	The two transaction pointers (transactions). The first is the transaction that stored or reconciled the partial; the second is the reconciliation or review that retired the claim, null while live. A changed claim's successor is the claim the same transaction inserted on the same partial with the same type; an absent claim has none

Constraints. UNIQUE (organization_id, id); composite foreign keys for partial_entity_id, derived_from_claim_id, and the two transaction pointers; CHECK (origin IN (...)); CHECK ((origin = 'derived') = (derived_from_claim_id IS NOT NULL)).

Mutable columns. The graph model otherwise rewrites nothing but a row's second transaction pointer, once, and entity status. normalized_value and normalizer_version are the one further exception, and the rebuild command is their only writer. The raw value is what invariant 1 protects; the canonical form is recomputable from it at any version, because version modules are never edited once a successor exists (conventions.md, rule 7).

When relationships arrive. Relationships will carry claims of their own (an investment's round and size), so a second claims table pointing at relationships is expected rather than a polymorphic target on this one (#18). This table may then be renamed to say it holds entity claims. Nothing here is shaped against that.

Origin¶

A closed set, checked at the database. Settled with Nacho, 2026-09-03.

`origin`	Produced by	`snippet`	Example
`extracted`	The LLM extraction pass, module 2	required	"John's email is john@acme.com" becomes an email claim
`structural`	A deterministic field read, no LLM	optional	A calendar attendee's address; a Platform contact's email field
`derived`	Extraction persistence, from another claim in the same output, with `derived_from_claim_id`	none	`domain(acme.com)` from `email(john@acme.com)`, issue #8
`manual`	A person, through the review API	none	A LinkedIn URL added to a profile

structural and extracted are distinct because promotion's "identifying claim from a trusted source" (architecture.md, Matching and resolution) must tell a parsed field from a model guess. Folding them loses that.

Supersession at claim level¶

Re-extraction of a changed event keeps unchanged claims, supersedes changed and absent ones under the reconcile transaction, and stores new ones (architecture.md, Changed-source reconciliation; D11). That is why claims carries the two transaction pointers and why a partial keeps its id across revisions (entities and links). Whole-partial replacement was decided against there: it would re-match every claim and mint duplicate unresolved entities. The reason a claim was retired is the retiring transaction: a system actor with new claims under the same transaction means the source changed, a human actor means a person retracted it, and the note says why.

Recommended. When a partial is superseded, its live claims are superseded explicitly under the same transaction. Deriving a claim's liveness from its partial's pointer instead would put a join in every read and could not be expressed in the partial indexes below.

Normalization¶

The store owns the normalizer call, at write and at query, so normalized_value and the search input always come from the same code. Extraction therefore stops emitting normalized_value in B2, a change under interfaces.md rule 2 recorded there on ratification. Keeping the normalizer in extraction would tie its version to the extraction run key, which projection rule 2 forbids (architecture.md, Projections): a normalizer bump must never trigger re-extraction.

Rebuild. A bump is per claim type. The rebuild command runs one UPDATE per type, reading value and writing normalized_value and normalizer_version for the active claims still stamped with an older version. No LLM stage is re-paid. Search matches on whatever form is stored and never filters by version: during a rebuild some rows carry the old form and some the new, which is no worse than the state before the rebuild began. If search filtered by version, the rows not yet rebuilt would return false no-matches, and a no-match is promotion-legal (invariant 5). The version column exists to find the rows that still need rebuilding.

Deferred by Nacho, 2026-09-03. Facets, several canonical forms per claim such as an email's domain or a location's country, are not in this design. Search keys beyond exact (trigram, phonetic, blocking) belong to matching at milestone 5 and are not in this design either; the extension path is in Alternatives. The policy for a value that fails normalization is also deferred: for now the claim is stored with normalized_value null, it is invisible to search, and the store does not reject the partial.

Indexes and query shapes¶

-- the B4 search hop: one range scan over one type, live rows only
CREATE INDEX ix_claims_search
  ON claims (organization_id, claim_type, normalized_value)
  INCLUDE (partial_entity_id)
  WHERE superseded_by_tx_id IS NULL AND normalized_value IS NOT NULL;

-- a partial's live claims: reconciliation, the entity view, invariant 2 checks
CREATE INDEX ix_claims_live_by_partial
  ON claims (organization_id, partial_entity_id) INCLUDE (claim_type)
  WHERE superseded_by_tx_id IS NULL;

-- children of a claim: the derived cascade
CREATE INDEX ix_claims_derived_from
  ON claims (organization_id, derived_from_claim_id)
  WHERE derived_from_claim_id IS NOT NULL;

-- rows an in-progress rebuild still has to visit
CREATE INDEX ix_claims_normalizer_version
  ON claims (organization_id, claim_type, normalizer_version)
  WHERE superseded_by_tx_id IS NULL;

-- audit, as on every graph table
CREATE INDEX ix_claims_created_by ON claims (organization_id, created_by_tx_id);
CREATE INDEX ix_claims_superseded_by ON claims (organization_id, superseded_by_tx_id)
  WHERE superseded_by_tx_id IS NOT NULL;

Search (B4). The store normalizes the input with the current normalizer, then:

SELECT e.id AS entity_id, e.status AS entity_status, c.id AS matched_claim_id
FROM claims c
JOIN entity_links l ON l.organization_id = c.organization_id
                   AND l.partial_entity_id = c.partial_entity_id
                   AND l.kind = 'member' AND l.superseded_by_tx_id IS NULL
JOIN entities e     ON e.organization_id = l.organization_id AND e.id = l.entity_id
WHERE c.organization_id = $org AND c.claim_type = $type
  AND c.normalized_value = $normalized AND c.superseded_by_tx_id IS NULL;

Three index hops, each over live rows only. Evidence links never enter it, which is invariant 3 in one predicate. No rows is no_match; a failed statement is error (invariant 5).

Writing claims¶

Two functions in the functions layer, with different failure semantics.

put_partial(partial, claims, run): the first write of a partial id. Inserts the partial and its claims, normalized, in one savepoint, idempotent by id. The same id with different content is an error. Bootstrap and manual creation use it too.
reconcile_claims(partial_id, claims, run): the partial exists and is live; the input is its full claim set from the new run. Claims align by (claim_type, normalized_value), falling back to the raw value when normalization failed; snippet differences do not count. Unchanged claims keep their row, changed and absent ones get the reconciling transaction as their second pointer, new ones are inserted under it. It returns the kept, superseded, and added ids, so the applier can supersede links driven by the superseded claims and route the added ones to matching. An unknown partial id is an error, never a create.

Recommended. Extraction aligns partials across runs; the store diffs the claim set inside an aligned pair. If extraction sent supersede lists instead, the store could not verify a supersession is legitimate, and a future manual edit path would reimplement the diff. Module 2's "the reconciliation diff" in the architecture's module table narrows to partial alignment at ratification.

Open items¶

The explicit claim cascade when a partial is superseded (Supersession).
The alignment line between extraction and reconcile_claims (Writing claims).
Manual entries and bootstrap records as sources, so every partial has an event and a time anchor, versus anchoring manual claims on created_at with the transaction log as their only provenance. Affects event_id nullability in entities and links and contract B5.
ontology_version (#12).

Alternatives considered¶

Relation claims in this table, as target_partial_id plus per-type attributes, with a check that exactly one of value and target is set. Written under D16; withdrawn 2026-09-03 when the devs agreed relationships need a model of their own (#18). The traversal query for invariant 9 goes with them.
A separate claim_values projection table, append-only, one row per claim and normalizer version. Dropped by Nacho on 2026-09-03: old canonical forms are recomputable from the raw value with the old version module, so keeping them buys only a side-by-side comparison that a rerun gives as well. The projection is still versioned and rebuildable from raw, as the architecture requires; it lives on the claim row.
Extraction normalizes. Rejected: the query side would normalize with a possibly different version; see Normalization.
A table per claim type. Rejected for the reason a table per entity type was (entities and links).
Search keys beyond exact, when matching needs them at milestone 5: a claim_search_keys (claim_id, strategy, strategy_version, key) table with one partial index per strategy, rows only for types whose registry entry declares the strategy. Adding a type is then no DDL; adding a strategy is one index. Deferred, not rejected.
Whole-partial supersession on re-extraction. Rejected in the architecture and in entities and links.