Architecture

The how behind the PRD: the pipeline, projections, changed-source reconciliation, module decomposition, and the directional design for matching and resolution. The vocabulary this doc uses (Source, Event, Claim, Entity, EntityLink) and the invariants it cites by number are defined once, in domain.md. The interface contracts modules build against live in interfaces.md.

Created 2026-09-01. Updated 2026-09-03 (concepts and invariants moved to domain.md, D28; v0 review: kind, groundedness, relation reconciliation, D22-D27; previously changed-source reconciliation D11/I4; module map vs build order, D15). Source of truth for visuals: the "Helmi Context Service" Excalidraw board (id a8d17080-29dd-47c2-98d7-b1fcaee9c0ab; the tunnel hostname rotates, the id does not). Snapshot: helmi-context-service.excalidraw.

Pipeline

sources -> events -> summarize (source + event) -> parse/chunk -> extract partial entities
        -> normalize hard claims
        -> [later] match -> entity_resolution_list -> apply (merge / evidence / create)
        -> [later] promote (evidence -> member, unresolved -> canonical), resolution judge

Summarization and extraction are the only steps that involve an LLM; both are versioned and re-runnable. Ingestion, composition, parsing, and normalization are deterministic and testable without a model.

Extraction output. Persisted as an immutable extraction_run keyed by (event revision, ontology version, prompt version, model); re-running adds a run and never overwrites one (invariant 1). Projecting a run into partial_entities and claims domain rows, canonicalized values included, happens in the same transaction as the run row, so a run can never half-land.

Projections

Two pieces of derived data are versioned and rebuildable rather than re-derived upstream. They are deterministic computations over stored input, not pipeline steps.

Projection	Derived from	Versioned by	Computed
renderings + chunks	source payload	parser_version	during ingestion
claim_values	raw claim values	normalizer_version	when extraction output is persisted

Three rules make rebuilding safe:

1. The raw value is always stored beside the derived one: the source payload, and the claim value as the LLM emitted it. A rebuild reads raw and never re-pays an LLM stage.
2. A projection's version never enters an LLM stage's run key. Bumping normalizer_version rebuilds claim_values in seconds; it must never trigger re-extraction. This is what keeps normalizer iteration free.
3. The rebuild command exists from day one. Parser and normalizer rules are exactly the things that will iterate against real data.

Canonicalization is not search-form expansion. Two things hide under "normalization" and only the first is in v1. Canonicalization stores one clean value per claim (email lowercased and de-tagged, registrable domain via the public suffix list, ISO codes, alias surface form): a handful of per-type validators feeding claim_values. Search-form expansion (phonetic keys, trigrams, blocking keys) exists only to generate match candidates, so it belongs to matching, where the claim-type registry already declares a search strategy per type (interfaces.md). It gets designed at PRD milestone 5, against measured data.

Changed-source reconciliation

The mechanics owed by D11 for PRD requirement I4. Driving case: calendar events mutate constantly (reschedules, attendee changes); corrected emails and edited documents follow the same path.

Source versions. A changed payload with the same external_id/external_system mints a new source row (I1) that supersedes the prior one, per the Source concept above. Nothing is deleted (invariant 1); the version chain is walkable through superseded_by.

Recomposition. Composition keys on external identity, not source id, so the new version resolves to the SAME event. The event keeps its id; its active source set swaps the superseded version for the new one and revision increments. The prior membership row is kept, superseded, for lineage.

Re-extraction. A revision bump forces re-extraction of the event under the current extractor_version. Extraction output is keyed by (event id, revision, extractor_version), so no run overwrites another (invariant 1).

Diff-based reconciliation. The new extractor output is compared against the event's active partial entities from the prior revision. Partials are aligned across runs by their identifying hard claims; within an aligned pair, claims are matched by (type, normalized value). Three outcomes:

• Unchanged: the prior claim row is kept active and the duplicate is not stored. Resolution state (links) is untouched, so nothing re-matches and no duplicate members appear.
• Changed or absent: the prior claim is superseded (superseded_reason: source_changed). Links whose driving_claim_id points at a superseded claim are superseded through the applier (invariants 6, 7).
• New: stored and sent down the normal matching path.

A prior partial with no aligned counterpart is fully stale: all its claims and links are superseded. A new partial with no aligned prior is genuinely new and matches normally.

Relation claims carry no value, so they align by (type, aligned-target partial, attributes) instead (D24). When a target partial is superseded and an aligned successor exists, a relation claim on an unchanged subject partial is re-issued against the successor and the prior claim superseded (source_changed), in the same reconciliation transaction; a relation whose target is fully stale is superseded with it. At milestone 3 links do not exist yet, so I4 end to end means claim reconciliation; link supersession activates with milestone 5.

Relation to extractor_version re-runs (E1). Two orthogonal axes. A version bump on unchanged content is additive: outputs coexist keyed by version, nothing is superseded. A revision bump under the same version is reconciling: stale output is superseded. A revision bump never backfills old extractor versions; only the current version re-runs.

Critical mass. Claims superseded by reconciliation stop counting toward critical mass and promotion the moment their status flips (D11). There is no recount job: critical-mass checks read active claims only.

Module decomposition

Five modules, each owning everything behind its contract. Modules are ownership and contract boundaries, not a staffing plan and not a build order: v1 is built by two engineers, and the order the modules are built in is the milestone sequence in the PRD (D15). The contracts themselves, their field lists, ownership, and change rules are in interfaces.md.

#	Module	Owns	Produces
1	Ingestion	Source adapters per external system, sanitize/chunk, content_hash dedup, source versioning and supersession, source composition and recomposition (event revision), summary generation	Event, summaries
2	Extraction	LLM prompts, ontology use, hard/soft split, invariant 2, the reconciliation diff on re-extraction	[PartialEntity]
3	Matching	Per-claim-type search adapters, tier routing, candidate generation	entity_resolution_list
4	Entity store + applier	The graph (entities, links), merge/evidence/create application, promotion, provenance, the search API	mutations, query surface
5	Bootstrap	Importing canonical entities from authoritative systems (the Platform, LinkedIn), organizations first (D26): canonical org endpoints are what activate latent relation claims (invariant 9)	canonical entities

Matching and resolution (directional design)

Settled in design, to be validated and parameterized against the measured extraction numbers (PRD requirement M1). Modules 3 and the applier side of 4 are deliberately built last.

Matching: per-claim-type adapters exhaust exact queries (email: exact, same-address normalization such as user+tag@, domain). An exact hit routes by the tier of the matching claim: identifying claim, merge (create a member link); non-identifying claim, attach as evidence. Contextual signals (nationality, language) never route a match, they only narrow candidates. No hit anywhere creates a new unresolved entity.

Time-aware consistency: matches confirm, but mismatches disconfirm asymmetrically. A conflicting constant claim is negative evidence, never an automatic split: extraction noise exists. A conflicting temporal claim disconfirms only when the claim type is exclusive and the time anchors overlap (one person cannot be in two places at the same time, but can be in two places on consecutive days). Accumulative temporal claims never disconfirm. For relation claims the anchor in these rules is the validity interval from the attributes when stated, with reported_at as the fallback (D19); two employments for one entity are consistent when the time slices differ.

Relation claims in matching: contextual, non-identifying signals, exactly like nationality. They narrow candidates and never route a match (D20). The critical-mass "mutually consistent" check and the resolution judge both apply these rules, and negative evidence is recorded like positive evidence, weighing against promotion.

Promotion, edge level (evidence to member): when accumulated evidence on one candidate reaches critical mass, the evidence link is superseded by a member link and sibling evidence links on losing candidates are superseded as "resolved elsewhere", in one transaction.

Promotion, entity level (unresolved to canonical): legal only when every applicable search returns a clear no-match, and either the entity holds an identifying claim from a trusted source, or evidence reaches critical mass (varied, mutually consistent partials accumulated over time).

The resolution judge (decided 2026-09-01, detail in R1 Context Embedding Report): when a new evidence link lands on an entity, an LLM judge re-evaluates that entity's evidence relationships. It compares the entity's member claims and their situating context (source and event summaries) against each evidence partial's claims and context, and returns one verdict per evidence link: promote (run the standard promotion transaction), supersede (judged_unrelated), or inconclusive (leave active). The judge appears in the resolution phase only, is versioned and provenance-recorded like any other stage, and runs in shadow mode until it clears an eval dataset. The summaries it consumes enter v1 ingestion scope (decided 2026-09-01), so the corpus is context-rich before the judge exists.

Open questions, answerable only with data:

1. The critical-mass model (a statistical definition, not a hand-tuned count).
2. Which claim types and sources count as "trusted" for promotion. Needs a source-trust registry, probably a property of Source.
3. Match confidence bands, if exact-only routing proves too strict in practice.

Stack

Python 3.12+, FastAPI, pydantic v2, Postgres, SQLAlchemy 2 + Alembic. Python because the entity-resolution ecosystem (splink, dedupe, recordlinkage, rapidfuzz, jellyfish) has no counterpart elsewhere, and because the service is mostly schema, which pydantic models once and serves to the database, the API, and the extractor alike. LLM calls go through OpenRouter. Search runs inside Postgres (citext, pg_trgm; pgvector when embedding search lands): the index is a projection of member claims and must update transactionally with link writes, which rules out an external search engine for v1.

References

• PRD: problem, goals, requirements, milestones, decisions log.
• domain.md: concepts and the ten invariants.
• scenarios/: worked inputs and the outcomes this design must produce.
• interfaces.md: the contracts between modules.
• conventions.md: the code conventions, tach-enforced where possible.
• relationships proposal: history of the relationship design (D16-D21), frozen; current mechanics are in this doc and interfaces.md.
• Excalidraw board a8d17080-29dd-47c2-98d7-b1fcaee9c0ab: full schemas, flow diagrams, module map, edge-type legend.