Skip to content

Indra: project skeleton and working conventions (proposal)

Status: proposal, 2026-09-01. Posted to #product-dev for agreement before we scaffold.

Frozen 2026-09-03 (D28). Decision 1 was taken as recorded below; the skeleton on main was scaffolded from sections 4 to 6 with deviations (for example indra/contracts/ and indra/llm/ sit at package top level). Current truth is the code, tach.toml, and architecture.md; this file is history.

Based on three research passes run today: (1) local-vs-cloud queue abstraction patterns, (2) repo structure in comparable OSS pipelines (dlt, graphiti, cognee, llama_index, unstructured), (3) AWS cost and fit math at our real volume. Sources at the end. The design itself (pipeline, invariants, contracts) is in the PRD and on the board; this doc is only about how the codebase and runtime are shaped.

TL;DR

  • Dev and prod run the same worker loop. Postgres is both the store and the queue (transactional outbox). No broker, no LocalStack, no emulation.
  • Deployment sequencing (decided, Nacho 2026-09-01): local only now → one small Fargate task at first deploy → SQS + Lambda + Neon if scale ever demands it. The deployment dive is deferred; the skeleton just has to keep all three true, which the queue port and the rules below do.
  • Flat package layout, one package per pipeline stage, boundaries enforced by tach in CI. A stage is the unit of ownership: each of us can take one and work without touching the others.
  • Everything append-only, versioned, org-scoped at the identity level. Stage functions are pure; idempotency is decided in exactly one place.

1. Deployment: local now, Fargate first, Lambda + Neon as the scale path

The original idea was SQS + Lambda for cost. The math at our volume (~200 LLM-stage runs/day averaging 3 min, plus fast stages) says otherwise:

Option Compute/mo Pooling/mo Total/mo
SQS + Lambda, whole pipeline ~$12 RDS Proxy ~$22 (flat, 24/7) ~$34
SQS + Lambda without RDS Proxy ~$12 $0 ~$12, but connection-storm fragile
One Fargate task (0.25 vCPU / 0.5 GB, 24/7) ~$9 $0 (long-lived process, normal pool) ~$9-13

Beyond cost, Lambda forces two design problems we would otherwise not have:

  1. The 15-minute hard cap. Extraction on a 90-minute transcript can run 5-15 min. Too close to the ceiling to build against.
  2. Wall-clock billing on idle waits. Our LLM stages spend most of their time waiting on the model API. Lambda bills all of it.

Fargate is infra we already run (keshi), holds a normal SQLAlchemy pool, has no timeout ceiling, and is the cheaper option. So: the worker loop is the deployment unit. Locally you run it with make run; in prod the same loop runs as a Fargate task.

Two things we keep from the SQS research regardless of target:

  • Per-org FIFO maps to SQS natively (MessageGroupId = organization_id) if we ever switch. The port below keeps that door open.
  • Messages are claim checks: (org_id, item_id) pointers, never content blobs. Better hygiene under any transport.

The Neon variant (the scale path)

Neon changes the Lambda math and is why the serverless door stays open instead of closed:

  • Neon ships a built-in PgBouncer pooler (up to 10k pooled connections, no extra cost), which deletes the RDS Proxy line (~$22/mo) that killed the Lambda option above. With Neon, Lambda ≈ $12/mo compute plus a few dollars of DB.
  • Neon is usage-based with no monthly minimum (since Dec 2025) and suspends compute after 5 idle minutes. But note the interaction: an always-on worker polling the outbox keeps the DB awake 24/7 (~$19/mo at 0.25 CU), while an event-driven SQS -> Lambda pipeline lets it actually sleep nights and weekends (~$5/mo). So at scale the two paths converge on cost and the decision becomes purely about the 15-minute ceiling and operational shape.
  • Neon doesn't fix Lambda's two workload problems (15-min cap, wall-clock billing on LLM waits). The job-granularity rule below is what fixes the first one.
  • Org precedent exists: orbit already runs on Neon. Branching also gives per-dev databases, useful for parallel migration work.

What the skeleton does now to keep this path open (cheap, done from day one):

  1. No job may be designed to exceed ~10 minutes. Concretely: extraction runs per chunk-batch, not per transcript. This bounds every job under the Lambda ceiling and means retries lose less work; worth it even if we never leave Fargate.
  2. core/db.py supports two connection modes behind one setting: direct + normal pool (worker) and pooled string + NullPool (Lambda). One gotcha encoded there: PgBouncer transaction mode breaks asyncpg's prepared-statement cache (statement_cache_size=0).
  3. Handlers stay transport-agnostic behind the queue port; the SQS adapter is a dormant stub, not a rewrite.

2. Queue: transactional outbox in Postgres, behind a port

No broker. One append-only jobs table. A stage that finishes enqueues the next stage's job in the same transaction as its output write; that is the entire choreography. Workers claim with FOR UPDATE SKIP LOCKED.

Handlers depend on a protocol, never on a transport:

class QueuePort(Protocol):
    async def enqueue(self, org_id: UUID, job: StageJob) -> None: ...


class PostgresQueue:  # dev + Fargate prod: the jobs table IS the queue
    ...


class SQSFifoQueue:  # dormant adapter: MessageGroupId=org_id, DeduplicationId=job.id
    ...

The one hard part, built in from day one: per-org sequential processing (PRD invariant 7) is not free in Postgres. A naive SKIP LOCKED LIMIT n happily runs two jobs from the same org concurrently. The claim query must exclude orgs that already have a job in flight (Procrastinate's lock model is the reference implementation to copy). This bug is invisible with one local worker, so the test suite runs the claim logic with 2+ concurrent workers.

Terminology rule: "Event" is a domain object (a composition of sources). The queue rows are jobs, never events.

3. Stage contract: pure functions, append-only runs

Synthesized from llama_index's ingestion cache (the idempotency formula), Dagster (code version vs data version as separate axes), and graphiti (bi-temporal edges).

Every stage is a pure async function. No DB writes inside. One wrapper decides idempotency:

class StageInput(BaseModel):
    organization_id: UUID
    payload: ...  # typed per stage
    content_hash: str


class StageConfig(BaseModel):
    prompt_version: str | None  # None for deterministic stages
    ontology_version: str | None


# run_stage() computes:
#   key = sha256(content_hash : stage_name : prompt_version : ontology_version)
# key exists   -> return the prior StageRun (no re-run)
# key is new   -> run fn, append a new StageRun row. Never update, never overwrite.

Consequences:

  • Re-running with identical inputs and versions is a no-op. Bumping a prompt or ontology version always mints a new run. This is PRD invariant 1 as code.
  • content_hash, prompt_version, ontology_version are separate stamped columns, not just folded into the key, so "what changed between runs" stays queryable.
  • Deterministic stages (compose, chunk, normalize) get real unit tests with zero mocks. LLM stages get the same interface with a fake client injected.
  • Claims are bi-temporal per graphiti: created_at/expired_at (when we learned it) separate from valid_at/invalid_at (when it was true). Supersession sets timestamps; nothing is ever deleted.

4. Folder structure

Flat layout, not src/. Verified against dlt, cognee, graphiti and FastAPI's own template: all four are flat. src/ guards a problem only pip-installed libraries have; Indra is a deployed service.

platform-indra/
├── pyproject.toml            # single source of truth: ruff, pyright, pytest, uv dependency-groups
├── uv.lock
├── .pre-commit-config.yaml   # ruff + hygiene only. Type-check and tests stay in CI (slow hooks get skipped)
├── Makefile                  # make test / lint / run: the entrypoints devs actually use
├── indra/
│   ├── api/                  # thin FastAPI: routers call stage services, never touch the DB directly
│   ├── core/                 # config (pydantic-settings), db session, tenancy. Never imports stages/
│   ├── stages/               # ONE PACKAGE PER STAGE: the unit of parallel ownership
│   │   ├── _contracts/       # shared StageInput/StageOutput types: the only cross-stage import allowed
│   │   ├── ingest/           # source adapters: meeting_recording.py, calendar_event.py, ...
│   │   ├── compose/          # sources -> events, deterministic join keys. LLM imports forbidden
│   │   ├── summarize/        # LLM stage. prompts/ live here as Python modules
│   │   ├── chunk/            # deterministic renderers + chunking. LLM imports forbidden
│   │   ├── extract/          # LLM stage. prompts/ + ontology/v0.py (a version is a module, never edited)
│   │   ├── normalize/        # per-claim-type normalizers. LLM imports forbidden
│   │   └── resolve/          # PRD milestone 5: interface stub only, so the boundary exists from day 1
│   ├── runs/                 # StageRun model + run_stage(): the ONE place idempotency is decided
│   ├── models/               # SQLAlchemy, one file per aggregate. base.py = OrgScopedBase (uuidv7 + organization_id)
│   ├── queue/                # QueuePort + PostgresQueue (+ dormant SQSFifoQueue)
│   ├── workers/              # the worker loop: local runner CLI and the Fargate entrypoint are the same code
│   ├── schemas/              # pydantic API I/O models
│   └── alembic/              # migrations inside the package: they ship in the same artifact as the code
├── tests/                    # mirrors indra/ 1:1. Integration tests marked by *_int.py suffix, not a folder
│   ├── unit/stages/...
│   └── contract/             # golden files (syrupy) per stage renderer/normalizer
└── tach.toml                 # boundary enforcement, runs in CI

Prompts and ontology versions are Python modules next to the stage that owns them (graphiti does exactly this). Type-checked, importable, diffable. ontology/v0.py is never edited after v1.py exists, so every historical StageRun stays reproducible against the version it actually ran with.

5. Conventions (the rules that keep this clean with 4 people in parallel)

  1. New stage = new package under stages/, one PR with three things: service.py, its _contracts/ types, and mirrored tests. Never a new function bolted into an existing stage.
  2. Stages never import each other. Only stages/_contracts/ and runs/. Enforced by tach in CI: a deep cross-stage import fails the build, not just review.
  3. compose/, chunk/, normalize/ cannot import any LLM client. Also tach-enforced. This is what keeps them testable with zero mocks.
  4. Stage functions are pure. Only runs/service.py writes the runs table; only the applier (later) writes entity state.
  5. Every table inherits OrgScopedBase. organization_id is part of identity, never a filter convention. (cognee retrofitted tenancy as a string-prefix hack every reader must remember; it is their biggest scar and we are not repeating it.)
  6. No native Postgres enums on evolving status columns (text + CHECK constraint instead). Another cognee scar: a frozen enum forced a second bolt-on status system on the same table.
  7. Version bumps add modules, never edit them. prompts/v2.py next to v1.py.
  8. One PR = one stage or one cross-cutting concern. A PR touching two stages' service.py at once means the boundary is leaking: split it or fix the boundary.
  9. No job designed to exceed ~10 minutes (see the Neon variant above). If a stage's work can grow with input size, it must shard (per chunk-batch, per source), not stretch.

6. Tooling

Concern Pick Note
Deps/runtime Python 3.12+, uv with PEP 735 dependency-groups dev group stays narrow (lint/type only)
API FastAPI + pydantic v2
DB Postgres, SQLAlchemy 2 + Alembic not SQLModel: append-only tables and API projections stay decoupled
LLM Anthropic SDK / OpenRouter + instructor retry-on-validation is the extraction shape
Boundaries tach actively maintained; enforces per-module public interfaces, which import-linter lacks
Lint/types ruff + pyright pyright now; Astral's ty is the one to watch, not yet the safe default
Tests pytest + syrupy golden files for renderers and normalizers
Hooks pre-commit: ruff + hygiene only slow hooks get skipped, so type-check and tests live in CI
Eval UI (P7) Streamlit throwaway by design

Anti-model, for the record: cognee. Five files and ~900 lines of orchestration indirection, a 20-parameter god function as the public API, DB context injected via globals, no boundary enforcement, tenancy retrofitted. We cloned it and read it; every one of those is a concrete scar we design against.

7. What gets scaffolded first (Phase 0)

  1. Repo skeleton exactly as the tree above: pyproject, uv, Makefile, pre-commit, tach.toml, CI config.
  2. Migration 1: organizations stub + OrgScopedBase + the jobs and stage_runs tables.
  3. The worker loop with the per-org claim query, plus the 2-concurrent-worker test that proves the FIFO invariant.
  4. run_stage() and the _contracts/ types.
  5. One example stage wired end to end (a trivial deterministic one) as the template everyone copies.

After that, stages can be picked up in parallel: ingest adapters, compose, chunk renderers, summarize, extract, normalize.

Decisions

  1. Taken (Nacho, 2026-09-01): local-only now; Fargate worker at first deploy; SQS + Lambda + Neon as the scale path, kept open by the queue port, the job-granularity rule, and the db-mode toggle. The deployment dive happens later, with real job-duration data from the fixture corpus.

Still asking the team to confirm:

  1. Postgres outbox as the queue; no broker; no LocalStack.
  2. The stage contract (pure functions, run_stage() idempotency, append-only runs).
  3. The tree and the 9 conventions, tach-enforced.

Sources (highlights)

  • dlt (stage-as-package, versioned state), graphiti (bi-temporal edges, prompts-as-modules, driver adapters), llama_index (ingestion idempotency hash), unstructured (LLM-free chunking isolation), cognee (read as the anti-model)
  • AWS pricing: Lambda, Fargate, RDS Proxy ($0.015/vCPU-hr of the underlying instance, 24/7)
  • SQS FIFO docs: MessageGroupId ordering, per-group blocking, visibility timeouts, the Jan 2026 1MB payload bump
  • Procrastinate docs: queueing locks (the per-key claim pattern to copy)
  • aws-lambda-powertools v3.34 / Mangum v0.22: both maintained, only needed if the SQS adapter wakes up
  • LocalStack: repo archived + releases gated behind auth in March 2026; another reason not to depend on emulation