Construction Ops Multi-Agent System

PROBLEM

The answer to a payment dispute lives in three places: a payments table, a contract, a site log.

USER

Project managers, admins and zone-restricted contractors (roles in the schema).

SOLUTION

A Supervisor agent routes the question to Payment, Contract and Progress specialists, then an Audit agent checks every claim.

MY OWNERSHIP

Solo project: architecture, backend, data model, tools, tests, React trace UI, Docker setup.

OUTCOME

Working end-to-end system with cited answers and access control proven by 11 tests. Baseline evaluation in progress.

“Is the Zone 3 contractor owed for drywall, and can the client withhold payment because inspection photos are missing?”

[Optional: one line on why you built this — e.g. what you saw while building Nirman]

Collaborators: None — solo build. Open-source used as-is: pgvector, Express, the Anthropic and OpenAI SDKs, Ollama models.

Switch to Engineer view (top right) for inputs, outputs and edge cases.

• Zone-ID array + ANY()

  AccessContext.allowedZoneIds → every tool’s WHERE clause

  The user’s scope travels as data into SQL: z.id = ANY($n::uuid[]). A tagged union (string[] | 'all') makes “no restriction” explicit instead of an empty list.

  Speeds up: Filtering to permitted zones inside the query

  O(k) per candidate row, k = zones a user holds (typically 1)

  Alternative · trade-off · at scale+

  Considered: Prompt instruction, or filtering rows in app code after the query

  Accepted: Every new tool must take the context and repeat the clause — enforced by its type signature, not by the database.

  At scale: Move the rule into Postgres row-level security so new queries can’t forget it.

• IVFFlat vector index

  doc_chunks.embedding (vector_cosine_ops, lists = 100)

  Top-K cosine search over contract chunks without comparing the query to every stored vector.

  Speeds up: Nearest-clause lookup for the Contract agent

  Exact scan O(n·d). IVFFlat ≈ O(lists·d + (probes/lists)·n·d)

  Alternative · trade-off · at scale+

  Considered: Exact scan (perfect recall), or HNSW (better recall/speed, more memory, slower build)

  Accepted: Approximate recall. With a small corpus, 100 lists is oversized — recall must be checked, or probes raised.

  At scale: Size lists to roughly rows/1000, or switch to HNSW once the corpus is large.

• B-tree foreign-key indexes

  zones.project_id, milestones.zone_id, payments.milestone_id, progress_logs.zone_id, …

  Every tool joins zones → milestones → payments and filters by project and zone.

  Speeds up: Join and filter lookups

  O(log n) per lookup vs O(n) sequential scan

  Alternative · trade-off · at scale+

  Considered: No secondary indexes (fine at seed-data size)

  Accepted: Small write and storage overhead per insert.

  At scale: Milestone names use ILIKE, which a B-tree can’t serve — add a pg_trgm index if names get searched at volume.

• Hash-map agent dispatch

  AGENT_RUNNERS: Record<SpecialistName, runner>

  The planner returns agent names as strings; a typed map turns each into its function and skips anything unknown.

  Speeds up: Routing a planned task to its specialist

  O(1) lookup

  Alternative · trade-off · at scale+

  Considered: switch / if-chain

  Accepted: None significant at three agents.

  At scale: Tasks run sequentially today — run independent specialists with Promise.all to cut latency.

• Ordered, append-only trace log

  In-memory TraceEvent[] + trace_steps rows keyed by step_index

  Each step is appended once, in order, and streamed and stored from the same event.

  Speeds up: Replaying and inspecting how an answer was produced

  O(1) append; one INSERT per step

  Alternative · trade-off · at scale+

  Considered: Unstructured console logging

  Accepted: A synchronous DB write per step adds latency; failures are swallowed so answers never break.

  At scale: Batch or queue trace writes, or export to OpenTelemetry.

• Greedy paragraph packing

  Contract ingestion (src/ingestion/chunk.ts)

  Keeps whole paragraphs — usually whole clauses — together, up to ~180 words per chunk.

  Speeds up: Clause-level retrieval with page citations

  O(n) single pass over the text

  Alternative · trade-off · at scale+

  Considered: Fixed-size token windows with overlap

  Accepted: No overlap, so a clause split across chunks can lose context. Page numbers are estimated (~400 words/page) on plain-text contracts.

  At scale: Parse real PDF page boundaries and add overlap between chunks.

1. DECISION 01

   Enforce access in SQL, not in the prompt

   Contractors may only see their zones, but the model picks tool parameters from free-text questions.

   CHOSE

   Tools accept an AccessContext — never a raw user ID — and apply the zone filter inside the query. A mismatched projectId throws.

   BECAUSE

   A prompt can be argued with. SQL can’t.

   RESULT

   11 tests: zero cross-zone rows with an explicit zone, no filter, milestone name only, or the wrong project.

   Options considered · cost accepted · at 10× scale+

   Options: System-prompt rule · Filter rows in app code after the query · Resolve scope once, bake it into every parameterized WHERE clause

   Cost accepted: Every new tool must thread the context and repeat the scope clause.

   At 10×: Postgres row-level security, so the database enforces scope even for queries written later.

2. DECISION 02

   Specialists + a bounded Audit, not one RAG pass

   Compound questions mix exact numbers, contract prose and site records.

   CHOSE

   A Supervisor plans through a forced tool call; three specialists each own one tool; Audit verifies claims and may trigger exactly one targeted retry.

   BECAUSE

   Numbers come from SQL, clauses from vectors, and a hard retry cap keeps cost and latency predictable.

   RESULT

   Every final claim is either backed by a cited source or rewritten as a caveat.

   Options considered · cost accepted · at 10× scale+

   Options: Single vector-search pipeline · One agent holding every tool · Supervisor with typed specialists · Open-ended self-correction loop

   Cost accepted: Up to 6 LLM calls per question on the happy path, up to 10 with a retry.

   At 10×: Run specialists in parallel and use a smaller model for planning and audit.

3. DECISION 03

   Retry LLM calls — never database writes

   External model APIs fail transiently (rate limits, 5xx, dropped connections).

   CHOSE

   withRetry: 30 s timeout, 500 ms × 2ⁿ backoff, only on 429, 5xx and network errors. Postgres gets a statement timeout and no retry.

   BECAUSE

   Retrying a non-idempotent INSERT on a transient failure can duplicate data.

   RESULT

   Failures are explicit; no duplicate writes by design.

   Options considered · cost accepted · at 10× scale+

   Options: Retry everything · Retry nothing · Retry only idempotent external calls

   Cost accepted: A transient DB failure surfaces as an error instead of self-healing.

   At 10×: Idempotency keys so writes can be retried safely, plus jitter and a circuit breaker on model calls.

4. DECISION 04

   Stream over SSE with errors as events

   An answer takes several model calls; users need to see progress, not a spinner.

   CHOSE

   GET /api/ask read by the browser’s native EventSource, emitting trace_id, step, done and ask_error.

   BECAUSE

   One-way streaming, zero client dependencies. Errors travel as events because EventSource can’t read a non-200 body.

   RESULT

   The trace panel fills step by step as agents finish.

   Options considered · cost accepted · at 10× scale+

   Options: Single JSON response · WebSockets · fetch() streaming over POST · SSE over GET with EventSource

   Cost accepted: The question travels in the URL, and EventSource can’t send an Authorization header.

   At 10×: POST with fetch streaming and an auth header; resume with Last-Event-ID.

1. 01 · RISK

   The model chooses tool parameters from the wording. The same question can arrive with an explicit zone, no zone, only a milestone name — “Drywall” exists in every zone — or the wrong project.

2. 02 · ROOT CAUSE

   Any scope applied after the model picks parameters depends on the model behaving.

3. 03 · INVESTIGATION

   Listed every parameter shape a phrasing can map to and treated each as an attack path against the seeded data (a Zone 3 contractor, a Zone A contractor, a full-access admin).

4. 04 · SOLUTION

   Resolve an AccessContext once per request. Every tool requires it, adds z.id = ANY(...) to its WHERE clause, rejects a mismatched projectId, and returns nothing when the user holds no zones.

5. 05 · VERIFICATION

   11 Vitest tests against the seeded database, including an admin regression test so the fix can’t over-block.

6. 06 · LESSON

   Prompts guide behaviour; permissions belong in code the model can’t negotiate with.

11

access-control tests across every tool-parameter shape

tests/access-control.test.ts

4

agents + Supervisor, each with one typed tool

src/agents, src/supervisor

1

retry cap on self-correction — the loop cannot run away

src/supervisor/index.ts

2

provider modes: Claude + OpenAI, or fully offline on Ollama

src/llm

• I turn ambiguity into a clear problemReframed “payment disputes” as a three-source reconciliation with citations.See evidence
• I make deliberate trade-offsFour decisions with the options, costs and 10× plans written down.See evidence
• I think in systems, not componentsAccess, orchestration, retries, tracing and storage designed as one flow.See evidence
• I own outcomes end to endSolo build from schema to Docker Compose to UI.See evidence
• I test what matters mostThe security boundary is the part with automated proof.See evidence
• I communicate technical ideas clearlyEach agent documented with what it does and what it must never do.See evidence
• I can name the next bottleneckSequential specialists, index tuning, row-level security.See evidence