> ## Documentation Index
> Fetch the complete documentation index at: https://docs.vinmake.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Idempotency, outbox and durability

> Re-running is safe, a crash mid-flight loses nothing, and a duplicate document group may enter the ledger once.

# Idempotency, outbox and durability

## What it guarantees

Re-running anything is safe.
A crash mid-flight loses nothing and strands nothing.
And one invoice becomes one journal entry, however many times it is uploaded, approved or retried.

Three separate mechanisms carry that, at three different layers.

## 1 · Ledger idempotency

Every entry carries a source key:

```
(org_id, source_type, source_id, entry_set)
```

The writer looks it up **before** it does anything else, under the advisory lock.
An exact replay returns the existing entry and writes nothing.
A reuse of the key with different header, line, currency, dimension or prepared base facts raises a typed source-replay conflict instead of returning an unrelated old entry.

New entries store a request fingerprint over those immutable facts.
Historical entries without one are compared by reconstructing the same identity from their immutable header and lines; GreatBook does not backfill a guess or reinterpret their stored checksum.

That is [GL invariant 10](/greatbook/capabilities/gl-invariants), and it is what makes retries, replays and agent re-runs safe.

There is also a database-level backstop.
A concurrent exact double-post resolves to the one durable entry.
A concurrent request carrying different facts conflicts and the caller rolls it back.

<Note>
  The key is namespaced **per intake row**, not per document reference.
  That is deliberate: two genuinely distinct documents that happen to quote the same bill number must not collapse onto one entry.
  The duplicate problem is solved one layer up, by the duplicate-group guard, which is a control a human participates in.
</Note>

## 2 · The intake pipeline: dedup at two levels

### A repeated document reference flags, it never crashes

`(org_id, doc_type, source_ref)` is a document's natural key, and it is a partial unique index.

The second upload of the same reference used to die at the terminal write and strand the row at `received` with a null document type - an orphan nobody could act on.
Now both queue stores raise a typed conflict, never a bare unique violation, and the terminal node recovers into a **flagged duplicate for a human**.

The Clerk never merges.
Deciding that two documents are the same document is a judgment with money attached, so it goes to a person.

### A duplicate group may enter the ledger once

The other half of that posture, and the one that protects the ledger.

Because the ledger's idempotency key is namespaced per intake row, approving two copies would write **two immutable entries for one invoice** - and an immutable entry can only be reversed.

So `intake_service.approve` **refuses** with a 409 when any other row of the same group is already accepted for posting: `active`, `posted`, or carrying a result entry id.

Three properties of that rule:

* It is stated over the **group** - every row connected by `duplicate_of` edges in either direction, transitively - never over one edge. Forward, reverse, sibling and chain are one rule, and enumerating topologies is how the next one is missed.
* Members that are merely pending, flagged or rejected do **not** block. That is the reconciliation the flag exists for, and over-refusing strands the whole group.
* The group lock must stay the **first** lock `approve` takes: the whole group, ordered by id, in one statement. Otherwise two reviewers approving two members deadlock into a Postgres serialisation failure the API renders as a 500.

### The version-group guard

The Typewriter adds a third case.
A document approved as v1 and then revised to v2 leaves v2 legitimately approvable and submittable, with a different PDF hash, therefore a different idempotency key, and nothing linking its intake row to v1's.

So the submit path asks for any live intake row filed by **any version of the group** - stated over the group, never over the `supersedes_id` edge - and files the new row `flagged` with `duplicate_of` pointing at the incumbent.

That is what makes the pair visible to the duplicate-group guard, which is the control that actually protects the ledger.

The incumbent read and the queue insert are **one critical section**, under a namespaced advisory lock held across the insert.
A check outside the lock is advisory, and two versions submitted concurrently would both see no incumbent.

## 3 · The approve handoff is an outbox, not a call

`intake_service.approve()` writes the status change, the audit event and an outbox row in **one transaction**, commits, and launches the Bookkeeper **only after that commit**.

```mermaid theme={null}
sequenceDiagram
  participant A as approve()
  participant DB as Postgres
  participant BK as Bookkeeper run
  A->>DB: BEGIN
  A->>DB: status → active
  A->>DB: intake_events audit row
  A->>DB: intake_launch_outbox row (pending)
  A->>DB: COMMIT
  A->>BK: launch (after the commit)
  BK->>DB: read intake_queue on its own connection
  Note over BK,DB: the row is committed, so the run can see it
  A->>DB: record the launch outcome on the job row
```

Moving the launch back inside the transaction is the bug this shape exists to prevent: the run reads `intake_queue` on a **different connection** and cannot see an uncommitted active row, so it dies with a not-postable error and the approved document is lost.

Three consequences:

* `launch_bookkeeper` **raises** on failure by design. The single owner of launch-failure handling records the outcome on the job row and in the audit trail, and returns it in the approve response, so a reviewer learns immediately.
* Two recovery paths exist: an operator retry endpoint, and a sweep that also picks up jobs left pending by a crash between the commit and the launch.
* `abandoned` is a dead letter and is **deliberately never auto-retried**. A job that has exhausted recovery needs a person, not another attempt.

## 4 · Durability: the checkpoint is the source of truth

Agent runs are checkpointed in Postgres, so a run - including a *paused* one - survives a process or container restart.

The rule that makes that safe:

> **The checkpoint is the in-flight source of truth; the queue row is a terminal projection.**

The Clerk graph writes to `intake_queue` at exactly two points: `ingest` (idempotent on the content-hash key) and the terminal `queue` node (one transaction).
Intermediate nodes never touch the database.

So a mid-run crash leaves at most a `received` row that the checkpoint resumes.
There is no orphan sitting at `classified` or `extracted` for someone to find later and wonder about.

Retries are bounded and typed: only transient faults - network, timeout, 5xx, rate limit - are retried, never a permanent shape error.
And the `validate` to `extract` repair loop is bounded, so a document the model cannot get right flags for a human instead of looping.

<Warning>
  Use the **session-mode** database DSN (port 5432) for the checkpointer, not the transaction pooler (6543).
  Both the saver and the GL's advisory lock need a stable session.
  This is the kind of configuration detail that works in development and produces intermittent, unattributable failures in production.
</Warning>

## Where it is enforced

| Concern                                                      | Code                                                                |
| ------------------------------------------------------------ | ------------------------------------------------------------------- |
| the ledger idempotency key, and the concurrent-post backstop | `backend/app/services/ledger_service.py`                            |
| the intake natural key and the duplicate flag                | migrations `0013_intake_idempotency`, `0018_intake_duplicate_of`    |
| the duplicate-group guard and its lock ordering              | `langgraph_chat/devcenter/intake_service.py`                        |
| the outbox                                                   | migration `0017_intake_launch_outbox`, `intake_service.approve`     |
| the version-group guard                                      | `langgraph_chat/devcenter/typewriter_service.py`, `submit_to_clerk` |
| the checkpointer                                             | `langgraph_chat/agents/common/checkpointer.py`                      |
| the two-write-point rule and bounded retries                 | `langgraph_chat/agents/clerk/graph.py`                              |

## The tests that would fail if it broke

| Suite                                           | What it proves                                                                        |
| ----------------------------------------------- | ------------------------------------------------------------------------------------- |
| `devcenter/tests/test_intake_launch_outbox.py`  | the outbox contract, including the recovery paths                                     |
| `devcenter/tests/test_duplicate_double_post.py` | a duplicate group enters the ledger once, across every edge topology                  |
| `agents/tests/test_dedup.py`                    | the flagged-duplicate recovery, offline, against the same index the database enforces |
| `agents/tests/test_hardening.py`                | checkpoint durability and the two-write-point rule, against a real Postgres           |

The dedup suite is worth a note: the in-memory queue store enforces the **same** partial unique index as Postgres, so the offline test exercises the real recovery path rather than a simplified one.

## What goes wrong without it

Without ledger idempotency, a retried webhook posts the invoice twice, and because entries are immutable the fix is a reversal that every downstream reader has to understand as noise.

Without the outbox, an approved document silently never reaches the Bookkeeper, and nobody finds out until a close reports approved documents with no journal entry.

Without the duplicate-group guard, the same invoice posts twice under two different reviewers, each of whom did their job correctly.

## Related

* [The 18 GL invariants](/greatbook/capabilities/gl-invariants) - invariants 10 and 16
* [The posting mandate](/greatbook/capabilities/posting-mandate) - why a replayed mandate is a no-op
* [The Clerk](/greatbook/agents/clerk) - the intake pipeline and its two write points
* [Composition and the render seam](/greatbook/capabilities/composition-and-rendering) - where the version-group guard originates