Durability

Effect is not BEAM. Durability14 lives in storage. Effect29 runs each fiber23 as a concurrent computation in one JavaScript runtime, not as an isolated process. Effect 4.0.0 has no OTP restart supervisor. Its structured fiber ownership is useful, but it does not rebuild lost state.

Remember from chapters 8 and 9 that processes lose their state on termination.

Saša Jurić, Elixir in Action, Other Use Cases for ETS (B2). Even OTP restart needs a separate state-recovery decision.

A slow provider must not become our outage. Each pattern below uses an Effect primitive at a real seam. Each one has a fence45 that fails on a planted copy of the bug. The public page is /systems/durability9.

When to use it

Load this skill before you change any of these:

For ordinary pure functions, use uncomplect. For finite modes, use add a lifecycle machine.

Patterns

  1. Lease a connection for the query
  2. Answer at commit9; dispatch off the request
  3. Set a provider timeout once
  4. Bound work in flight
  5. Scope34 waiters to the request
  6. Climb a flag with a ledger
  7. Fail open on liveness, closed on harm

The references hold the detail:

Read the actual boundary

Read the project’s instructions, the pinned versions, the service26, the storage adapter27, the acknowledgement path, and the restoration tests. Check Effect APIs against the installed source. The mappings here are our reading of Effect, not the book authors’ advice about Effect.

For each accepted operation, name:

If no durable record exists, call the work best-effort. A running fiber, an in-memory queue, or a machine snapshot held only in memory is not that record.

Work through a change

  1. Name the slow thing: the provider, pool, or store, and the worst latency it can show.
  2. List what the change holds while it waits: a connection, the request, a permit, or a shared waiter.
  3. Name the durable record and the owner that redrives it.
  4. Pick the pattern from the list. Read its principle file.
  5. Put the primitive at one seam. Read the timeout from a Context.Reference or Config, never a literal.
  6. Write a property or model test that generates the slow case and the crash.
  7. Plant the bug. The test must fail. Restore the code.
  8. Run pnpm turbo run check test build.

Principles, ranked

Rank by risk of lost work, repeated effects, and cascading failure. The numbers are this skill’s priorities, not a ranking from the books. Every Effect answer below exists in the pinned 4.0.0; the Effect map shows the check.

  1. Persist accepted intent and recovery progress. Name the backend.
    • Failure prevented: A healthy restarted worker silently loses accepted work.
    • Effect answer and remaining work: effect/workflow: Workflow, Activity, WorkflowEngine. Select a persistent engine; WorkflowEngine.layerMemory is for tests. rat-stack’s join intake uses a contact marker and a sweep instead.
    • Book evidence: B1, B2, B3
  2. Make repeated external effects safe before enabling retries.
    • Failure prevented: A crash after remote success repeats a write.
    • Effect answer and remaining work: Activity.idempotencyKey names work. The receiver or authoritative store must enforce deduplication14; drovr does it by submission id.
    • Book evidence: B5, B6
  3. Couple state and checkpoint10 where one transaction can own both. Acknowledge only the promised handoff.
    • Failure prevented: A checkpoint advances past missing work, or replay reapplies committed changes.
    • Effect answer and remaining work: SqlClient.withTransaction. No atomic transaction spans arbitrary providers.
    • Book evidence: B5, Chs. 7, 9
  4. Give every task a lifetime owner and every failed worker a recovery owner.
    • Failure prevented: Escaped tasks, unnoticed death, or endless restart churn.
    • Effect answer and remaining work: Effect.forkScoped, Scope, FiberSet. Add restart policy in XState or a host supervisor.
    • Book evidence: B1, B2, B3
  5. Bound waiting and carry the caller’s remaining deadline.
    • Failure prevented: Abandoned calls keep consuming scarce capacity.
    • Effect answer and remaining work: Effect.timeoutOption. A timeout does not prove a remote write stopped.
    • Book evidence: B4 §5.1; B12 Ch. 22
  6. Put retries at one informed boundary, with jitter and a finite budget.
    • Failure prevented: Nested loops and synchronized retries amplify an outage.
    • Effect answer and remaining work: Effect.retry with Schedule.exponential, jittered, recurs, upTo. Shared retry accounting is separate policy.
    • Book evidence: B4 §5.1; B9; B10 §10.6; B12 Ch. 22
  7. Bound admitted work and isolate scarce capacity by workload.
    • Failure prevented: A noisy caller fills every slot and backlog grows without limit.
    • Effect answer and remaining work: Bounded Effect.forEach, Semaphore, Queue.bounded; Pool for a real resource pool10.
    • Book evidence: B3; B4 §§5.3, 5.6
  8. Stop calling a known-unhealthy dependency and probe recovery explicitly.
    • Failure prevented: Retrying doomed calls exhausts the capacity recovery needs.
    • Effect answer and remaining work: No circuit-breaker API in 4.0.0. Model closed, open, and half-open in XState.
    • Book evidence: B4 §5.2; B11
  9. Recover from known state, with stable replay inputs and recoverable deadlines.
    • Failure prevented: Restart changes old decisions or forgets scheduled work.
    • Effect answer and remaining work: Activity, DurableClock, DurableDeferred under a persistent engine.
    • Book evidence: B1; B3; B5 Ch. 6
  10. Preserve old payloads’ meaning across deployments.
    • Failure prevented: Old accepted work becomes undecodable during recovery.
    • Effect answer and remaining work: Schema.decodeUnknownEffect decodes; an optional key keeps old records valid. Define version support and transition tests.
    • Book evidence: B5 Ch. 3; B14 §3.11
  11. Park exhausted or malformed work durably and give it a return path.
    • Failure prevented: Poison work14 disappears, retries forever, or parks invisibly.
    • Effect answer and remaining work: PersistedQueue, DurableQueue keep failure records. Redrive, visibility, and repair stay application work.
    • Book evidence: B5 Ch. 9; B9
  12. Keep expected refusals, transient faults, and defects distinct.
    • Failure prevented: A crash becomes a permanent refusal, or a refusal is retried.
    • Effect answer and remaining work: Schema.TaggedError, Effect.catchTag.
    • Book evidence: B7; B10 Ch. 10
  13. Recycle accumulation and bound results at their source.
    • Failure prevented: Retained records or a huge query exhaust storage or memory.
    • Effect answer and remaining work: Cache, Scope, Stream.take, PersistedQueue.layerCleanup. Add retention and query limits.
    • Book evidence: B4 §§4.4, 5.4
  14. Prove recovery with deterministic histories and bad dependencies.
    • Failure prevented: A design passes happy-path tests but loses work at the commit and acknowledgement gap.
    • Effect answer and remaining work: TestClock, Deferred, Arbitrary, it.effect.prop. Write the model, inject faults, test real storage separately.
    • Book evidence: B4 §5.7; B11; B15
  15. Observe useful outcomes, backlog, and saturation.
    • Failure prevented: Fast failures, parked work, or growing queues look healthy.
    • Effect answer and remaining work: Metric.counter, gauge, histogram. Dashboards and alerts sit outside the constructors.
    • Book evidence: B8; B12 Ch. 26

Protect critical clients by giving them their own pool to call.

Michael T. Nygard, Release It!, §5.3 (B4). One global limit does not reserve capacity for critical work.

Choose what owns recovery

This is house composition, not a claim that the authors recommend these products.

Keep one recovery authority for each operation. When an existing Durable Object or database already owns progress, adapt it; do not add a second ledger by default.

Effect.forkDetach is the detached-fiber name in 4.0.0; there is no forkDaemon. Detaching changes lifetime inside a live runtime. It does not persist work.

Check the uncomfortable boundaries

Use the guarantee the contract27 promises, not a generic claim of exactly-once execution.

These are proposed checks derived from the cited failure modes. Passing an in-memory test does not prove the backend’s crash contract.

Lore

The durability index34 gives the reading order. Start with these pages:

The durability bibliography33 lists the books and quotes behind these pages.

Finish

Report the persistence owner, recovery owner, operation identity, acknowledgement boundary, remaining uncertainty, and checks run. Name what the design deletes. Leave a missing guarantee explicit.

Bibliography

The full bibliography lists the chapters read, edition notes, and every checked quote.

Last changed 2026-10-24 in 9bb993d. Source on GitHub. Change log.