Home
[01]Engineering field notes

Production lessons, tested before they are published

These field notes are for backend engineers, technical founders, and product teams making decisions that will have to hold up in production. Each article starts with a real failure, measurement, implementation detail, or reproducible experiment, not a rewritten documentation page.

The writing follows three tracks. Distributed systems and integrations covers queues, webhooks, retries, idempotency, and the failure boundaries between services. Data consistency and performance looks at schema design, query behavior, transactions, and the evidence needed before optimizing. Node.js runtime and reliability focuses on event-loop behavior, memory, observability, and operating services after launch.

A useful note should leave you with a concrete mental model, a way to reproduce the behavior, and implementation tradeoffs you can apply to your own system. Source code, diagrams, version numbers, and limitations are included when they materially support the conclusion. Start with the article index below, or review thebackend case studiesfor examples of product work under delivery constraints.

Node.js Reliability & OperationsField note / 01

Diagnosing Slow Node.js APIs: Event Loop, Database, and p95

Trace slow Node.js APIs by separating queueing, event-loop delay, pool wait, database time, downstream calls, CPU, GC, and p95 latency.

  • Tail latency
  • Queueing delay
  • Event-loop delay
13 min readbenchmarkdiagramprimary-sourceOpen note
Data Consistency & PerformanceField note / 02

MongoDB or PostgreSQL for Multi-Tenant SaaS: Choose by Access Pattern

Choose MongoDB or PostgreSQL for multi-tenant SaaS by testing access patterns, invariants, isolation, growth, recovery, and operational ownership.

  • Tenant isolation
  • Compound indexes
  • Row-level security
15 min readdiagramprimary-sourceOpen note
Data Consistency & PerformanceField note / 03

PostgreSQL Schema Migrations for SaaS Without Downtime

Plan PostgreSQL SaaS migrations with bounded lock waits, compatible releases, batched backfills, validated constraints, and clear recovery steps.

  • Table lock modes
  • Lock queues
  • Expand and contract
14 min readsource-codediagramprimary-sourceOpen note
Distributed Systems & IntegrationsField note / 04

Reliable Third-Party Integrations: Timeouts, Backoff, and Reconciliation

Design reliable Node.js integrations with phase timeouts, bounded retries, jitter, durable commands, idempotency, dead letters, and reconciliation.

  • Transactional outbox
  • Idempotency keys
  • Retry budgets
15 min readbenchmarksource-codediagramprimary-sourceOpen note
Distributed Systems & IntegrationsField note / 05

Webhook Retries Without Duplicate Side Effects in Node.js

Build a Node.js and PostgreSQL webhook inbox that handles retries, concurrent deliveries, lost acknowledgements, and crash recovery safely.

  • At-least-once delivery
  • Webhook signatures
  • Replay protection
15 min readsource-codediagramprimary-sourceOpen note
Distributed Systems & IntegrationsField note / 06

Why BullMQ Jobs Run Twice: Retries, Stalls & Idempotency

A controlled BullMQ reproduction showing why retries and stalled jobs repeat processing, and how a durable idempotency key protects side effects.

  • At-least-once processing
  • Worker locks
  • Stalled jobs
12 min readsource-codediagramprimary-sourceOpen note