Consistency choices for unreliable networks

Users don’t experience your lab p50. They experience elevators, buses, and partial writes. On a vaccine tracking product reaching 10,000+ daily users, “works on Wi‑Fi” was not an architecture.

The senior decision was classifying state: what must be correct immediately, what can reconcile later, and how the client communicates uncertainty without looking broken. Sync and fallbacks were core design, not an edge-case backlog item.

We still held a sub-300ms healthy-path target on the backend — performance under good conditions builds trust — but we designed primary flows for interruption. Retries without semantics create duplicate chaos; reconciliation without UX creates silent wrongness.

That mindset transfers to AI and cloud systems: assume partial failure, make recovery explicit, and measure under pressure. Resilience is a product and API contract, not a library checkbox.

Senior takeaway: write the degradation story before the feature list. If the system only works in the demo network, you don’t have a system yet.