Back to blog

EngineeringJun 24, 2026 · 9 min read

Why we rebuiltthe HUB365sync enginefrom scratch

Three years of patches taught us exactly where the original design was fighting us. How we decided a rewrite was the smaller risk — and what we would do differently.

Office building at night, three floors of lit windows with people working at their desks

Every engineer knows the rule: never rewrite a working system. For three years we followed it. The HUB365 sync engine moved millions of records a day, clients were happy, and every incident had a patch. Then the patches started patching each other.

The original engine was designed in 2020 for ten clients and a single region. By 2025 it served two hundred clients across three regions, and every assumption baked into its core — one writer, ordered delivery, small payloads — had quietly become false.

The moment the math flipped

We tracked it honestly for two quarters: 38% of engineering time was going into keeping sync healthy. Not building — keeping alive. When maintenance costs more than a rebuild amortized over two years, the "risky" option is staying put.

So we wrote the smallest possible spec. No new features. Same external behavior, byte for byte, verified by replaying six months of production traffic against both engines in parallel.

A rewrite is only reckless when you change what the system does. We changed nothing — except everything underneath.

What we would do differently

Start the shadow traffic replay on day one, not month three. The replay harness found nine behavioral differences we would never have caught in review — including two bugs in the old engine that clients had silently built workarounds for.

We kept both bugs. That was the hardest and most correct decision of the project: compatibility means compatible, not correct.

The new engine shipped in March. Sync incidents are down 94%, and for the first time since 2023, the on-call channel went a full month without a page.

Related insights

Keep reading