Multi-channel distributor · Pipeline · Platform
From 18-hour batch to 90-second publish
Replaced a brittle nightly ETL with event-driven propagation across PIM, search, and storefront.
18h→90s
Publish latency
99.97%
Pipeline success rate
0
Weekend cutover incidents
Period
2024 · 6 months
Stack
Kafka · TypeScript · Algolia · Commercetools
Problem
The nightly batch had become a business constraint — promos couldn't ship same-day, price updates lagged a full day, and operations scheduled around it.
Approach
Treated the pipeline as a product: SLOs, observability, and a rollback path before we started replacing the legs.
Kept the batch path running for a full cycle in parallel and diffed the two outputs daily — cutover happened only after a week of clean diffs.
Hindsight
The hard part wasn't the pipe — it was convincing every downstream consumer that the new contract was actually stable.