Engineering
Cutting courier allocation latency to 40ms.
How we re-architected Shipclues Intelligence v2: lane caches, async carrier polling, and the trade-offs that shaved hundreds of milliseconds off every order.
Allocation sits on the critical path of every order. When a brand pushes an order, our engine has to score every eligible carrier on cost, ETA, and lane success, then commit to one, before the label can print. The first version of this did the obvious thing and paid for it in latency. Here is how Intelligence v2 got the decision down to a typical 40ms.
~40ms
typical allocation decision, p50
The problem: synchronous carrier polling
v1 queried carrier systems in sequence at decision time. Correct, simple, and far too slow: a single slow upstream blocked the whole decision, and tail latency was brutal during peak. The decision was only as fast as the slowest carrier we asked.
Lane caches: precompute the boring part
Most of what allocation needs does not change order to order. Coverage, indicative ETA, and rolling success rate for a given lane are stable over minutes, not milliseconds. We moved those into a lane cache that is refreshed asynchronously, so the hot path reads precomputed signal instead of calling out live.
What stays live
Order-specific inputs (weight, declared value, COD flag, the risk score from RTO Shield) are computed in-request. The split is the whole trick: cache what is shared and slow-moving, compute what is unique to the order.
Async carrier polling and graceful degradation
Carrier health is now polled out of band and folded into the cache. If an upstream degrades, allocation routes around it without anyone waiting on a timeout. The decision never blocks on a flaky carrier; it simply has fresh-enough data to avoid the bad one.
- Lane signal: cached, refreshed asynchronously.
- Carrier health: polled out of band, never on the hot path.
- Order specifics: computed in-request, every time.
The trade-offs we accepted
Caching means occasionally acting on signal that is a minute stale. We decided that a marginally older success rate, applied instantly, beats a perfectly fresh number that arrives 400ms late and stalls the order. For allocation, decisiveness compounds; latency on the critical path does not.
Figures in this article are directional placeholders. TODO: VERIFY before launch — replace with audited operational data before indexing.