A product suggestion that lands after the moment has passed is worthless. Here's how Wink suggests products while the customer is still talking.
Wink listens to live sales calls and puts relevant products in front of the agent while the call is happening. When a caller asks for "the earrings that go with the pendant I bought," the agent should see those earrings before saying "let me check." Most of the obvious designs are too slow for that.
This post covers the shape of our approach: what we let large language models do, what we keep away from them, and how small learned models decide when to speak at all.
The latency budget of a live call
In conversation, people expect a response within a second or two of finishing a sentence. A rep who waits longer starts filling the silence: "one moment," "let me look that up," or a hold. After that, the suggestion is mostly clutter.
So the budget runs from the customer's last word to a card on screen, and inside it we need speech recognition, understanding, search, ranking and delivery. Every component either fits in that window or runs somewhere it can't block it.
The naive design, asking a large model to pick a product from the transcript, spends most of that budget before any search happens, and its slow moments are unpredictable. Those are what agents remember.
Start before the sentence ends
Streaming speech recognition produces text while the customer is still speaking. That early text keeps changing, so we don't act on it blindly, but once it looks stable we start preparing: working out what to look for and warming up the search. If the sentence ends differently than expected, that work is thrown away and redone.
Much of the work is done by the time the customer stops talking.
A small gate decides whether to speak
Most of a sales call is not a shopping moment: greetings, identity checks, address readbacks, payment, small talk. An assistant that reacts to all of it gets ignored within a week.
So the first decision on every turn is not "what should we suggest" but "is this a moment worth suggesting anything at all." That decision is made by a small classifier trained on our own labeled call moments, not by a large language model. It's cheap enough to run on every turn of every call. Simple signals sit beside it: a customer reading out an item number is always worth answering.
This one model does two jobs: it filters noise, which protects agent trust, and it keeps the expensive machinery idle until it's needed.
A fast lane built on search, not generation
When something is worth answering, the obvious next step is to hand the problem to an LLM. Mostly, we don't.
- Spoken item numbers. If a caller reads out a product number, we look it up exactly in the catalog. No model, no fuzzy search. An unknown number is never guessed.
- What the customer already has. At the start of each call we load the customer's recent purchases. Customers often refer to something they own, so it's ready as a reference before anyone speaks.
- Search. Retrieval combines lexical search, which is good at exact names and numbers, with dense vector search, which is good at descriptions like "the green one with the little leaves." Results are filtered on what's actually sellable: in stock, sensible price.
- Complementary items. Matching and related products come from the catalog itself, anchored on what the customer owns or is talking about, not invented by a model.
Language models do have a job here, but a narrow one: helping turn messy speech into a good search, and double-checking that the top result fits what was asked. Neither writes the answer from its own knowledge. Every product shown is a real catalog item with real stock.
Models trained on what sells
Some decisions are better made by models trained on outcomes than by prompting. Where Wink picks an offer for a customer from their history alone, outside a live conversation, the choice comes from learned ranking models trained on what customers actually bought, with a few hard business rules on top, such as never re-offering something durable the customer already owns.
These models are fast, give the same answer for the same input, and improve from what really happens. A large model asked the same question is slower and has no memory of which suggestions worked last month. We apply the same thinking on the live call: what gets shown is tuned on whether agents pitched it and whether customers bought.
Two lanes that never wait for each other
There are two kinds of help on a call. Direct answers respond to something the customer just asked: "do you have it in blue." Proactive suggestions look for openings the customer didn't ask about: a matching item, a complementary product, something from their history.
Direct answers are the core of the product. So they run separately, with one hard rule: proactive work never delays a direct answer. When the two collide, the answer wins and the suggestion waits its turn.
The rule is simple, and it erodes quietly: one "quick check" added in the wrong place and every direct answer gets slower. Writing it down and testing for it keeps it true.
The slow lane is welcome, just not in the way
Some things do need a large model: untangling a rambling multi-part request, judging whether an unprompted suggestion fits the conversation, writing a natural one-line pitch the agent can say out loud.
All of these run, but none sit between the customer and the first card. The card goes out as soon as the fast lane has it, and the pitch line arrives a moment later on the card already on screen. Heavier, slower search only runs when the fast lane comes up empty. If the slow lane is slow on a given turn, nobody notices, because nothing was waiting on it.
The unglamorous speedups
Much of the latency we removed came from ordinary engineering.
- Precompute per call. The customer's profile and purchase history are fetched once at call start, not on every turn.
- Cache searches. Repeated queries within a short window, common when a customer circles back, are answered from cache.
- Warm everything. Models are loaded and run once, and network connections are opened, before a worker accepts its first call. Cold starts are where the worst outliers come from.
- Batch across calls. Small models handle requests from many calls at once rather than one at a time.
- Bound everything. Every external call has a timeout and a fallback. A slow dependency produces no card, not a late one.
Evaluating with real calls
Unit tests can't tell you whether a suggestion was good. To measure that, we replay recorded calls through the full pipeline, with real timing, and look at what the agent would have seen and when. Reviewers rate the suggestions without knowing which version produced them, and we track the time from the customer's words to a card on screen.
The replay set has to look like real traffic, with easy calls and hard ones mixed together. A set made mostly of easy cases will tell you everything is fine.
Learning from outcomes
Every suggestion is logged with what happened next: whether the agent pitched it and whether the customer bought it. Those outcomes feed back into the ranking and acceptance models, so the system improves on the catalog and customers it actually serves. They also show us mistakes, like a kind of suggestion agents always ignore.
What we'd tell another team
Decide what has to be fast and keep large models out of that path. Use small learned models for the frequent decisions: whether to speak, what to rank first. Let large models do what only they can, in parallel. Test with real calls, measure from the customer's last word, and remember that a fast wrong suggestion still costs you agent trust.
If you want the broader picture of this category, read what real-time agent assist is. For the sales side, see how to upsell on inbound calls. Wink AI is built by Ambi Labs in Boston; you can see how Wink works or read about the team.