Milliseconds to Match: Criteo's AdTech AI & the Future of Commerce w/ Diarmuid Gill & Liva Ralaivola
Summary
- Criteo’s core moat is the combination of fresh commerce data and millisecond inference, not a single secret model. Its network ingests product information from 17,000 retailers daily—sometimes several times daily—then matches cached user and product representations across millions of products while bidding billions of times. The governing engineering rule is the “best possible answer in the time constraints that you have.”
- The OpenAI partnership pairs broad LLM reasoning with inventory truth that changes too quickly for model weights. An LLM can behave like a “really kick-ass sales assistant,” but it may miss flash pricing, stockouts, or catalog changes; Criteo supplies real-time availability and pricing so recommendations remain actionable. MCP-style protocols could make Criteo’s existing tools portable across conversational surfaces without bespoke integrations.
- Criteo has turned its recommendation stack into a modular foundation-model platform. The company moved from handcrafted sparse vectors of roughly 2^12 to 2^20 dimensions and logistic regression to approximately 200–1,000 learned DeepKNN features; it is now building 3–4 foundation models for products, user timelines, and related domains. Shared embeddings give internal teams a “hot start” on acquisition, discovery, social, CTV, and LLM products.
- Performance comes from pushing almost everything offline while preserving the user’s latest action online. Models are trained and embeddings cached ahead of time, but smaller or adapted models must update the representation at runtime without breaking vector comparability. That architecture must remain accurate and reliable through Black Friday loads approaching 300% of normal—“the same car, same machinery” driven for a week at 300 miles an hour.
- Management treats privacy as the commercial condition for effective personalization, not merely a compliance cost. Criteo says it uses anonymous IDs rather than personally identifiable information, exposes AdChoices explanations and opt-outs, and deploys its European-compliant approach globally. “If it’s creepy, it won’t work”: lost trust reduces engagement, advertiser returns, and ultimately the ad funding that keeps content outside paywalls.
- Generative creative could expand the addressable advertising market by removing a major constraint on smaller advertisers. Criteo and Waymark are lowering the cost of campaign production, while Criteo’s existing dynamic-creative system can arrange pre-generated “Lego” assets at runtime. Fully generating each ad during page load remains too slow; Diarmuid and Liva expect audience-level variation and user-initiated personalization to create more utility than a unique ad for every individual.
- The longer-term scenario is not advertising’s disappearance but its convergence with agent-mediated search. A user might instruct an assistant to evaluate 100 shoes, surface six advertising choices, or show exactly five trips—turning exposure into something explicitly requested and bounded. Nathan Labenz called this “the collapse of search and advertising”: matching costs fall, but trusted access to current commerce data becomes more valuable.
Deep dive
1. Relevant advertising survives only as a transparent value exchange
Nathan opens with the ominous framing of what he believes was a Bernie Sanders conversation with Claude. Diarmuid’s corrective: Criteo says it collects no personal information, only a random anonymous ID plus signals such as products viewed, clicked, liked, rejected, or purchased. The industry’s obligation is to explain what is collected and why.
Diarmuid calls advertising “the lubricant that keeps the internet open and free.” Relevance makes an ad engaging rather than mentally filtered noise; advertiser spending then funds services and content that might otherwise move behind paywalls. Criteo’s AdChoices icon lets users see why an ad appeared and opt out.
Nathan adds the economic case: affordable commercial recommendation systems help long-tail businesses and niche products remain viable. Liva stresses that Criteo, as the intermediary, often has “the least data and the most challenging task,” using sparse cues to make consumers, merchants, and publishers better off.
2. An anonymous browser ID becomes a priced auction in milliseconds
When someone visits a retailer, a cookie can store a random ID on the device. Subsequent activity associates that ID with a product or broader segment: viewing a phone might indicate a tech enthusiast, while iPhone and Android interest can imply different profiles without revealing a person’s identity.
When another site offers an ad placement, multiple ad-tech providers bid as delegates for advertisers. Criteo estimates the opportunity’s value from perhaps 150 features—including viewed products, purchase history, site context, device, and audience membership—then predicts a click, site visit, and eventual purchase. That expected return determines the bid.
Liva reduces the intellectual core to binary classification: “Should we bid or should we not bid on that placement?” If Criteo wins, another decision selects which products to display. Offline training and extensive caching let inference, product selection, and bidding happen within the page-load window.
The sophistication creates a trade-off. Deep-learning models improve relevance and revenue prediction, but “the more sophisticated the models are, the less easy it is to understand what they have computed.” Criteo researches explainability, yet Liva is explicit that obtaining both maximum utility and transparent explanations remains difficult.
3. OpenAI supplies reasoning while Criteo supplies what is actually for sale
Nathan asks whether conversational systems create fundamentally richer user profiles. Liva’s answer is deliberately incomplete: the partnership is new, privacy and data exchange remain unresolved, and Criteo is still determining which conversational signals are useful. What is clear is that chat creates a new commerce surface.
Diarmuid separates world knowledge from commerce truth. An LLM’s knowledge reflects the point at which it was trained, but prices, promotions, and inventory quickly become stale—especially during Black Friday flash pricing or stockouts. Criteo ingests catalog data from 17,000 retailers daily, sometimes multiple times per day.
The proposed hybrid gives the LLM current product tools rather than asking its weights to memorize a changing catalog. A customer could receive the equivalent of a “really kick-ass sales assistant” that knows the full assortment, explains trade-offs, tolerates follow-up questions, and never gets bored—without recommending an unavailable item at the wrong price.
Diarmuid pushes back on Nathan’s suggestion that chat simply repeats open-web advertising with a richer query. The step-change is product discovery: sustained dialogue can uncover requirements and compare alternatives. Liva adds that MCP-style protocols make years of Criteo tools accessible to new agents without adapting each one separately to every website or surface.
4. Deep learning replaced the feature-engineering treadmill
Criteo’s legacy system encoded sites, products, and behavior in sparse vectors with roughly 2^12 to 2^20 possible inputs, depending on the encoding. Those representations supported fast sparse-matrix computation and logistic regression—effectively a linear model or “one neuron”—but humans had to invent and maintain the features.
The Criteo AI Lab was created in 2018 to make those representations learned rather than handcrafted. Liva says its proprietary DeepKNN approach now derives approximately 200–1,000 features from the data, with downstream models performing the bidding and classification tasks.
The constraint never changed: accuracy must coexist with milliseconds of latency, billions of daily decisions, and production-grade reliability.
Diarmuid’s Black Friday analogy captures the operational burden: take the same car, run it for a week at “300 miles an hour,” then return it to normal operation for the rest of the year. Peak commerce can drive systems toward 300% of ordinary load without allowing behavior or reliability to degrade.
5. Reusable embeddings turn commerce data into a companywide platform
Criteo is developing 3–4 foundation models rather than one monolith, covering domains such as products and user timelines. Each is intended to create semantically meaningful embeddings that internal teams can reuse as inputs, avoiding the need to relearn the structure of commerce for every new prediction task.
The geometry directly supports recommendations. Nearby product vectors suggest similar products; users closest to a product define a potential audience; and products near a user representation become recommendations. The representations need not be human-interpretable, but they must preserve enough signal for similarity to carry commercial meaning.
At a hackathon one month before the interview, teams used shared foundation-model embeddings as a “hot start” or “warm start.” The platform challenge is now operational: refreshing and versioning those vectors like software so a new model can break every dependent system.
Diarmuid connects that leverage to product expansion. Criteo began with lower-funnel retargeting but is moving into customer acquisition, product discovery, the open web, social, connected TV, and LLMs. Starting “already halfway there from a performance point of view” accelerates the path to advertiser-grade ROAS.
6. Precomputation wins the auction, but live context supplies the edge
Nathan recognizes an architecture resembling Stripe’s payment models: rich user and product encodings are computed in advance, leaving a cheap vector comparison at runtime. Liva largely agrees, but emphasizes the hard exception—the latest action may materially change intent and must update the embedding online.
That update requires smaller or adapted versions of the large models, fast enough for live serving but aligned closely enough that similarity calculations remain valid. Diarmuid frames the compromise plainly: unlimited time might produce a perfect answer, but real-time bidding needs the “best possible answer in the time constraints that you have.”
Criteo has also explored conversational context for 2–3 years. One employee built a Slack agent that recommended troubleshooting answers from internal messages; another experiment compressed a conversation into a summary vector and tested whether it improved a downstream model. Liva presents both as de-risking exercises, not settled production architecture.
7. Trust constrains agents and cross-device identity
The OpenAI integration is still being scoped. Diarmuid says an ad will not make sense in every conversation, and Criteo should receive “only the information we need” for a particular opportunity. Both companies, in his telling, center consent and privacy while they determine the appropriate exchange.
Liva’s trustworthy-machine-learning program covers hallucinations, privacy safety, and whether recommendation is appropriate for the person or context. He anticipates stronger external pressure, possibly regulatory, and says Criteo is preparing early—“we prefer to be on the safe side.”
Cross-device recognition follows the same boundary. A shared Wi-Fi network can misidentify household members, whereas authenticated use across devices can establish identity deterministically. Probabilistic matching is reserved for broad, untargeted uses; an opt-out must not be bypassed by rediscovering the person through another device.
Diarmuid’s commercial test is sharper than a compliance checklist: “If it’s creepy, it won’t work.” A trustworthy ad feels useful and earns a click; an intrusive one harms the user experience and destroys the advertiser value Criteo is paid to produce.
8. Europe is a design advantage, not an AI handicap
Nathan voices the common first-order claim that “you can’t do AI in Europe.” Diarmuid calls it “fundamentally wrong,” citing the French team’s quality as “off the charts.” Liva points to France’s formal training in mathematics, computer science, and engineering as precisely the foundation needed to model specialized AI problems.
Diarmuid traces that culture through Laplace, Lagrange, Fermat, and Galois, while noting present-day companies such as Mistral. The claim is not that Europe has no constraints; it is that rigor in abstraction and formalization supplies the raw material for innovation.
Being born in Europe made careful data handling a first principle. Criteo operates across the US, Canada, and Europe, but says it uses the same European-compliant approach globally rather than becoming looser elsewhere. Diarmuid sees California’s CCPA and CPRA and European rules as increasingly influenced by one another.
Nathan asks directly whether Europeans receive better protection or Americans receive better ads. Diarmuid does not choose a winner; he returns to consent, transparency, opt-outs, and an intact exchange among users, advertisers, and publishers. Remove advertiser returns, he argues, and free content must find another monetization model.
9. Open science works because the data, problems, and culture remain scarce
Nathan is struck that the AI Lab publicly lists roughly 50 people despite the era of Zuckerberg “blank checks.” Diarmuid’s retention thesis is cultural: talented colleagues, hard production problems, and long employee tenure create an environment people will travel and stay for, even when outside competition intensifies.
Liva says research scientists must publish reproducible work and maintain a public presence. Criteo also sponsors PhD students who work beside researchers on real projects, publish their results, and sometimes join full time, keeping the company connected to European academia.
The problems bridge theory and deployment: researchers can pursue upstream questions whose output reaches production in one or two years. Nathan’s suggested moat logic is that openness is affordable when the difficult-to-copy assets are the commerce network, proprietary data flow, operational system, and opportunity to test at scale.
10. Generative creative widens the market before it reaches runtime
Nathan identifies missing creative as a major barrier keeping smaller advertisers off platforms. Diarmuid sees generative AI democratizing high-quality production, with Waymark supporting Criteo advertisers and the self-service Creadits Go product aimed at making campaign creation accessible to the mid and long tail.
Criteo already assembles dynamic ads from templates and visual assets—Liva’s “Lego” analogy. Generative systems can create those pieces offline, while the existing engine arranges them during the auction. Waiting five seconds for a fresh image is untenable today; Liva speculates that full or device-assisted runtime generation might arrive in two or three years.
Nathan tests the extreme case: could every person receive a unique story? Diarmuid expects most value at the audience level, where brands retain control of look and feel; he sees little proof that individual-level variation would drive enough extra sales. Liva’s alternative is privacy-preserving personalization on the user’s own device.
User initiation changes the boundary. Asking an agent for wedding outfits in the South of France, a virtual try-on, or a preview inside one’s home makes hyper-personalization useful rather than creepy. Nathan raises the possible value of matching ad tone to context; Diarmuid is more cautious about contextual insertion, emphasizing that the ad must not intrude on the content.
11. Human control remains the invariant across models and media
Brand guidelines, liability, and consumer agency keep humans in the creative loop even as the technical possibility space expands. Liva says every model project should ask, “Where is the human?” Removing the person’s right to decide can make automation legally and practically untenable.
The same principle applies beyond creative—to bidding, agents, identity, and product recommendation. The difficult design choice is not whether to preserve human involvement, but where to place it so that automation creates leverage without leaving users unable to understand or override a consequential decision.
This also helps explain why conversational systems can feel acceptable despite receiving richer information: Nathan observes that people seem more comfortable with personalization in chat or agent contexts, and Diarmuid agrees when “the user’s asking for it.”
12. Advertising may become product discovery that people explicitly request
Nathan’s closing tension is economic: if AI raises prosperity, human time becomes more valuable and advertising inventory could appreciate; yet agents may also collapse search and matching costs by evaluating every shoe on a buyer’s behalf, reducing merchants’ need to bid for attention.
Diarmuid expects agents plus fresh commerce data to produce better decisions, not eliminate commercial discovery. A user can specify “I don’t care about the price; I just want the best product possible,” then receive an accurate recommendation, supporting products, and current availability with less guesswork than conventional advertising.
Liva imagines the value exchange becoming explicit: turn a knob telling an assistant to inspect 10, 12, or 100 shoes, select six of them to show as advertising, and let the human choose—or request exactly five trips, “not more, not less.” The intermediary filters abundance while preserving deliberate exposure.
Nathan calls the scenario “the collapse of search and advertising”: if advertising becomes sufficiently useful, it may outperform search. Diarmuid doubts the singularity framing, and Liva keeps the five-year outlook speculative, but their shared speculative endpoint is that ads could move behind the agent and reappear as bounded discovery the consumer asks to see.