Skip to content
Neural Daily

BusinessNews

How ChatGPT's ad stack works: an LLM orchestrator

OpenAI engineers explained how ChatGPT's ad system works. At its core is a post-trained LLM that drives retrieval, ranking and the auction through tool calls.

By: Mikola Máté Patrik · · 2 min read

OpenAI cover image: How we built ChatGPT's novel ad stack, with chat and user icons feeding a network icon
Image: OpenAI · Press kit · Official article social image (og:image) · Source

What changed?

On October 21, OpenAI published an engineering post (blog post) on ChatGPT's ad stack. Traditional ad systems react to a single search or page view. In a conversation, intent develops across many turns, so OpenAI built its own system from scratch.

The core is a post-trained frontier LLM the team calls the orchestrator. It runs in an agentic loop, making tool calls to each part of the ad system: candidate retrieval, filtering, relevance and engagement scoring, and finally the auction. The LLM does not set prices or pick the winner; it nominates candidates for the auction tool.

Three key design decisions

  • Stateful ad sessions: the system remembers context across turns and across conversations instead of treating each message as a standalone request.
  • The orchestrator writes short summaries and multiple queries from the conversation, turns them into embeddings and retrieves candidates with approximate nearest neighbor (ANN) search across several indices.
  • Ad selection runs in a separate service, with separate models and code, alongside answer generation. If the ad system is slow or down, ChatGPT's response is not delayed.

If an ad is not ready before the answer finishes, it is simply not shown for that turn. OpenAI says returning no ad is an ordinary, successful outcome.

Why it matters

For developers this is one of the most detailed public accounts of using an LLM as the controller of a high-traffic production decision flow while keeping critical steps (pricing, auction, policy) in deterministic tools. The pattern transfers: the LLM interprets and orchestrates, a separate tool makes the financial or regulated decision.

The evaluation loop is notable too: the strongest models judge the system's output and that feedback trains smaller, cheaper orchestration and retrieval models. OpenAI says the system was built with Codex, and ChatGPT Ads reached a $1 billion annualized revenue run rate less than 200 days after launch, with more than 100,000 advertisers.

What to watch

The post gives no latency figures, model sizes or exact timeout values, so performance claims rest on OpenAI's own statements. If you design agentic systems, the optional side path in its own service and the idea that no result is a valid result are two easy ideas to borrow.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles