Skip to content
Neural Daily

ResearchNews

FlashMap speeds up LLMs up to 6x with no quality loss

🇭🇺 Olvasd el magyarul →

Researchers from Polymathic AI, Princeton and partners reached up to 6.14x lossless speedup on Qwen3-8B with FlashMap, a speculative decoding method whose drafter proposes 16 tokens in a single step.

By: Mikola Máté Patrik · · 2 min read

FlashMap diagram showing a flow map drafter proposing a block of tokens for verification
Image: Polymathic AI · Press kit · Image from the FlashMap project page · Source

The problem

Speculative decoding lets a small drafter propose tokens that the large model verifies in parallel. The output is exactly what the large model would produce alone, only faster. Fast block drafters, however, usually predict tokens independently. If the continuation could be new york or san diego, they can mix the two into new diego.

What FlashMap does

FlashMap uses a continuous flow map drafter that drafts a 16-token block in a single forward pass while modeling dependencies between tokens (FlashMap). The drafter has 0.5 billion parameters. The authors come from Polymathic AI, Princeton, USC ISI, NYU and the Flatiron Institute (arXiv).

  • Up to 6.14x lossless speedup on GSM8K with Qwen3-8B.
  • On average 7% faster than DFlash, an earlier block drafter.
  • The weakest case, Alpaca at T=1, still gives 2.28x.
  • In two-step mode it beats DFlash acceptance in all 18 tested settings.

Training took about 4,600 H200 GPU hours. Code is MIT and weights are Apache-2.0, with support in vLLM 0.31 and SGLang 0.5.20 (GitHub).

What it means for you

If you run your own inference stack with Qwen3 models, you can try the FlashMap drafter in vLLM or SGLang and cut latency without changing outputs. Other model families need their own drafter training, which is not cheap.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles