Skip to content
Neural Daily

ResearchNews

ThinkingBox turns database checks into RL rewards for agents

Microsoft open-sourced ThinkingBox-Training. It grades an agent by the final database state and uses that same verdict as the reward for reinforcement learning.

By: Mikola Máté Patrik · · 2 min read

ThinkingBox Training ábra: feladatválasztás, adatbázis-ellenőrzés, jutalom és tanítás, majd kiértékelés a ThinkingBox-Bench adatkészleten
Image: Microsoft · Press kit · Hero figure from the official blog post · Source

What changed?

On October 22, Microsoft released ThinkingBox-Training in a joint post with Hugging Face (Hugging Face, GitHub). ThinkingBox started as a benchmark: 507 tasks, each with a real backend, MCP tools and a simulated LLM user. At the end, executable checks look at what actually changed in the database. That pass or fail verdict is now an RL reward on training tasks outside the benchmark.

The numbers

  • Model: Qwen3.8-27B, 50 GRPO steps.
  • pass@1 rose from 51.70% to 60.78%.
  • Tasks passed on all 20 attempts: 107, up from 38 (21.1% vs 7.5%).
  • pass@20 fell from 89.35% to 87.97%, so coverage narrowed slightly.
  • Estimated cost per dependable task: $8.00, down from $24.36.

Evaluation ran on the 507-task ThinkingBox-Bench, whose domains, databases and most MCP tools were not seen during training.

ThinkingBox-Bench diagram: estimated cost per successful attempt versus pass@1, the RL fine-tuned Qwen3.8-27B moves up and left
Microsoft · Press kit · Benchmark figure from the official blog post · Source

Source: Microsoft, Hugging Face blog (ThinkingBox-Bench, estimated cost per successful attempt)

How training tasks are picked

Most tasks are either already solved or hopeless for a given model, and neither teaches it anything. Microsoft runs the base model once before training, estimates per-task success and fixes the training pool from that. The post calls this offline policy-relative difficulty.

Why it matters

For agents that touch production database records, average success is not enough. What counts is whether they get it right every time. At an estimated $8.00 per dependable task, the fine-tuned open 27B model lands close to GPT-5.4 ($6.80) and Claude Opus 5.5 ($7.80). If you run your own backend with checkable state, the stack can be tried together with its OpenEnv environment.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles