How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models

David Abitbol runs AI operations at Revolut, a European digital bank with about 80 million customers. In under twelve minutes at AI Engineer New York he covers how Revolut moved most of its coding-agent traffic from frontier APIs to a self-hosted open model, and he brings numbers: engineer lead times from a 2,000-person sample, GPU count, latency targets, cost per million tokens, and the tuning changes that mattered. It’s one of the more concrete inference-economics talks this year.

Unlimited tokens moved lead time more than coding time

Revolut tracks two numbers for about 2,000 engineers: coding time (hours) and lead time (days from picking up a feature to merge-ready). The first AI rollout, Gemini with access to internal data, cut lead time by about 10% and coding time by about 20%, mostly by replacing Stack Overflow and documentation lookups.

Claude Code went out next, under strict per-engineer budgets because some people were spending thousands of dollars a month without a clear return, and Abitbol says it showed little measurable effect at first. What changed things was self-hosting open models from April 2026 (GLM 5.1 at the time), with no token limits at all. Coding and lead time then improved by roughly 60–70%.

The lead-time gain beat the coding-time gain, and he reads that as evidence that coding was never the main bottleneck. Once nobody is rationing tokens, engineers also point agents at email, Jira and Confluence. Revolut now treats open source as the default with predictable flat billing, and keeps frontier models (Codex, Claude Code, Gemini) for the tasks where they’re clearly better.

The numbers that keep engineers happy

Serving 2,000 engineers takes 56 NVIDIA B200s running GLM 5.3 (around 800B parameters) on Together AI dedicated inference.

Time to first token has to stay under 3 seconds. Abitbol’s evidence is his Slack: past about 5 seconds, complaints start arriving. Throughput matters less. Around 200 tokens/s is enough because people now run tasks in parallel and check back instead of watching thinking tokens stream, and engineers who work after 5pm, when the GPUs are idle, get around 800 tokens/s.

The blended cost is $0.30 per million tokens with cache hits free, and the invoice is about a tenth of what Revolut was paying Anthropic and OpenAI.

Tune before you scale

“If you just get GPUs and upload weights on a GPU and you start doing some inference, you’re going to get terrible performance.”

From April to August the GPU count stayed at 56. Load went from about 3 to over 7 requests per second (+76%), while time to first token fell from 14.26 s mid-migration to 1.68 s (−88%). His slide credits about 80% of the gain to serving parameters, not hardware. The levers he lists:

• Context length. “Long context is a scam”: 1M-token windows eat KV cache for little benefit, and 256k is what coding actually needs.
• Quantisation. On Blackwell, run FP4 (NVFP4 preferred). Going from FP8 to FP4 saves roughly 40% of the GPU footprint with, by his account, no accuracy loss.
• Tensor parallelism. GLM fits on TP4, but TP8 may work better because it leaves more KV cache.
• Cache hit rate. Coding workloads should sit above 90%, and sticky sessions help get there.
• Further options: speculative decoding, prefill/decode disaggregation, sequential prefill.

Production setup

GLM is text-only. Abitbol’s rule is to take the smarter text model every time and handle images separately: a local router sends image tasks to a vision-capable model while the main loop stays on GLM, and there’s a GLM vision variant for cases that need it (their UI designers use it to go from mockup to working interface). The endpoints are OpenAI-compatible with the Responses API, so they plug straight into OpenCode and Codex.

For the CIO in the room

On worries about Chinese open models, his answer is that Revolut downloads weights and runs them on US-based infrastructure, and no request goes to a Chinese endpoint. As an EU-regulated bank Revolut is under some of the strictest rules, and the approach that worked is classifying coding agents as productivity tools with a human in the loop, which don’t need the audit treatment of autonomous agents. Backing that classification requires:

• A sandbox that enforces human approval for operations you haven’t allowed (his example is allowing grep but not Gradle), stops insiders misusing it, and limits prompt-injection damage.
• Data privacy: zero data retention at the inference provider, and processing only in jurisdictions covered by data-equivalency frameworks (US, EU, UK, Israel).
• Strong OpenTelemetry logging so incidents can be investigated, with usage analytics as a side benefit.

Key takeaways

1. Removing token budgets mattered more than which model was in use, and lead time improved more than coding time because engineers started using agents for non-coding work.
2. At about 2,000 engineers, 56 B200s running an ~800B open model came to about $0.30 per million tokens, roughly a tenth of Revolut’s frontier API bill.
3. Time to first token under 3 seconds is the latency that matters for coding agents. Raw tokens per second matters less once people run tasks in parallel.
4. Serving configuration (context cap, FP4, tensor parallelism, cache hits) produced an 88% TTFT improvement at 76% more load with no new GPUs.
5. Very long context windows cost KV cache and don’t help much for coding, and capping at 256k is a reasonable default.
6. Pick the strongest text model and route images to another model rather than settling for a weaker multimodal one.
7. In a regulated setting, a sandbox that guarantees human approval is what lets coding agents be classed as productivity tools.

Source

• Talk: How Revolut Cut AI Coding Costs 90% With Self-Hosted Open Models (slides: “Shipping like a startup, regulated like a bank”)
• Speaker: David Abitbol, Ops & Infrastructure Leader, Revolut
• Event: AI Engineer, New York
• Length: 12 min
• Video: https://www.youtube.com/watch?v=Ok6QdZHCcI8