The Intelligence Ladder – Reducing LLM costs without compromising on quality

Imagine processing 50,000 incoming documents a day – invoices, contracts, forms. Classifying them, extracting data, forwarding them. With GPT-5.4 or Claude Opus, you’d pay several thousand euros a month for this. With the right OpenWeight model, the cost remains in the low three-figure range – whilst maintaining the same quality. This article shows you how to get there – across three levels that we call the ‘intelligence ladder’.

Contents

1. The bill nobody wanted to see
2. The Thesis
3. The Intelligence Ladder
4. Level 1: Architecture
5. Level 2: Prompt
6. Level 3: Reliability
7. Conclusion

The bill nobody wanted to see

According to Menlo Ventures, corporate LLM spending has more than doubled in six months – from 3.5 to 8.4 billion US dollars.^[1] Whilst individual models are becoming ten times cheaper each year, the bills aren’t going down: teams are immediately channelling the savings into the next frontier model.^[1] The real problem is rarely token consumption. It’s the choice of model.

The thesis

The most expensive model is the one you call too often.

Back to your documents. The first instinct in any project: use the best model available to get the demo up and running. Understandable – for the demo. In production, with 50,000 requests a day, this adds up to a six-figure annual bill. And often, the quality actually deteriorates: oversized models take longer, generate more tokens and, for narrowly defined tasks, are less deterministic than a smaller, well-trained model.

The solution isn’t further abstraction, but an architectural decision: the right model for the right task, at the right time. To achieve this, you need a map of the terrain.

The Intelligence Ladder

The ladder has three levels – and most teams today optimise on only one of them, usually level 2.

You work from the top down: first the architecture, then the prompts for the chosen model, then reliability. If you work the other way round, you optimise locally but waste resources globally.

Level 1: Architecture

The most effective lever. Three tiers have established themselves on the market: Frontier models (GPT-5.4, Claude Opus, Gemini 2.5 Pro) with full reasoning capabilities and the highest price. Efficient tier (GPT-5.4 mini, Claude Sonnet and Haiku, Gemini Flash) – distilled versions of frontier models, positioned accordingly. SLMs and open-source models (GPT-5.4 nano, Llama, Qwen, gpt-oss) – the latter mostly licensed under Apache 2.0 or MIT, available via providers such as OpenRouter, Fireworks or Groq.

Prices as at April 2026, rounded.^[2]

For your documents, this means that extracting an invoice into 20 fields or sorting a contract into 15 categories is not an open-ended reasoning task. It is a well-defined extraction or classification task – and the output rate on GPT-5.4 nano is 20 times cheaper than on Claude Opus, and 12 times cheaper than on GPT-5.4. On an open-weight SLM, this cost could increase by two orders of magnitude. No reasoning uplift in the world justifies that.

This pattern isn’t limited to documents. It applies wherever high volume meets clearly defined decisions – routing support tickets, filtering log anomalies, normalising product data, checking compliance texts against rules. The specific figures may vary, but the logic remains the same.

A second architectural question: reasoning model or chain-of-thought prompt? A well-trained Efficient-tier model with a structured CoT outperforms a poorly prompted Frontier model with Extended Thinking in many cases – at a fraction of the cost.

Not every use case belongs in the lower tier. Where open reasoning is required without clear heuristics, where there are high correctness requirements with low volume, or where there is no training set for optimisation, the Frontier model remains the right choice. The best example is agentic coding with Claude Code or Codex: here, you need precisely the kind of open reasoning that the smaller models cannot provide – a cheaper model does not simply perform slightly worse, but becomes practically unusable.

And the balance shifts: for high-volume tasks, the cost savings from the model are multiplied by the number of requests. In coding, a developer might make 50 requests a day – the token price isn’t the dominant cost factor; developer time is. Here, a more expensive model is often the more economical choice.

Level 2: Prompt

Prompt engineering for small models is not the same as for large ones. GPT-5 .4 or Claude Opus are forgiving of vague phrasing, whereas GPT-5.4 nano or an Open-Weight SLM are not. A prompt that works 95 per cent of the time with GPT-5.4 drops to 60 per cent with GPT-5.4 nano. This is not an argument against the small model – it is an argument for better prompting. ‘Better’ does not mean ‘longer’, but rather: systematic, evaluated and optimised.

This is where frameworks such as DSPy (Stanford, Apache-2.0)^[3] come in. The core idea: prompts are not hand-written, but compiled. You define a signature, a metric and a small training set – around 500 manually annotated documents – and an optimiser automatically tries out hundreds of variants. A case study from 2025 documents how DSPy optimisation boosted GPT-4o-mini’s accuracy from 10.8 per cent to 47.6 per cent in a topic assignment task.^[4] The figures vary depending on the task, but the mechanism is robust: what previously required the Efficient Tier often runs on an Open-Weight SLM after optimisation – with the full cost difference.

When is it worth the effort? When three conditions are met: high volume, an objectively measurable success metric, and a stable, recurring task. For one-off applications, quality that is difficult to measure, or use cases without training data, traditional prompt engineering remains the more pragmatic approach.

Level 3: Reliability

Smaller models are cheaper, but also less reliable. Without reliability engineering, switching from GPT-5.4 to GPT-5.4 nano leads to more incidents and, ultimately, higher overall costs than before. Three patterns drive this decline:

Structured Output: JSON Mode, Tool Calling, Response Schemas. In your document example: invoice number, amount, date and supplier as a enforced JSON schema – no free-text responses, no ‘hallucinations’ about made-up fields.

Validation layer. Pydantic, guardrails, JSON Schema. Every output is checked against expected structures before it flows into the next system.

Retry with Escalation. If GPT-5.4 nano fails, the system automatically escalates to GPT-5.4 mini or Claude Haiku. If that fails too, it escalates to GPT-5.4 or Claude Opus. 95 per cent of documents are charged at the lower rate, whilst the remaining 5 per cent benefit from the robustness of the more expensive option – and your bill will still be in the low double figures compared to before.

Without these patterns, downgrading is risky. With them, it becomes the default.

Conclusion

The Intelligence Ladder is neither a framework nor a tool – it is a way of thinking. If you work through the three levels for your use cases, you will find double-digit potential for cost savings in most architectures without any loss of quality.

If you want to know where the greatest potential lies in your architecture, a structured approach will help: first, take stock of your current LLM calls – which model is used for which task, and at what volume? Then assess them across the three levels – where is the discrepancy between the requirement and the chosen model greatest? And finally, run a pilot on the use case with the greatest leverage, which will serve as a template for others.

This is precisely the support we offer at iteratec – designing, evaluating and building LLM architectures to production standards.

Do you have any questions or need assistance?

You can find out more about the possibilities for IT security and resilience for your business on our website. Get in touch with us now.

Sources:

[1] Menlo Ventures, 2025 Mid-Year LLM Market Update (July 2025). https://menlovc.com/perspective/2025-mid-year-llm-market-update/

[2] Prices according to the providers’ official pricing pages (Anthropic, OpenAI, Google) or via OpenRouter for open-weight models. Open-weight prices vary depending on the provider (Fireworks, Together.ai, Groq, etc.) and batch or caching discounts. As at April 2026.

[3] Khattab et al., DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines, Stanford NLP, 2023. https://github.com/stanfordnlp/dspy

[4] Is It Time To Treat Prompts As Code? A Multi-Use Case Study For Prompt Optimisation Using DSPy, arXiv:2507.03620 (2025). https://arxiv.org/abs/2507.03620