How to Cut Your LLM Costs by 60% with Model Routing
Most engineering teams we talk to are routing every request to GPT-4 or Claude 3 Opus — even when the task is trivial. The result is a burn rate that’s 5-10x higher than it needs to be.
The insight: tasks vary in difficulty
A customer support chatbot, a code autocomplete, and a complex legal contract analysis are wildly different in capability requirements. Yet most teams use the same model for all three.
How model routing works
Westtree’s API gateway lets you route requests to different models based on the request type. Here’s a real example:
- Simple intent classification → Gemini 1.5 Flash ($0.075/M input)
- Multi-turn conversation → GPT-4o ($5/M input)
- Complex reasoning → Claude 3.5 Sonnet ($3/M input)
The math
Suppose your traffic is 70% simple, 20% medium, 10% hard. With naive routing (all GPT-4), you pay roughly $50/M tokens blended. With smart routing, you pay about $20/M — a 60% saving.
Implementation
Setting up routing in Westtree takes about 10 minutes. Define rules in your dashboard, and the gateway handles the rest automatically.