The pitch from a year ago was simple: bigger models, better answers. In practice, procurement teams found the bills first and the benefits later. A wave of fine-tuned small models is now handling ticket triage, document extraction and internal search for a fraction of the cost.
Why the economics flipped
Inference, not training, dominates the budget once a product ships. A compact model served on owned hardware removes per-token pricing entirely, and latency drops below what most users notice. The trade-off is scope: these systems are good at one job, not every job.
Key takeaways
- Fine-tuned 7B models now match larger APIs on narrow tasks
- Hosting on a single GPU removes per-token pricing
- Teams keep a frontier model only for the hard 5% of requests
This story is part of our ongoing coverage of ai. We will update it as more details emerge.
