Nate Jones reports back from Dreamforce, where he asked Marc Benioff about AI adoption and then joined a panel on the cost of running agents at scale. His argument: agents got good enough that people use them for bigger jobs, which multiplies token spend (more runs times far more tokens per run), so the fix isn't just buying a cheaper model "" it's going back to a blank sheet and deleting handoffs that only existed because old systems couldn't talk to each other. He then layers on model routing via classifiers, matching harness thickness to model capability, and evals as the way to know any of it actually worked.
- Benioff's answer on adoption that stuck with Nate was about interface: putting agents where people already work (e.g., Slack) removes the need for reluctant users to go discover a new product, and one small kept promise leads to repeat use and team-wide spread.
- Since roughly January, models became reliable enough at tool use that a single request now triggers many model calls; combine 10x more runs with 100x more tokens per run and you're at 1000x consumption without anyone doing anything strange.
- With frontier models charging around $50 per million output tokens, routing every ordinary decision to the most expensive intelligence is a design decision made by whoever built the harness "" not the fault of "greedy" employees using too many tokens.
- The interoffice-envelope metaphor: dropping an agent at every desk of a legacy process makes each stop faster but still passes the same envelope around the building; the real win is asking why the envelope has to visit those desks at all, since zero tokens is the cheapest cost.
- Business owners, not just engineers, have to be in agent design, because an engineer can make a step cheaper but usually lacks the authority to declare that sales ops no longer needs a document.
- Work sits on a continuum from deterministic to interpretive, so let ordinary software (a pricing tool, the CRM) handle settled rules and use a classifier to send standard requests to a cheaper open-weights model while reserving frontier models for genuine edge cases.
- Harness thickness should match the model: a smaller model needs heavy structure, tools and checks to be predictable, while a frontier model doing investigative work needs a thin harness so you don't waste the IQ points you're paying for.
- Evals are the missing piece and a human skill "" you can't tell a redesigned process works because the agent sounds pleased with itself; someone who knows the work has to define what a correct, not merely technically valid, result looks like.
The clearest articulation of the video's central argument "" automating a legacy process is not the same as redesigning it.
You still may be passing the same envelope around the building. You're just doing it with agents. You've made each stop faster without asking why the envelope has to visit all of those desks along the way.
A concrete, counterintuitive design rule most teams miss when they swap in a cheaper model but leave the scaffolding unchanged.
If I force the model to follow every little reasoning step I wrote for a weaker model, I get in the way of the IQ points I'm paying for.
You can't hand that giant design to a person and then act surprised when they adopt AI and it suddenly costs you the company money.
If you can solve a millennium prize problem like Navier-Stokes, you are well past the intelligence level you need to update your CRM.
Why would we give an agent real responsibility in the company and never evaluate it?