The five costs vendors leave off the invoice when you let AI run your whole workflow.
I run delivery teams during the day and a small AI lab at night, so I've watched a lot of these automated flows up close. Not the keynote version. The 2am version, where the run is still going, the token meter is climbing, and nobody in the room can say what the agent is actually doing right now.
The pitch is always the same. Hand the work to AI, walk away, come back to a finished result. The pitch is real for narrow, repeatable tasks. It falls apart the moment the work is open-ended, and the gap between those two cases is exactly what gets hidden. So here's the honest invoice, line by line, from someone who keeps paying it.
The five line items that never make it onto the slide. We'll take them one at a time.
An autonomous flow doesn't think in steps the way you do. It loops. It re-reads its own context, retries, second-guesses, explores branches you never asked for. Each loop spends tokens, and tokens are money.
The part that stings isn't the size of the bill. It's that a big bill buys you no guarantee. I've had runs spend the equivalent of a long work session and land on something I had to throw away. You paid for the thinking and still own the problem.
Before you automate anything open-ended, put a number on it. Cap the spend per run. Watch your first ten runs and write down what each one cost and whether it actually shipped. If most of the money goes to runs you discard, the flow isn't saving you anything, it's just moving the cost from your hours to your card.
When a person does a task, you can ask them where they are. An autonomous agent gives you a wall of output and a result, and the reasoning in between is mostly opaque. You see what it decided, not why.
That opacity is fine right up until the output is wrong. Then you're debugging a black box. You can't put a breakpoint on a hunch. Often the fastest fix is to scrap the whole run and start over with a tighter instruction, which means the "automation" just cost you a full redo.
You can ask a person where they're stuck. With an agent, you get the answer and have to reverse-engineer the path.
The control you give up here is the real cost, not the compute. A workflow you can't inspect mid-flight is a workflow you can't correct mid-flight. You either trust it blindly or you wait for the end and hope.
Ask an agent for a simple thing and it will often hand you a complicated one. A small script becomes a framework. A one-line fix becomes a refactor with three new abstractions you didn't ask for and now have to maintain.
This happens because the model has seen a lot of "thorough" code, and to it, thorough looks like more. It chases the look of complete. Left alone, it builds the cathedral when you needed a shed.
The cost lands later, when someone has to read, change, or trust that code. Over-engineered output is slower to review and easier to break. If you don't push back hard and ask for the simplest version, the flow quietly trades your future maintenance time for the appearance of effort today.
Every model makes things up. A fake API, a confident wrong number, a citation that doesn't exist. In a chat you catch it because you're reading every line. In an autonomous flow, that invented fact becomes an input to the next step, and the next, before you ever see it.
Here's the worst part. When it does happen, you often have no clean way to control it. You can't reliably tell the model "stop being wrong about this," because it doesn't know it's wrong. The error is laundered through ten confident steps and arrives looking like a finished result.
One made-up fact early in the flow doesn't stay one error. It becomes the ground every later step stands on.
This is why I trust an automated flow least on exactly the work that matters most: anything with real facts and real numbers behind it. The flow is most dangerous precisely where being wrong costs the most.
I use these tools every day and I'll say it plainly: full workflow automation is a research preview. Treating it as a finished product is the mistake. It's powerful in the hands of someone who knows when it's lying and how to box it in. It's a trap for anyone who takes the output at face value.
That's the line vendors blur. The demo implies "anyone can do this." The reality is closer to "an expert can supervise this." If you're not the expert yet, automating a flow end to end still needs the expertise. It just stays hidden until something breaks, and then it's the whole problem.
The popular fix for all of the above is the multi-agent council. Run several agents on the same flow, have them cross-check each other, and let the disagreement catch the errors. Grok ships a version of this. On paper it's a fact-checking committee.
I tried it. Here's the catch: the agents are usually the same underlying model. So when the model is wrong, every agent is wrong the same way. They agree on the mistake and hand it back to you with extra confidence. The committee can't catch the error because every member shares the blind spot.
Five reviewers, one brain. They don't cross-check the error, they co-sign it.
Real cross-checking needs independence. Different models, trained differently, failing differently, so one catches what another misses. And that's where it gets hard. Mixing models from different vendors inside one flow runs straight into security boundaries, data-sharing rules, latency, cost, and a pile of integration work most teams won't take on. The honest version of multi-agent is expensive and rare. The cheap version, same model wearing five hats, gives you the comfort of review with none of the protection.
Context windows are finite. Feed a model more than it can hold and it starts dropping or blurring the earlier material. On a small task you never hit the wall. On a large one, with many sessions stitched together, you hit it constantly, and that's where hallucination breeds. The model forgets a constraint from three steps back and confidently invents a replacement.
The bigger the project and the more sessions it spans, the more this compounds. No memory system on the market today fully solves it. There are clever tricks, retrieval, summaries, scratchpads, but none of them give the model a perfect, durable memory of everything that came before. So the failure rate doesn't stay flat as you scale. It climbs.
Each new session pushes older context toward the edge of the window. The error rate doesn't hold steady as you scale, it grows.
Put all of it together and you land where the experienced people already are. The most effective setup today is a human holding the reins and steering, with AI doing the heavy lifting inside boundaries the human sets and checks.
That's how you get the speed without inheriting the failure modes. The human scopes the task small enough to verify, reviews at checkpoints instead of only at the end, catches the hallucination before it propagates, and kills the over-engineering before it ships. AI moves fast. The human decides what's true and what counts as done.
The working pattern. AI does the work, the human owns the boundaries, the checkpoints, and the call on what's true.
Letting AI handle absolutely everything sounds efficient and reads well in a pitch deck. In practice it's how you ship confident nonsense at scale. The teams getting real value treat the model as a strong component inside a human-run process and never as the process itself.
Companies sell the convenience of automation and stay quiet on the rest. Nvidia and Microsoft announce an AI laptop with features that sound impossible, and the caveats live in the footnotes. The same shape shows up in the polished automated workflows the AI labs themselves demo. The capability is real. The framing leaves out the cost, the failure modes, and the expertise required to run it safely.
The end user walks away thinking the tool does more, and does it more reliably, than it actually does. That's not a small thing. It sets people up to hand real work to a system they don't understand, and to be surprised when it breaks in exactly the ways the people who built it could have warned them about.
If you're using or evaluating an automated AI flow, you don't need to abandon it. You need to run it like an adult.
The five-minute discipline that turns an automated flow from a gamble into a tool.
Cap the token spend per run and watch what your first runs actually cost versus what they ship. Scope every task small enough that you can verify the result by hand. Put review checkpoints in the middle of the flow, not just at the end, so you can correct it before a wrong step poisons the rest.
Treat every fact, number, and citation the flow produces as unverified until you've checked it yourself, especially on anything that carries real consequences. And when the output looks more complicated than your problem, send it back and ask for the simplest version that works.
None of that is glamorous. It's the unglamorous discipline that decides whether AI saves you time or quietly costs you more than it gives. The model is getting better fast. The judgment about when to trust it, and when to keep your hands on the reins, is still yours to supply.
1 to 2 emails a month. Easy to unsubscribe.
1 to 2 emails a month. Easy to unsubscribe.
Comments