Stop Using AI For This
Kyle Cook argues that developers are wrapping deterministic logic in English prompts and AI "skills" when they should just write a program. He uses Gary Tan's G stack skills as an example: a markdown-to-PDF skill that is essentially code and if-statements written out in prose, fed into a non-deterministic machine. His fix is to generate a CLI tool or small scripts once, let the AI call them, and isolate AI to the genuinely ambiguous step "- like transcription in his own video editor "- so one hallucination doesn't break an entire chain.
- Many AI "skills" are just programs in disguise: blocks of code plus if-statements written in English, which means deterministic logic is being run through a non-deterministic system that can hallucinate.
- A better pattern is to encapsulate that logic in a single command-line tool or several small scripts, and let the AI decide which tool to call rather than carrying out all the steps itself.
- Every invocation of a long skill costs tokens twice "- the AI must read all the text and then think and output "- whereas writing the code once and calling it is effectively free thereafter.
- Cook frames this as caching: we cache generated HTML to avoid regenerating it per request, yet we've forgotten the same principle with AI and burn tokens re-deriving the same result.
- In his own video editor, AI only produces a transcript written to a JSON file; the JavaScript program reads it, lets him edit, and calls FFmpeg with no AI involved.
- Chaining AI through multiple steps means a single mistake anywhere breaks the output and forces a rerun from the start; isolating the AI step means you only rerun that step.
- Using AI to write the deterministic code is encouraged "- what's wasteful is using English instructions to make AI perform the deterministic task itself.
- He acknowledges agents are the right call when the task is genuinely ambiguous, but not for spelled-out procedures like git commit and push.
This is the core argument, delivered with the concrete PDF-skill example in front of him, plus the proposed fix of a single CLI tool.
It sounds like a program to me. It sounds like code. It sounds like something that should run deterministically the exact same way every single time. And instead... we're feeding it into a non-deterministic machine that can sometimes hallucinate.
A sharp reframing that turns an abstract complaint about tokens into a familiar engineering principle.
We have learned that caching is an incredibly important thing to do as you want to scale out your application... We've essentially completely forgotten that when it comes to AI.
Do you know what that sounds like? It sounds like a program to me.
Instead of creating a program which does what we want us to do, the caching portion, we're instead forcing the AI to burn tokens and energy and power every time we want to do something.
I guarantee you at some point along that chain it's going to make a single mistake and if it does, everything in that chain is essentially broken.