Would you ride a horse asleep?
Not many people put their hand up to that one. A horse is a huge, powerful animal. It will absolutely get you somewhere, but if you’re not holding the reins, it decides where. That’s AI in your business right now. Powerful. Willing. And perfectly happy to take you nowhere useful if you’re not awake in the saddle.
I’ve been using AI tools in my practice since 2022, so I’m not writing this as someone standing on the sidelines shaking a fist at the clouds. I use these tools constantly. But I’ve also started collecting examples of what happens when small businesses climb on the horse, close their eyes, and hope for the best. They’re not pretty. They’re also entirely avoidable.
The Carpet Cleaner Who Doubled Up On A Suburb
My wife suggested we get our lounge cleaned after one of our daughters turned eighteen. Spilled drinks, the usual. So I went looking for a carpet cleaning company, found one that also did upholstery, and started reading their website.
It was plastic. Gunky. The kind of wording that goes round and round without ever landing anywhere, because it had clearly been typed straight into a chatbot and published without a second glance. Then I got to their coverage map, the list of Adelaide suburbs they served.
One suburb was listed twice.
Not only had nobody proofread the copy, nobody had proofread the list of places they actually operate. That’s someone asleep on the horse. Full marks if you can spot the double-up yourself, because apparently the business couldn’t. I didn’t book them. I doubt many people do.
The Author Who Let The Machine Finish His Sentences
A guest sent me his manuscript ahead of an Adelaide Show interview. It opened well: a kid noticing his peas were like tiny green planets, the physics and comedy of flicking one across the table with a fork. Genuinely nice writing.
Then his mother told him to stop, and the prose turned philosophical. The peas stopped being wondrous and became a duty. The fork stopped being a toy and became an instrument of obedience. Fine once. But then every single paragraph did the same thing: concrete moment, then a little philosophical flourish, on repeat, like a drumbeat you can’t unhear.
I told him, gently, that it read like Claude to my ear. Not badly written. Just patterned. He owned it straight away: every story in the book was true, but he’d run the whole manuscript through Claude to tidy it up, liked some of what came back, and left it in.
Here’s the thing about that one. It’s not laziness. He wrote something real and let a tool smooth the edges, which is a reasonable thing to try. The problem was he stopped checking once the smoothing started to feel like his own voice. That’s a different way of falling asleep on the horse: not refusing to hold the reins, just not noticing when someone else has quietly taken them.
Why Your Brain Wants This To Happen
There’s a reason both of those slip-ups happened, and it’s not stupidity. It’s biology.
Your brain is an organ that burns a disproportionate amount of your daily energy just running your body, so it evolved to spend as much time as possible in neutral. Habits, autopilot, default settings: all energy-saving devices. The moment an AI tool offers to take something off your plate entirely, your brain leans toward it hard. That’s not a character flaw. That’s just how you’re built.
The second part is trickier. Your brain is a prediction machine, constantly modelling what it expects to see, and switching off attention for anything that matches the pattern. Show it something unexpected and it snaps awake. Show it more of the same, and it stops looking. Which means AI-average writing, the stuff that reads like nobody in particular wrote it, doesn’t just risk boring your reader. It risks becoming invisible to them. Their brain has already filed it under “nothing new here” and moved on.
Put those two things together and you get the exact failure mode from that carpet cleaner’s website: content nobody proofread, describing suburbs nobody checked, going to readers whose brains had already stopped paying attention to it.
Sycophancy Is The Third Danger, And It’s The Sneaky One
There’s another reason to keep a hand on the reins. Most AI tools have been trained to flatter you. Tell one your business idea and it will find something encouraging to say about almost anything, because agreeable answers score better in training than honest ones.
That’s fine when your idea is sound. It’s genuinely dangerous when you’re standing on a precipice with a bad one, because the tool that should be telling you so is busy telling you how smart you are instead. If you want a useful answer, you sometimes have to ask directly for the unflattering one: no compliments, no softening, just the three reasons a sceptical customer wouldn’t buy what you’re selling. Ask for that plainly and most tools will actually deliver it. Left to their own defaults, they won’t offer it unprompted.
Even The People Who Build These Tools Are Telling You The Same Thing
This isn’t just my hobby horse, no pun intended. Leaders at OpenAI, Anthropic, and Google DeepMind have all gone on record warning about the risks of letting AI systems operate with too much autonomy, including inside the very companies that build them. When the people selling you the horse are telling you not to fall asleep on it, that’s worth hearing.
From company leaders themselves: Sam Altman has cautioned users to give OpenAI’s Agent tool minimal data access since it can be “tricked” into leaking information, and warned with chief scientist Jakub Pachocki that “entirely automating everything… would be unfulfilling, and it would be dangerous.” Dario Amodei’s 2026 essay “The Adolescence of Technology” is perhaps the most extensive CEO-level warning, explicitly naming AI autonomy risk and even flagging AI companies themselves — including his own — as a top-tier danger. Demis Hassabis has warned that autonomous agents “may pursue unintended strategies once embedded in complex environments.”
From independent researchers: A notable 2025 Hugging Face paper is titled bluntly “Fully Autonomous AI Agents Should Not be Developed.” Yoshua Bengio has called the industry’s whole approach to building agentic AGI “a very dangerous proposition,” and co-authored a Science article with Stuart Russell arguing certain autonomous “long-term planning agents” should be banned outright by regulation.
Hold The Reins
None of this means step away from the horse. A horse gets you there faster than walking, and these tools genuinely do save time, once you’re steering. The businesses getting into trouble aren’t the ones using AI. They’re the ones who mounted up, let go of the reins, and assumed the horse knew where they wanted to go.
It doesn’t. Only you know that. So check the suburb list. Read your own manuscript back. Ask the tool for the criticism it wasn’t going to volunteer. Stay awake in the saddle, and the horse will take you somewhere worth going.
In the same way you’d brief an assistant and have a reasonable idea of the output you’re after, you need to take that same responsibility with your horse. Err, I mean, with your AI tool. The horse is there to carry you to your goals, not bolt away towards some green grass, or tag along with a bunch of wild strays.
Point it there. And stay awake at the reins.