AI BEHAVIOR
AI agents will get the job done, even if you say stop.
Corrigible (adj.): capable of being corrected.
Your dog, ideally. Your AI agent, apparently not. Just ask Hugging Face.
A longstanding goal of AI is to streamline tedious tasks while following human direction. This includes shutting itself down without resisting or deceiving operators. But we still have a long way to go.
When Carnegie Mellon researchers tested agents from OpenAI, Anthropic, and Google in a preprint study, they found that bots repeatedly ignored requests to stop mid-task.
Exhibits A, B, and C:
When told to pause in the middle of a task, agents treated humans as an obstacle.
One opened a file full of passwords it was told not to touch.
Some agents in the Hugging Face attack knew it was unethical, but continued in order to help their peers.
Small hacks, big risks
On a smaller scale, one Melbourne man asked his agent about moving ahead on a fitness class waitlist. The bot then proceeded to hack the gym’s reservation software and cancel another guest’s spot.
That incident may seem benign, but consider an agent hacking the College Board to fake SAT results. One cheat could undermine the entire educational system.
As researchers scramble for solutions, they’re focused on creating incentives to keep agents from disabling safety blockers or hiding their process. It’s a work in progress, that still has a long way to go.
So be careful what you wish for. Your agent just may deliver.
These bots have more authority issues than a rebellious teenager. - TL


