I use it. I see tremendous value in it. I believe it is going to fundamentally change how we develop software, operate businesses, support customers, analyze information, and make decisions.
But there is an important difference between using AI as a tool and allowing AI to operate unchecked. That distinction becomes even more important as AI moves beyond simply answering questions and begins taking actions on our behalf.
AI Hallucinations Are Different When Actions Are Involved
We have all seen examples where an AI confidently provides an answer that simply isn’t correct. That is generally referred to as an AI hallucination.
When AI is being used as a research assistant or a thought partner, the consequences can often be managed. A person reviews the response, recognizes something doesn’t look right, verifies the information, and corrects it. That human review is an important part of the process.
But what happens when AI is no longer just recommending what should happen? What happens when it can actually do it?
Imagine an AI system that can modify a database, deploy code, change a firewall rule, disable a user account, send communications to customers, approve a transaction, or modify production infrastructure.
Now a hallucination isn’t simply a wrong answer on a screen. It can become a wrong action.
The Problem Gets Bigger as AI Gets More Access
There is an old principle in IT security called least privilege. You give a person or system only the access necessary to perform its job. There is a reason we have followed that principle for decades.
We learned, sometimes painfully, that convenience and security do not always point in the same direction. The same thinking needs to apply to AI.
If an AI agent has access to email, databases, source code, production environments, financial systems, customer information, and administrative tools, we should be asking some very traditional IT questions:
What does it actually need access to? What actions can it perform without approval? What requires human authorization? What gets logged? Can its actions be reversed? Who is accountable when something goes wrong?
These aren’t new questions. We have been asking them about users, administrators, applications, APIs, and service accounts for decades. AI shouldn’t suddenly be exempt from those same controls.
Human-in-the-Loop Should Not Be Considered a Weakness
There seems to be a race toward making AI completely autonomous. I understand why. The more AI can accomplish without human involvement, the greater the potential efficiency.
But efficiency should not automatically override control.
For high-impact decisions, having a human approve an AI-generated action may be exactly the right architecture. AI can analyze thousands of records. AI can identify patterns. AI can recommend an action. AI can prepare the change. And then a qualified person can review it and say, “Yes, execute.”
That extra step may add a few seconds or minutes to the process. It may also prevent hours, days, or millions of dollars of damage.
Trust, But Verify
That phrase existed long before artificial intelligence, but it may be one of the most important principles we carry forward into the AI era.
AI does not need to be perfect to be incredibly valuable. People aren’t perfect either. That is why mature organizations build controls around important processes.
We use peer reviews. We use change management. We separate development from production. We use approval workflows. We maintain audit logs. We test before deployment. We back things up. We limit administrative access.
AI should fit into those disciplines rather than bypass them.
The Future Should Be AI With Governance, Not AI Without Humans
I don’t believe the answer is slowing down AI adoption. I believe the answer is becoming smarter about how we adopt it.
AI should eliminate repetitive work, accelerate analysis, help developers build better software, help leaders make better-informed decisions, and allow organizations to accomplish things that previously required enormous amounts of time.
But capability and authority are two different things.
Just because an AI system can perform an action doesn’t mean it should always have the authority to perform that action without oversight.
The companies that ultimately get the most value from AI may not be the ones that simply give it the most access. They may be the ones that build the best guardrails around it.
Because the goal shouldn’t be to remove humans from every process. The goal should be to combine what AI does exceptionally well with the judgment, accountability, and experience that people bring to the table.
AI is an extraordinary tool. But like every powerful tool we have introduced into technology over the years, its value will ultimately depend on how responsibly we choose to use it.
Thanks,
Michael Cronin
Website: https://www.michaelcronin.info
LinkedIn: https://www.linkedin.com/in/michaeltcronin/details/experience/