Stop Policing AI’s Thoughts. Start Containing Its Fires.
Inside a conversation with Aleks Jakulin on why AI governance is broken—and what comes next.
In the early days of the internet, we worried about access.
Then we worried about scale.
Now, with artificial intelligence everywhere, we’re worrying about control.
And according to Aleks Jakulin, we’re worrying about the wrong thing.
“We’re trying to police thoughts,” he says. “But AI outputs are just thoughts. The real issue is action.”
Jakulin isn’t a typical AI commentator. He helped create the image formats—PNG and JPEG-LS—that quietly power the modern web. He contributed early work validating deep learning. He’s built companies, sold them, taught at Columbia, and now runs Data.Flowers, a startup focused on something most AI companies ignore: data infrastructure.
So when he says today’s AI governance is misguided, it’s not a hot take. It’s a systems critique.
The Thought Police Problem
Right now, much of AI safety focuses on outputs—what models say, whether they hallucinate, whether they generate harmful content.
Jakulin thinks that’s fundamentally backwards.
“It’s like having someone inside your head checking every thought,” he says. “That’s the risk—not the thoughts themselves.”
Humans generate bad ideas constantly. Society doesn’t regulate thoughts—it regulates actions. But in AI, we’ve flipped that model: aggressively filtering outputs while barely addressing what happens when AI systems act in the real world.
Meanwhile, autonomous agents are already entering systems designed for humans—financial systems, hiring pipelines, communication channels.
And those systems are breaking.
AI Is Fire, Not Software
Jakulin’s central metaphor is deceptively simple:
“AI is like fire.”
We don’t regulate fire itself. We regulate its effects:
Fire codes
Fire extinguishers
Fire departments
We assume fires will happen—and design systems to contain them.
AI, he argues, should be treated the same way.
Instead, we’re acting as if we can prevent “bad AI thoughts” from ever emerging. Meanwhile, we’re not building:
rollback mechanisms
audit trails
accountability loops
resilient institutions
“We pretend we can control fire,” he says. “But we don’t even have fire departments.”
The Real Risk: Fragile Systems
The biggest danger isn’t AI going rogue. It’s AI breaking systems that were never designed for it.
Consider:
Hiring systems flooded with AI-generated applications
Financial systems overwhelmed by synthetic identities
Customer service channels flooded by bots
Email and authentication systems losing meaning
These systems worked because they assumed human effort as a constraint. AI removes that constraint.
“All systems that depend on effort—writing emails, filling forms—are going to fail,” Jakulin says.
This isn’t a future risk. It’s already happening.
Accountability Is the Missing Layer
If there’s one idea Jakulin returns to repeatedly, it’s accountability.
Not abstract ethics. Not vague transparency.
Concrete, enforceable accountability.
“We don’t hold people accountable enough,” he says. “So they don’t ensure their tools are accountable either.”
His model is closer to aviation than software:
black boxes
incident reporting
reproducibility
traceability
In aviation, failures are expected—but deeply analyzed. In AI, failures are frequent—but often invisible.
He points to emerging efforts like OECD’s AI incident tracking as a start, but far from sufficient. What’s missing is infrastructure that makes accountability unavoidable.
Garbage In, Garbage Out—Still
While policymakers argue about model outputs, Jakulin focuses on something simpler: inputs.
Training data. Context data. Retrieval pipelines.
“If you want food safety, you inspect the ingredients—not the cooking.”
AI models trained on messy, uncurated, or unauthorized data will produce messy outputs. Systems using retrieval (RAG) without governance will amplify the problem.
Yet most regulation ignores this layer.
This is where his company, Data.Flowers, positions itself: not as another AI app, but as a “reality layer” for data—structured, governed, auditable.
The Oligopoly Illusion
There’s a widespread belief that AI will concentrate power in a handful of companies.
Jakulin disagrees.
“AI is a decentralizing technology.”
Why?
Models are getting cheaper
Open-source capabilities are catching up quickly
Individuals can run powerful systems locally
The real bottleneck isn’t compute or models. It’s infrastructure.
“I can build an AI app in five minutes,” he says. “But I don’t have the rails it can run on.”
Those rails—data governance, provenance, permissions—don’t exist yet at scale.
And whoever builds them will shape the next decade.
Ethics Is Not the First Line of Defense
Jakulin is blunt about the current AI ethics landscape.
Too much focus on ideology.
Too little focus on mechanism design.
“Ethics is the instrument of last resort.”
Instead, systems should be designed so that:
bad actions don’t pay off
good behavior is incentivized
failures are contained
This is game theory, not philosophy.
It’s also how most robust systems—from markets to infrastructure—actually work.
What Success Looks Like
So what does “good” AI governance look like in 10 years?
Not perfect models.
Not zero risk.
Something else entirely:
Systems that fail gracefully
Actions that are traceable and reversible
Institutions resilient to automation
A society that becomes more intelligent, not less
“The goal,” Jakulin says, “is human flourishing.”
That includes something unexpected: faster human development. He predicts AI could compress education timelines, enabling people to reach full capability years earlier.
The Quiet Shift: From Text to Vision
One of the most overlooked transformations, he says, is how we communicate.
We’re still interacting with AI mostly through text—like it’s 1960.
But that’s about to change.
“Our visual cortex is our mind’s GPU.”
AI-generated diagrams, maps, and infographics could dramatically increase how quickly humans understand complex ideas.
Less text. More structure.
Less explanation. More insight.
The Bottom Line
The dominant narrative around AI governance is about control:
controlling models
controlling outputs
controlling access
Jakulin’s argument flips that:
You can’t control AI.
You can only control the systems around it.
And right now, those systems are brittle.
The next phase of AI won’t be defined by better models.
It will be defined by better infrastructure.
Or, as he puts it:
“We don’t need to stop the fire. We need to build a world that doesn’t burn down when it starts.”
This story was generated automatically from a podcast with Alec Crawford from Artificial Intelligence Risk, Inc. (click through to view):
