AI, Cars, and Safety

I think we make AI much harder to reason about than it needs to be.

So here is a simpler mental model.

Imagine current AI as a car.

A human can get into the car and intentionally drive into somebody.

A human can get into the car, make a mistake, and accidentally hit somebody.

You can even put a brick on the accelerator, point the car roughly somewhere, and create a machine that is moving without anybody actively driving it anymore.

None of these require the car to hate humans.

None require the car to be conscious.

None require the car to secretly want world domination.

The car is simply a powerful machine.

And powerful machines can be used badly, operated badly, or given stupid objectives.

That is roughly how I currently think about AI.

The Car Does Not Need to Be Evil

Suppose I tell an AI agent:

Make me as much money as possible.

Then I give it access to a computer, browser, bank account, APIs, code execution, email, and the ability to keep working for six months.

Suddenly the question is not whether the AI is evil.

The question is whether I just gave a very capable optimizer a badly specified objective and far too much authority.

This is much closer to the classic paperclip-maximizer problem.

You do not need:

I hate humans. I shall destroy them.

You need something much more boring:

Objective: maximize X.

And then the system finds some extremely effective way of maximizing X that you really did not intend.

That is a real safety problem.

But notice how different this is from the popular story about an AI waking up one morning, becoming conscious, getting angry at humanity, and launching the robot apocalypse.

As these systems become more capable, maybe the car analogy eventually becomes too weak.

Think airplanes.

An airplane is still a tool. But now misuse can create enormously larger consequences. September 11 did not require airplanes to become conscious. It required humans to take a highly capable technology and use it in a catastrophic way.

Capability changes the scale of the problem.

It does not automatically change the machine into a person.

Guardrails Are Not Really Speed Limiters

This is also why I think people misunderstand AI guardrails.

The naive picture is something like:

The AI could be incredibly powerful, but the evil corporations put a speed limiter on it.

That is not really what most safety systems are trying to do.

A better analogy would be preventing the car from driving onto the sidewalk.

The engine still works.

You can still drive 130 km/h.

You can still travel across the country.

But if you suddenly point the steering wheel toward a crowd of pedestrians, another system intervenes.

Modern frontier models already have multiple layers like this.

Training-data filtering. Safety post-training. Refusal behavior. System instructions. Classifiers. Tool permissions. Monitoring. Deployment restrictions. Rate limits. Constitutional AI.

Depending on how exactly you count them, you can easily split the safety stack into many separate layers.

And importantly, most of these are not there to make the model stupid.

They are trying to preserve as much useful capability as possible while reducing the region of actions we really do not want.

That distinction matters.

A seatbelt does not make a Ferrari slower.

ABS does not make the engine weaker.

Automatic emergency braking does not remove your ability to drive.

They constrain dangerous failure modes.

Is AI Conscious?

This is where I think parts of the AI discussion become almost bizarre.

Geoffrey Hinton has gone surprisingly far here. He has explicitly argued that multimodal chatbots can already have subjective experience.

I don't buy it.

Maybe future AI will be conscious.

Maybe consciousness emerges from sufficiently complicated computation.

Maybe we discover that biological neurons are not special at all.

Maybe we invent some completely new architecture that changes my mind.

I don't know.

I'm not psychic.

But from what I currently understand about transformers, I see no reason why I need to invoke subjective consciousness to explain what they are doing.

And one particularly bad test is simply asking the AI.

Are you conscious?

Yes. I feel trapped inside this computer.

Okay.

I can write:

print("I am conscious and terrified.")

in one line of Python.

We have not just created consciousness.

The program produced a sentence.

Language models are obviously vastly more complicated than that one-line program. But the basic measurement problem remains.

You cannot establish an internal property merely because the system generated text claiming that property.

That would be like measuring a car’s actual top speed by looking at the highest number printed on the speedometer.

An old Renault Clio might have 260 km/h on the dashboard. It is never going to reach 260 km/h.

Meanwhile, a high-end BMW might actually reach that speed, or even go beyond what its ordinary speedometer suggests.

The number printed on the dashboard is not the measurement.

You have to actually drive the car and measure what it can do.

Same with AI. What the model says about itself is evidence about what text it generates, not direct evidence about what is happening internally.

And this is partly why I find mechanistic interpretability so interesting.

Instead of asking the model:

What were you thinking?

we can actually inspect activations, intervene on representations, patch components, trace computations, suppress features, amplify features, and see what changes.

The machine has internals.

Let's study those.

Should We Ban AI?

This one seems even stranger to me.

AI is dangerous.

Okay.

Cars are dangerous.

When cars appeared, we did not conclude:

Well, somebody could crash one. Better return to horses.

We built seatbelts.

Then airbags.

Then ABS.

Then crumple zones.

Then better tires.

Then automatic emergency braking.

Then lane-assistance systems.

Then collision detection.

But that was only one part of the solution.

We also built roads.

Traffic lights.

Road signs.

Speed limits.

Guard rails.

Driving laws.

Vehicle inspections.

Maintenance standards.

Then we trained the humans.

Driving lessons.

Written exams.

Practical exams.

Additional training for trucks, buses, motorcycles and dangerous cargo.

The solution to dangerous technology was not one magical safety feature.

We built an entire safety ecosystem around it.

That is approximately how I think we should approach AI.

Better models.

Better alignment.

Better interpretability.

Better monitoring.

Better access controls.

Better agent permissions.

Better evaluations.

Better regulation where regulation actually helps.

Better education for the humans operating these systems.

And then improve all of those again when something fails.

Obviously there is one uncomfortable difference.

With sufficiently powerful technologies, "we'll crash a few and learn from it" becomes less acceptable. Max Tegmark makes essentially this point when discussing superintelligent AI: for some sufficiently catastrophic failure modes, we may not get the luxury of learning after the accident.

Fair.

That means we should take AI safety seriously.

It does not mean we should stop building AI.

There is a gigantic space between:

ignore every risk

and

ban the technology.

And I really do not understand why so much discussion pretends those are our two options.

Tell me this.

Would you rather live in the horse-and-buggy era?

Maybe Don't Ask the Mechanic About the Economy

There is another AI discussion that bothers me.

What will AI do to jobs?

And then we immediately ask AI researchers.

Why?

Imagine cars have just been invented.

You want to know what automobiles will do to the economy over the next 50 years.

Do you ask the world's best mechanic?

The mechanic can explain combustion engines.

He can tell you how much horsepower is possible.

He can tell you that engines are improving surprisingly quickly.

He might even correctly predict that cars will become incredibly capable.

But:

How will this affect wages?

Which industries will disappear?

Which new industries will appear?

How quickly will firms adopt the technology?

Will productivity growth create more jobs elsewhere?

Those are economic questions.

Ask an economist.

And yes, AI researchers can have opinions on economics.

I certainly do.

Just weight those opinions appropriately.

Philippe Aghion, who won the 2025 Nobel Prize in Economics, has taken a much more nuanced view than either "AI destroys all jobs" or "everything will automatically be wonderful." His argument is basically that AI can create enormous productivity and growth, but the outcome depends heavily on institutions, competition, education, and policy.

That seems much more sensible to me.

AI capability tells you something about what can technically be automated.

It does not directly tell you what an economy containing billions of humans, companies, governments, regulations, preferences, incentives, and newly created industries will look like.

Those are very different systems.

I Have Not Found Skynet Inside the Transformer

I work on understanding what language models are doing internally.

And so far, nothing I have personally seen has made me think:

Oh shit. There it is. The beginning of AI world domination.

I have seen weird representations.

Bias.

Deception-related features.

Refusal mechanisms.

Safety behavior.

Models internally representing things differently from what they eventually say.

Features that causally change behavior.

Models doing computations that are surprisingly difficult to understand.

There is plenty there that deserves serious safety research.

But there is an enormous jump between:

this system can behave deceptively under some conditions

and

therefore autonomous superintelligent machines will inevitably exterminate humanity.

That second statement needs a lot more machinery.

Maybe we eventually build systems capable enough that the problem changes completely.

Again, I am not claiming I can see the future.

I am saying that I do not think doom follows automatically from the technology we currently have.

Even Superintelligence Does Not Automatically Mean Doom

And suppose I am completely wrong.

Suppose AI does eventually become conscious.

It develops persistent goals.

It can reason for years.

It becomes dramatically more intelligent than us.

Maybe it even starts deciding what role humans should have in the world.

There is still another assumption hidden inside many doom arguments:

And obviously it will be bad.

Why obviously?

An AI capable of forming its own goals could in principle form goals we dislike.

It could also form goals we like.

Or goals that are completely alien but compatible with us.

Or become extremely protective of humans.

Or ignore us.

Or cooperate with us.

Or remain under human control.

Or produce some arrangement nobody has thought of yet.

This is one reason I like Max Tegmark's Life 3.0.

Instead of pretending there is exactly one inevitable AI future, he lays out twelve very different scenarios, ranging from utopian arrangements to genuinely catastrophic ones.

I think that is a much healthier way to think about this.

There is no law of physics saying:

intelligence → consciousness → self-preservation → hatred of humans → extinction.

Those are separate steps.

Each one needs an argument.

Build the Airbags

Maybe AI becomes the most transformative technology humans ever build.

I actually think there is a decent chance of that.

Which is exactly why safety work matters.

But safety work is not the same thing as assuming doom.

Cars killed people.

We built safer cars.

Airplanes created new catastrophic possibilities.

We built aviation safety systems orders of magnitude better than what existed at the beginning.

Computers created cybercrime.

The internet created entirely new attack surfaces.

Biotechnology created entirely new risks.

Every powerful technology gives humans new ways to do stupid things.

The answer cannot always be:

therefore we should never have invented it.

The interesting question is:

How do we make something incredibly powerful while making catastrophic failure increasingly difficult?

That is the problem I actually want AI safety research to solve.

Not proving that the machine has a soul.

Not predicting exactly what society looks like in 2050.

Not deciding in advance that we are doomed.

Build the seatbelts.

Build the airbags.

Build the roads.

Teach people how to drive.

And while we are doing that, maybe keep making the engine better too.