I was reading an essay by Jack Clark, one of Anthropic’s co-founders. It’s a thoughtful piece, worth your time if you care about what happens when intelligence evolves faster than our ability to steer it. Jack clearly has a bias but is trying to counter it against strong incentives.

At the end, Jack links to a post by Mechanize — an AI company openly pursuing the full automation of human labour. The contrast couldn’t be starker: one side asking how to keep AI aligned with humanity, the other wondering whether we should replace human output altogether.

AI is an incredibly exciting and promising technology; I use it daily, and it’s already changing how I think and work. But precisely because it’s so powerful, we need to remember: its potential to reshape society is unlike anything before.

Large Language Models aren’t explicitly programmed with rules; instead, they learn statistical patterns in language during training. This process leads to emergent, sometimes unpredictable, abilities as they generate new text based on what they’ve learned. That’s extraordinary — but it’s also a reminder of how little we truly control. Which is why I can’t buy into Mechanize’s confidence. There’s no mention of safety, of alignment, of who really benefits. That kind of optimism feels hype driven.

A calculator never knew it was a tool. AI might. And that changes everything.

The Utopian Promise

There’s a seductive logic to full automation: even if replacing all human labour is inevitable, we should accelerate it — because the upside will outweigh the costs.

Upside for whom, exactly?

In reality, the benefits never flow evenly. They pool around those who build, own, and deploy the systems, while the rest are told to be grateful for “abundance” that never quite arrives.

The story repeats: outsourcing, offshoring, financialisation — each sold as progress, each quietly hollowing out the middle. Now automation arrives with the same promise of liberation. Maybe. Or maybe it just frees capital from labour.

The danger isn’t technology itself; it’s the incentives shaping its direction. Speed, scale, monopoly. Not safety, patience, or shared benefit.

We don’t need to automate everything. We need to decide what’s worth automating — and what must remain human. Because the goal of technology was never to make people obsolete; it was to make life better.

The future I want isn’t anti-automation. It’s pro-alignment, where intelligence amplifies creativity, empathy, and understanding, not just profit margins.

The Article By Jack Clark - https://importai.substack.com/p/import-ai-431-technological-optimism

The Article By Mechanize - https://www.mechanize.work/blog/technological-determinism/