The Machine Should Make Us More Human

The best sentence in Thinking Machines Lab's essay is not technical.

It is moral.

"The mission of Thinking Machines is to build AI that extends human will and judgment."

That sentence matters because most of the AI race is still pointed in the other direction.

The frontier race asks how much work a model can do alone. How long its autonomous time horizon is. How many software tasks it can finish without help. How many humans can be removed from the process.

Those are useful measures. They are not enough.

A civilization is not improved merely because fewer humans are needed to move tokens through workflows. A company is not made wiser because more judgment has been hidden inside a model trained somewhere else. A person is not made freer because the machine can now produce more output without asking what should be wanted in the first place.

The future worth building is not one where humans remain as ceremonial approvers.

It is one where AI gives human beings more contact with reality, more leverage over their own work, and more room to make things that would otherwise stay unborn.

The wrong dream

There is a strange poverty in the way we talk about AI.

We describe the greatest cognitive technology ever built as if its highest purpose is to make us unnecessary.

The promise is always the same. Fewer people. Fewer meetings. Fewer writers. Fewer accountants. Fewer programmers. Fewer decisions. More autonomy. More throughput. More tasks completed by machines while humans sit somewhere above the system, approving outcomes they no longer understand.

This is an old dream wearing a neural-network costume.

It is the dream of central planning. One intelligence sees more, knows more, optimizes more, and eventually coordinates the messy world from above. The only thing that changes is the dashboard. Once it was a ministry. Then a spreadsheet. Then a platform. Now a frontier model.

But the problem was never merely that the planner was not smart enough.

Hayek's point in The Use of Knowledge in Society was more radical. Productive knowledge is not sitting somewhere waiting to be aggregated. It is scattered through people, places, practices, instincts, habits, prices, mistakes, and feedback. It is tacit. Local. Fleeting. Often half-conscious. It lives in the shopkeeper rearranging a shelf, the chef adjusting a recipe, the accountant who notices a pattern in a client's messy books, the trader who feels something in the flow, the founder who sees a customer hesitate before they can explain why.

That knowledge cannot simply be uploaded into a central brain without losing the very thing that made it useful.

Thinking Machines makes the same move for AI. Most AI is trained in a handful of places, frozen, and then distributed outward. It may be brilliant. It may be useful. But it is not shaped deeply by the people it serves. It does not learn enough from the work they do together.

That is the hidden danger.

Not that AI becomes too intelligent.

That intelligence becomes too centralized.

Work is where knowledge is born

One of the strongest parts of the Thinking Machines essay is its insistence that knowledge is not a static repository.

Knowledge is made in the work.

This sounds obvious until you notice how much AI product thinking assumes the opposite. The user has some knowledge. The system extracts it. The model learns a representation. The workflow is automated. The person moves out of the loop.

But real work does not behave like that.

Work changes the person doing it. You start with an intention, touch the material, receive feedback, adjust, notice something unexpected, change the goal, and become slightly different through the process. The chef does not merely execute a recipe. The recipe emerges through heat, smell, failure, hunger, memory, and taste. The founder does not merely run a plan. The plan is corrected by customers, cash, timing, competitors, shame, luck, and stubbornness.

This is why Truth Is Not a Thought matters so much to me. Truth is not something the mind owns at a distance. It is contact. It is what pushes back. It is the texture of reality meeting the nervous system.

AI can help with that contact. It can make the loop faster. It can reveal patterns. It can reduce friction. It can make the next experiment cheaper.

But if AI removes people from the place where feedback happens, it does not extend human judgment. It atrophies it.

That is the difference between a tool and a substitution machine.

A good tool makes the hand more capable. A bad tool makes the hand irrelevant and then wonders why the body has become weak.

The human is not a bottleneck

The phrase "human in the loop" has become almost unbearable.

It sounds responsible, but it often hides a contemptuous picture of the human being. The machine does the real work. The human sits in the loop as a safety valve, a compliance token, a liability shield, or a final click.

That is not human agency. That is bureaucracy with better autocomplete.

In The Last Human in the Loop, I argued that the human is not valuable because he can press approve. The human is valuable because he can stand outside the system and ask what the system is for.

AI can optimize a path. It cannot decide, by itself, which paths are worth walking.

That distinction will become more important as models become more capable. The common benchmark asks how long a model can operate autonomously. METR tracks the time horizon of tasks models can complete on their own. That is useful. I want better agents. I run my life and company with them. I want models that can code, search, plan, repair, write, test, and execute without me babysitting every step.

But autonomy is not wisdom.

A model that can work for ten hours without interruption may be economically powerful. It may also be spiritually stupid if it is optimizing the wrong thing with great discipline.

The better benchmark is harder:

Does this AI make the human more capable of judgment?

Does it preserve contact with the work?

Does it increase the number of people who can make things?

Does it keep weirdness alive?

Those questions are harder to measure than task completion. That does not make them softer. It makes them closer to reality.

Making, not consuming

There is a warmer version of the same argument in Anish Acharya's X article, The Most Human Technology Ever Made, written in response to Thinking Machines.

The line that matters is simple: people are happier when they make things.

That may be the most important argument for AI that almost nobody in enterprise software knows how to price.

The old productivity story says people want to save time. Sometimes they do. But often people want to spend time better. They do not want an empty life with more optimized consumption. They want to build the table, make the song, write the essay, design the tool, launch the tiny business, fix the family workflow, code the strange personal app nobody would ever fund.

Social media became slop because consumption outpaced creation. A billion people were given infinite feeds and a few were given the tools to shape the feed. The result was predictable: passive attention became the business model.

AI could repeat that disaster at a higher level.

It could become infinite synthetic consumption. More videos, more posts, more avatars, more generated noise, more perfectly personalized distraction.

Or it could become anti-slop.

It could collapse the cost of trying things. It could let the electrician build a tool, the accountant build an agent, the teacher build a curriculum, the child build a game, the father build a family operating system, the small firm build software that used to require a venture-backed team.

That is the version of AI worth defending.

Not AI as replacement for human making.

AI as the return of making.

The taste gap returns

When output becomes abundant, judgment becomes scarce.

That was the argument in The Taste Gap. AI does not make taste irrelevant. It exposes how rare taste really is.

The same is true for organizations.

If every company can generate copy, code, dashboards, designs, memos, analyses, and automations, then the advantage shifts. It is no longer enough to produce. Everyone can produce. The question becomes whether the organization knows what is worth producing.

This is where the Thinking Machines thesis becomes strategically important.

A single generic model for every customer has an economic incentive to absorb what makes each customer distinct. It standardizes. It compresses. It turns living knowledge into product behavior. That can be useful, but it also creates sameness.

The better path is different.

Organizations need AI that helps them cultivate their own knowledge. Their way of serving customers. Their judgment. Their taste. Their weirdness. Their local truth.

For AI4 Accountancy, this is not abstract. The goal is not to replace accountants with a generic finance oracle. The goal is to build a learning loop around the real work of small firms: their documents, clients, exceptions, rhythms, mistakes, questions, and judgment. The software should not erase the accountant's craft. It should make the craft sharper, faster, more scalable, and more visible.

The moat is not only the model.

The moat is the loop between model, work, human judgment, and reality.

Decentralized alignment

The most political part of the Thinking Machines essay is also the most important.

Values, like knowledge, are distributed.

Today the values and voice of AI are decided in a handful of places. Even with good intentions, that concentrates power. One model character. One safety style. One set of defaults. One moral posture. One hidden theology of what a helpful machine should be.

A more moral AI is not enough if the morality is chosen by a few.

This does not mean every person should be able to turn a frontier model into a weapon. Safety matters. Ownership without responsibility is not sovereignty. It is childishness.

But centralized alignment has its own danger. It turns ethics into a product setting controlled by institutions that are themselves political, commercial, cultural, and captureable.

A human future requires many AIs shaped in many places. Models that disagree. Organizations that tune their tools toward their own real values. Individuals who can build assistants that reflect their actual life rather than the average preference of a lab's training loop.

This is not just a technical architecture.

It is a civilizational preference.

Do we want one intelligence to average us into safety?

Or do we want an ecosystem where humans remain capable of choosing, shaping, arguing, making, and taking responsibility?

I know which future feels alive.

The future worth building

The mistake is to ask whether AI will be autonomous or human.

It will be both. It already is.

Machines should do what machines can do reliably. I do not want humans wasting their lives on glue work, admin, repetitive checking, and bureaucratic theater. I use AI every day because it gives me leverage. It helps me run systems that would otherwise be too heavy. It lets me think and do at the same time.

But leverage is only good when it serves a human end.

A shovel extends the hand. A telescope extends the eye. A piano extends musical intention. A market extends distributed judgment. A good AI should extend conscious action.

That is the frame I keep returning to: technological anthropocentrism.

Technology should make human beings more capable of truth, curiosity, and beauty. It should bring us into better contact with reality. It should expand the range of what we can ask, make, notice, repair, and love. It should not turn us into passive consumers of machine output or ceremonial managers of systems that no longer need us.

The question is not whether AI can do more without us.

It can.

The question is whether, after it does, there is more of us left in the world.

More judgment.

More taste.

More contact.

More courage to make.

More weirdness.

More human beings who are not merely optimized, but alive.

That is the future worth building.