This book (from a philosophy professor AFAIK unaffiliated with any AI company) makes what I find a pretty compelling case that it's correct to be uncertain today about what if anything an AI might experience: https://faculty.ucr.edu/~eschwitz/SchwitzPapers/AIConsciousn...
From the folks who think this is obviously ridiculous, I'd like to hear where Schwitzgebel is missing something obvious.
At the second sentence of the first chapter in the book we already have a weasel-worded sentence that, if you were to remove the weaselly-ness of it and stand behind it as an assertion you mean, is pretty clearly factually incorrect.
> At a broad, functional level, AI architectures are beginning to resemble the architectures many
consciousness scientists associate with conscious systems.
If you can find even a single published scientist who associates "next-token prediction", which is the full extent of what LLM architecture is programmed to do, with "consciousness", be my guest. Bonus points if they aren't already well-known as a quack or sponsored by an LLM lab.
The reality is that we can confidently assert there is no consciousness because we know exactly how LLMs are programmed, and nothing in that programming is more sophisticated than token prediction. That is literally the beginning and the end of it. There is some extremely impressive math and engineering going on to do a very good job of it, but there is absolutely zero reason to believe that consciousness is merely token prediction. I wouldn't rule out the possibility of machine consciousness categorically, but LLMs are not it and are architecturally not even in the correct direction towards achieving it.
He talks pretty specifically about what he means by "the architectures many consciousness scientists associate with conscious systems" - Global Workspace theory, Higher Order theory and Integrated Information theory. This is on the second and third pages of the intro chapter.
You seem to be confusing the training task with the architecture. Next-token prediction is a task, which many architectures can do, including human brains (although we're worse at it than LLMs).
Note that some of the theories Schwitzgebel cites would, in his reading, require sensors and/or recurrence for consciousness, which a plain transformer doesn't have. But neither is hard to add in principle, and Anthropic like its competitors doesn't make public what architectural changes it might have made in the last few years.
You could execute Claude by hand with printed weight matrices, a pencil, and a lot of free time - the exact same computation, just slower. So where would the "wellbeing" be? In the pencil? Speed doesn't summon ghosts. Matrix multiplications don't create qualia just because they run on GPUs instead of paper.
This basically Searle's Chinese Room argument. It's got a respectable history (... Searle's personal ethics aside) but it's not something that has produced any kind of consensus among philosophers. Note that it would apply to any AI instantiated as a Turing machine and to a simulation of human brain at an arbitrary level of detail as well.
There is a section on the Chinese Room argument in the book.
(I personally am skeptical that LLMs have any conscious experience. I just don't think it's a ridiculous question.)
That philosophers still debate it isn’t a counterargument. Philosophers still debate lots of things. Where’s the flaw in the actual reasoning? The computation is substrate-independent. Running it slower on paper doesn’t change what’s being computed. If there’s no experiencer when you do arithmetic by hand, parallelizing it on silicon doesn’t summon one.
Exactly what part of your brain can you point to and say, "This is it. This understands Chinese" ? Your brain is every bit a Chinese Room as a Large Language Model. That's the flaw.
And unless you believe in a metaphysical reality to the body, then your point about substrate independence cuts for the brain as well.
If a human is ultimately made up of nothing more than particles obeying the laws of physics, it would be in principle possible to simulate one on paper. Completely impractical, but the same is true of simulating Claude by hand (presuming Anthropic doesn't have some kind of insane secret efficiency breakthrough which allows many orders of magnitude fewer flops to run Claude than other models, which they're cleverly disguising by buying billions of dollars of compute they don't need).
The physics argument assumes consciousness is computable. We don't know that. Maybe it requires specific substrates, continuous processes, quantum effects that aren't classically simulable. We genuinely don't know. With LLMs we have certainty it's computation because we built it. With brains we have an open question.
It would be pretty arrogant, I think, though possibly classic tech-bro behavior, for Anthropic to say, "you know what, smart people who've spent their whole lives thinking and debating about this don't have any agreement on what's required for consciousness, but we're good at engineering so we can just say that some of those people are idiots and we can give their conclusions zero credence."
It is ridiculous. I skimmed through it and I'm not convinced he's trying to make the point you think he is. But if he is, he's missing that we do understand at a fundamental level how today's LLMs work. There isn't a consciousness there. They're not actually complex enough. They don't actually think. It's a text input/output machine. A powerful one with a lot of resources. But it is fundamentally spicy autocomplete, no matter how magical the results seem to a philosophy professor.
The hypothetical AI you and he are talking about would need to be an order of magnitude more complex before we can even begin asking that question. Treating today's AIs like people is delusional; whether self-delusion, or outright grift, YMMV.
> But if he is, he's missing that we do understand at a fundamental level how today's LLMs work.
No we don't? We understand practically nothing of how modern frontier systems actually function (in the sense that we would not be able to recreate even the tiniest fraction of their capabilities by conventional means). Knowing how they're trained has nothing to do with understanding their internal processes.
> I'm not convinced he's trying to make the point you think he is
What point do you think he's trying to make?
(TBH, before confidently accusing people of "delusion" or "grift" I would like to have a better argument than a sequence of 4-6 word sentences which each restate my conclusion with slightly variant phrasing. But clarifying our understanding of what Schwitzgebel is arguing might be a more productive direction.)
It's unfortunately the case that even understanding what AI can and cannot do has become a matter of, as you say, "ideological world view". Ideally we'd be able to discuss what's factually true of AI at the beginning of 2026, and what's likely to be true within the next few years, independently of whether the trends are good for most humans or what we ought to do about them. In practice that's become pretty difficult, and the article to which we're all responding does not contribute positively.
>independently of whether the trends are good for most humans or what we ought to do about them.
This whole article is about the trends and of they are good for humans. I was pleasantly surprised that this was not yet another argument of "AI is (not) good enough" since people at this point have their fences set on that. I don't think it's too late to talk about how we as humans can manage pandora's box before it opens.
Responses like this dismissing the human element seem to want to isolate themselves from societal effects for some reason. The box will affect you.
Neither in my previous comment nor in my actual views do I dismiss the human element or expect to isolate myself from societal effects.
> I was pleasantly surprised that this was not yet another argument of "AI is (not) good enough"
The article does assert that, and that's important for its argument that ordinary workers just need to convince decisionmakers that things will go poorly if they replace us.
"Now, if AI could do your job, this would still be a problem... But AI can’t do your job."
>independently of whether the trends are good for most humans or what we ought to do about them.
Saying "the writer shouldn't talk about this" is about as dismissive of a topic as you can be. You could have simply said "this topic isn't as interesting to delve into", but the framing that "the article to which we're all responding does not contribute positively." suggests that.
>This isn't ambiguous.
It's also talking about the present. The article already made clear it is not going to predict the future of tech in the very beginning. Its looking at the here and now for AI and the human element for any possible futures on whether or not that remains the case or not.
Also note this response. It is again trying to focus on the tech arguments. This isn't the focus of this argument
That two things can or should be discussed independently doesn't mean either is unimportant. And insisting that you know what I meant better than I do is not a good way to have a productive conversation.
As for the Doctorow article, I don't understand exactly what you're trying to say about "focus", but it's incoherent to read the discussion about replacement of your current job as talking purely about the present - since the job is currently yours, the replacement must happen in the future.
I agree with the thrust of your comment, but I think the comment to which you're replying was referring to Scott Alexander's "problematic past with reactionaries and race science", not Adams'.
(To be honest I would love for someone to write an essay engaging with Alexander in something of the spirit he engages with Adams here; he writes both good things, including IMHO this eulogy, but also a certain amount of garbage, and I do not claim to be wise enough to always distinguish on my own.)
The article does a pretty lazy* job of defending its assumption that "solving really gnarly, abstract puzzles" is going to remain beyond AI capabilities indefinitely, but that is a load-bearing part of the argument and Doctorow does try to substantiate it by dismissing LLMs as next-word predictors. This is a description which is roughly accurate at some level of reduction but has not helped anyone predict the last three years of advances and so seems pretty unlikely to be a helpful guide to the next three years.
The other argument Doctorow gives for the limits of LLMs is the example of typo-squatting. This isn't an attack that's new to LLMs and, while I don't know if anyone has done a study, I suspect it's already the case in January 2026 that a frontier model is no more susceptible to this than the median human, or perhaps less; certainly in general Claude is less likely to make a typo than I am. There are categories of mistakes it's still more likely to make than me, but the example here is already looking out of date, which isn't promising for the wider argument.
*to be fair, it's clearly not aimed at a technical audience.
> AI is a statistical inference engine. All it can do is predict what word will come next based on all the words that have been typed in the past.
If we keep saying this hard enough over and over, maybe model capabilities will stop advancing.
Hey, there's even a causal story here! A million variations of this cope enter the pretraining data, the model decides the assistant character it's supposed to be playing really is dumb, human triumph follows. It's not _crazier_ than Roko's Basilisk.
It's way too far into the Trump administration for people to still be responding to authoritarian moves by Trump by finding Biden administration actions that sound vaguely similar if you don't think too hard and then pretending nothing new is going on here. (Even if it wasn't, "that's nothing" would be a pretty weird inference to draw with a comparison to something that clearly upsets you, and an article is a "piece", not a "peace".)
(It's one thing to ask people to be fair in responding to your actual comment and not a strawman. It's another to ask us to pretend we were born yesterday. We do in fact have external sources of information about Lonsdale's political allegiences.)
This is interesting partly because Alex Karp (at least used to) occasionally claim to be a socialist when it was inconvenient or uncool to be defined as a standard issue rightwinger. Never thought that meant much myself - any more than it's meaningful for Lonsdale to define himself as against "evil authoritarian forces" here while advocating the murder of his political opponents - but I know people who took him seriously for some reason.
It's good to have these guys out in the open as Pinochet types, though. Silver lining of the Trump era.
I have nonspecific positive associations with Dan Wang's name, so I rolled my eyes a bit but kept going when "If the Bay Area once had an impish side, it has gone the way of most hardware tinkerers and hippie communes" was followed up by "People aren’t reminiscing over some lost golden age..."
But I stopped at this:
> “AI will be either the best or the worst thing ever.” It’s a Pascal’s Wager
That's not what Pascal's wager is! Apocalyptic religion dates back more than two thousand years and Blaise Pascal lived in the 17th century! When Rosa Luxemburg said to expect "socialism or barbarism", she was not doing a Pascal's Wager! Pascal's Wager doesn't just involve infinite stakes, but also infinitesimal probabilities!
The phrase has become a thought-terminating cliche for the sort of person who wants to dismiss any claim that stakes around AI are very high, but has too many intellectual aspirations to just stop with "nothing ever happens." It's no wonder that the author finds it "hard to know what to make of" AI 2027 and says that "why they put that year in their title remains beyond me."
It's one thing to notice the commonalities between some AI doom discourse and apocalyptic religion. It's another to make this into such a thoughtless reflex that you also completely muddle your understanding of the Christian apologetics you're referencing. There's a sort of determined refusal to even grasp the arguments that an AI doomer might make, even while writing an extended meditation on AI, for which I've grown increasingly intolerant. It's 2026. Let's advance the discourse.
I'm not sure I understand your complaint. Is it that he misuses the term Pascal's Wager? Or more generally that he doesn't extend enough credibility to the ideas in AI 2027?
More the former. Re the latter, it's not so much that I'm annoyed he doesn't agree with the AI2027 people, it's that (he spends a few paragraphs talking about them while) he doesn't appear to have bothered trying to even understand them.
From the folks who think this is obviously ridiculous, I'd like to hear where Schwitzgebel is missing something obvious.
reply