The brain is a composit of cells from two lineages that separate very early in development and reflect a deeper spatial and functional separation deep in the history of metazoan (read: animal) evolution.
Tying all this to humans specifically is just confusing. This is something that’s seen in a substantial portion of animals, and as such has nothing to do with “ poetry, mathematics and wondering about our own origins”. These features of humans evolved hundreds of millions of years after this two lineage composite organization of animal brains evolved.
I wish the article hadn’t gone out of its way in confusing these issues to push the “relevance” of the research. It’s a pretty remarkable finding anyway.
I’m a materialist. I believe rabbits are conscious. And I can belive there’s a kind of consciousness for a group of humans like a nation state.
However, the United States isn’t just a group of humans. It’s also a group of bison. Wild horses. Flies. Bacteria. Bats. Dung beetles. And whatever entity occupies the White House.
If the United States is conscious, it’s unclear why it’s consciousness stops at its fictitious, human imagined border. Except for humans, no other animal cares or knows when they cross the border.
So a much better claim can be made for the Earth and the entire web of life.
At which point we’re at Gaia hypothesis/Earth Systems science territory, and I’m fine with that too.
I still remain a materialist. And don’t consider the United States to be particularly conscious as an entity.
The United States is not a geographical area, it is an abstract system of organization of humans that largely group together and make rules and modifications to about a geographic area. If all Americans cease to exist, can it really be said that the United States still exists because there are squirrels?
Indeed. Thanks for the great explanation. Honestly, the complexity would be very annoying if it wasn’t also invigorating and fascinating.
The interconnections between genes has led to proposals of “omnigenic” models, the idea that all genes expressed in a cell affect every given trait of the cell, to some extent.
When it comes to something like autism, it’s a word covering a vast number of traits and behaviors of many different cells that gives rise to that set of effects as seen from outside.
And the issue with genetic testing is, we often only look at specific regions of the genome. Even when we do whole genome tests, and compare whole genomes of autistic folks, what we find is no single origin but many overlapping patterns. No definitive set is found because we are looking at complex processes that have many nodes where mutations can mildly tweak the function. Many such functionally relevant mutations can chain together to give you the signatures of autism, and no two are the same.
All these reasons also point to why we are increasingly realizing autism isn’t a “disease” with a “cure”. It’s a way in which people can be people. It’s time society accepted that and expands itself to accommodate.
Hmm let me try then. AI agents do not experience real time. This is understandable given their design, but it’s also something we have good empirical evidence for. They cannot, especially over the long horizon, track how much real time has passed as they complete their tasks. And they are not off by a few minutes but often bizarrely off, even mixing across past present and future.
Biology, on the other hand, is nothing but timed processes in a loop, the most obvious to us being the circadian cycle. As estimators of wall clock time, biology isn’t great, but when it comes to internal processes, and most certainly learning, memory, sensing, locomotion… biology is rhythmic in behavior, and the rhythms go all the way down to gene expression. More, these rhythms are, except during sleep, constantly entraining to signals from the environment that indicate time, most importantly light.
I think it’s a fairly unremarkable claim that agency and consciousness are temporal processes that depend on systems having an internal sense of time. How else can you anticipate? How can a system that can be literally turned off ever succeed in an environment where time never stops?
I agree, there is certainly a clear operational difference in how human brains and these models produce language. I'm not convinced, though, that experiencing real time is a requirement of conscious thought. A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information. The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on). I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself (even though it might be difficult to put ourselves in the shoes of such an entity).
> A cognitive system (assuming these models and the brain are both examples of this) needs to proceed from state A to state B as it processes information
Is that all it needs to do? A rock, in response to a sound signal (make it as informative as you want) can also be said to go from state A (molecules at rest) to state B (molecules vibrating in response to the sound). Is the rock a cognitive system?
Or let’s go up a level of complexity. Is a thermostat a cognitive system? It doesn’t just go from state A to B but measures changing stages against a reference. Is it a cognitive system?
> The time interval between these states can be arbitrary, it seems to me (setting aside problems with disconnecting the mind from its substrate, which does of course need rhythms at various frequencies; oxygen at a higher frequency than sugar, and so on)
Not sure how you land on this. Going from A to B, in your definition, is a purely internal state transition, right? If the motivation to go to state B is external (as will often be the case with a conscious agent), and the system is offline entirely while it switches from A to B, how is it to reverse or stall its transition if external signals contradict the earlier decision?
It’s a very bizarre definition of consciousness, that you don’t need to be in time. Seems to break the word beyond meaning, and allows it to be applied to any system capable of change.
> I'm not sure that going from A to B quickly, or slowly, or with intervals of a thousand years affects that entity's consciousness in of itself
It’s not about the speed of transition from A to B. Instead it’s about the internal processes being temporally commensurate, and entrainable to environmental periodicities.
All of biology (plants and many bacteria included) has its dynamics are organized around an internally generated rhythm. This rhythm entrains to the external environment, shifting its phase to match (mismatch leads to issues like jet lag: a place where your consciousness shows its temporal boundaries as it is abruptly subjected to an environment out of sync with its current phase tethered to another location). Crucially, the internal rhythm continues to run in isolated conditions (total darkness), and have been measured to be around 24 hours, but slightly off, within and across species (and the same holds for endogenous tidal rhythms, which also don’t exactly track tidal period left to themselves).
Everything, including learning and memory, but also sleep, sensory and locomotor function, shows organization around this internal temporal axis. The key advantage this gives, evolutionarily, is anticipation. The sunflower rises to face the east, not in reaction to sunrise, but in advance of it (many lovely YouTube videos of this), so it can maximally extract the nutrition it needs from whatever sunlight it can get. A purely reactive system would waste a ton of time after sunrise getting subsystems ready to respond.
Now, there is no broadly agreed theory of consciousness that points to these oscillations as the source. For one, Cyanobacteria and plants have them, a too much of science has rested on the baseless assumption of consciousness being some unique, or advanced thing.
But the evidence is rather overwhelming that consciousness is very much shaped by these internal oscillations. Deep cave experiments have shown what happens to cognition, and consciousness, when the clock free-runs (that is, keeps cycling with no entraining light or temperature signals from the environment). Studies have also shown that restoring the amplitude of these rhythms in patients with “diseases of consciousness” leads to improvement of symptoms.
There’s also a LOT of molecular evidence showing how the circadian clock affects learning and memory. Most critically, actual synapses are never static. Major components are on a 24 hour cycle, and recent research has shown that major clock proteins are at the synapse organizing its dynamics, and synaptic proteins critical to learning and memory loop back to affect the clocks phase and amplitude.
Timing is, beyond doubt, a huge aspect of what biology is. And every bit of evidence we have, from the molecular level to observable behavior we can feel ourselves, says that consciousness is fundamentally linked to how the body organizes its internal rhythms and responds to external time.
I mean there is probably a cluster of neurons somewhere in the brain that if damaged would also prevent you from perceiving time going forward. Would it immediately strip you of consciousness? I don't think so.
Nope. Every cell is an autonomous circadian oscillator. Not just every neuron, but every cell. Red blood cells don’t have a nucleus, but they do have a metabolic clock.
You and I are clusters of about 36 trillion cells that are all able to individually keep time. They synchronize and orchestrate their timekeeping, and entrain to the external environment, and there is a cluster of cells, the suprachiasmatic nucleus, that is the master orchestrator of circadian time, sitting right behind the eyes as light is the main signal it uses to tell external time.
We know its function can get dampened with age.
However, please note this is not how we perceive time, in the sense of being able to count 60 seconds and have it match the wall clock. That’s a subsystem. But the internal temporal order of your body is cell by cell, not dictated from one spot in your body.
Our cells perceive each other, communicate with each other, synchronize with each other. The concept of self rides atop that. Stop the cells communicating and you stop perceiving yourself or the environment.
For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
>track how much real time has passed as they complete their tasks.
Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
>is nothing but timed processes in a loop,
I mean, so is an agents harness. You can make as many loops as you'd like here.
>For around 8 hours a day, neither do you. Not sure if this has anything to do with the subject at all.
You mean sleep? Actually the answer is pretty complicated. Your body clock is very much on during sleep. It responds to temperature changes. Sound and light responsiveness is obviously dampened, but hardly absent.
You are unaware of wall clock time. You’re in an altered state of consciousness that needs your eyes closed after all. However, you are not timeless, nor is the entirety of your body and brain unaware of environmental signals for time.
> Humans don't do this either. You use context clues from the world around you. If I lock you in a room with no windows or a dark cave your timing senses can go all fucky really quick.
Again, that’s wall clock time. Our time perception does indeed get fucked up in total darkness. But our internal clock ticks on. I’d recommend reading about the Aschoff Bunker experiments, which first proved this rigorously.
> I mean, so is an agents harness. You can make as many loops as you'd like here.
An agents harness doesn’t touch its weights. A figure 8 on paper is a loop. That doesn’t mean it’s the same as a dynamical loop that’s self sustaining and internally organized.
Are you just reading the words and randomly grabbing related concepts to say nothing here is meaningful?
If AI is conscious, then Pluto is a planet, the Sun is a galaxy, and a black hole is a star.
I’m glad to see someone in a position of any power in the AI world state baldly that AI isn’t conscious. There are times when it feels like we’ve reached complete delulu land on this topic, so it’s a breath of fresh air to see someone not dance around this.
None of this means artificial consciousness cannot be achieved. But the way we’re reacting to these models is proof, from a natural experiment, that a conscious machine should not exist, and certainly shouldn’t be produced as a utilitarian tool that is sold for profit!
I think bearish on LLMs for automation, and bullish for LLM+human experts in specific fields, is about the right expectation for current architectures.
Apart from issues with task generalization, or perhaps related to it, is the fact that LLMs have real trouble with timekeeping, and cannot estimate the real world time it will take them to do things very well. This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures.
In narrow domains with more deterministic outputs though, this is less of an issue, and we see multiple agents succeed much better.
The fusion of that capacity, with humans in the loop able to better direct such agents and act as their temporal tethers, is where I think the real action will be for a while at least.
My belief is that LLMs will fundamentally change how we approach domain expertise. From what I've seen, SDEs tend to be over-specialized compared to what the company actually needs to implement due to the need to understand enough of the domain to pick a best path. If an LLM can see the domain enough so that someone in an adjacent field can be confident in their approach and quickly change course then you don't need as many niche SDEs
I think automation is coming but it will be way more gnarly than frontier labs want public to believe. Value is just too big, when you can automate most of eg customer support it will create huge savings and same time customer satisfaction will get better.
> same time customer satisfaction will get better.
This part just can't be true though, right?
We have all been in numerous customer service scenarios where all we want to do is talk to a real human and that is denied to us, and it's a terrible customer experience!
Sure, using an LLM would be better than some of the sort of "menu option" style customer service calls. But there is no way it's better than talking to an actual human being
This. Listening and creative customer support rep, especially one who can even escalate fundamental product issues to company leadership is worth weight of gold.
> This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures.
Any reason why that can't be solved through context management and keep-forward scaffolding?
And who’s to manage context? And who’s building the scaffolding? Yes, AI can be used for both, but you do realize that all this being self-contained and regulated internally is what makes biological agents successful agents, right? If you break the process apart and need to dial back in these aspects, and can only do so with human input, or another agent which will need the same handholding the one whose issues you’re solving for, where’s the agency?
Does RSI necessarily need to be pointed towards “AGI”, whatever that is?
Or can you have a recursive self improvement loop where the objective is to create models that are more legible to humans? Or better at attributing the source text if it’s meaningfully similar to output? Or architectures for models that are increasingly better at human-AI cowork rather than automation?
From my own understanding, nothing at all says the frontier is defined by the quest for “AGI”. This definition of the frontier assumes human intelligence itself has peaked and will remain stable, so how well defined can this goal ever be, if AGI stands in comparison to human intelligence for its definition?
To me, Amodei’s writing just reeks of posturing. Genuine action driven by this fear they claim to have would be meaningful.
Even setting aside moral quandaries, isn’t it basic project process to have your company’s internal goals be tethered to what people want, rather than what you calculate is inevitable?
There are genuinely other frontiers to explore, and Anthropic would do a lot to mitigate the current slide if it decided to put its resources towards another direction for AI.
Take back the agency that you are so blithely surrendering to models you do not understand. There is no inevitability to this path. There are critical, meaningful choices, and Anthropic wouldn’t be violating capitalism by taking an alternate that is more tethered to what users want and need. Maybe by asking them first, at scale.
In short, after after training AI on an extraordinarily amount of human cognitive output, we are now facing the possibility that our ability to train by working on hard problems will be slowly stripped away at least in some domains.
It’s like someone offers to build mag lev gym weights. It’s very cool that I can now lift the 500 pound weight with a finger. But what will I do when there’s no power and 500 pounds to lift?
Of course, cognition isn’t a single outcome problem like weight lifting. But we build cognition not wholly unlike how we build muscle: one needs resistance. Otherwise I’m not at all confident we “learn” in any depth.
The brain is a composit of cells from two lineages that separate very early in development and reflect a deeper spatial and functional separation deep in the history of metazoan (read: animal) evolution.
Tying all this to humans specifically is just confusing. This is something that’s seen in a substantial portion of animals, and as such has nothing to do with “ poetry, mathematics and wondering about our own origins”. These features of humans evolved hundreds of millions of years after this two lineage composite organization of animal brains evolved.
I wish the article hadn’t gone out of its way in confusing these issues to push the “relevance” of the research. It’s a pretty remarkable finding anyway.
reply