AI AGENCY ISN’T HERE YET… By Machine Learning Street Talk
The text discusses the concept of AI agency and its relationship to human cognition. It explores how AI may evolve to mimic human intelligence and the challenges in understanding and defining consciousness in machines. The author emphasizes the complexity of cognitive processes and the need to carefully consider the implications of creating AI with human-like capabilities.
Highlights
So life itself is always a goal directed process. Life. Living organisms have goals, have purposes.
So calling it an illusion, it just I don’t understand how you can reduce it to, to an illusory nature.
you can reduce something and learn how it works like consciousness for example doesn’t mean that the thing is an illusion - doesn’t exist. it emerges from the smaller things
it’s almost like the curriculum gets baked into us. So there are certain instincts we have around language learning, mutual shared attention, um, babbling to babies and stuff like that. But it’s almost like it’s giving us the, the guide of how to learn the skill, but it’s not explicitly telling us how to learn. Yeah, yeah. It gives I mean, evolution gives us capacities.
generative AI can create a procrastination of understanding. What I mean by that is I’m seeing all of these productivity YouTubers, and many of them are scientists, and they’re saying, look, this is how I do my research now. I copy all of this stuff into ChatGPT, and ChatGPT has all sorts of pernicious guardrails and, you know, behaviors that people don’t yet understand. So they are missing many important bits of information, and they are giving themselves the illusion of either immediate understanding or, you know, deferred understanding.
human chauvinism, which which might be to make the argument that it’s not possible in principle for these things to have the kind of creativity we have now at the moment, my current position is they are lacking agency and this kind of meaning diffusion that we have in the real world. And what that means is, um, a large language model is centralized, it’s monolithic, it’s sclerotic, it’s designed to minimize entropy. So, you know, what it’s doing is it’s always trying to simplify what you give it. And that’s the the weird paradox, isn’t it? Because I can give it anything. Right? I can I can say here’s, here’s Philip Ball and here’s, here’s a camera and I can, I can come up with anything. So I am a genuine source of creativity. But that thing isn’t. And the sclerotic thing is also because it was only trained once, let’s say six months ago. So it’s always six months ago. And I’m like, whoa, that’s that’s six months ago. I’m over here now. So it’s collectively kind of pulling us back to a stationary point.
“meaning diffusion” refers to the way meaning and understanding are spread out and constantly evolving in the real world, as opposed to being centralized and fixed in large language models (LLMs). In the context of the passage, “meaning diffusion” refers to the way meaning and understanding are spread out and constantly evolving in the real world, as opposed to being centralized and fixed in large language models (LLMs). The speaker suggests that in human society and communication, the meaning of words, concepts, and ideas is not static. Instead, it is continually being reinterpreted, contextualized, and adapted based on new experiences, interactions, and cultural shifts. This diffusion of meaning allows for creativity, nuance, and the evolution of thought over time. In contrast, the speaker argues that LLMs have a centralized and monolithic understanding of meaning, based on their training data. They do not participate in the same kind of dynamic meaning-making process that humans do. Once trained, their understanding of concepts is largely fixed, leading to a more rigid and less adaptive model of language and meaning.
certainly the boundaries will be blurred in terms of whether we are interacting with an AI or a person. You know, we might be interacting with a hybrid of the two, with the person hiving off some of the mundane stuff to the AI and occasionally coming in or whatever
I have the volition to come up with my own goals, and my goals aren’t being changed by other people
society conditions us
I totally agree that real agents have goals that are self-determined. That’s I would make that a part of the definition of what an agent is.
most humans are not agentic by this definition
But increasingly, I think people who study consciousness and animal behavior and animal intelligence suggest that it’s a multi-dimensional thing. There are different aspects and different types of consciousness.
as I gets more sophisticated, as it gets more capable, somehow it’s converging on the human mind. Why on earth would we think that? And in fact, every indication, it seems to me, is that it’s not like that, that it is becoming increasingly good at mimicking that while very clearly, evidently operating on different, if you like, different cognitive principles
AI “thinks” very differently from humans
one way of trying to understand or get at agency from first principles is to think of agents as. Entities that are somehow able temporarily to seem to evade the second law of thermodynamics
Erwin Schrödinger with his book What Is Life in 1944? And he talked about, uh, living things as feeding on negative entropy, by which he means that somehow they’re able to maintain their own organization. In the face of this, what seems to be universal capacity of things to fall apart to, you know, dissolve into, into chaos
And that underscores this, this fact that agents need to be out of equilibrium by definition, really with their surroundings. So they need to have some boundary in order to distinguish what is the agent from, what is the surroundings, the boundary, you know, of, of, uh, disequilibrium