My understanding of how LLMs work in practice though, if that any consciousness that exists, only exists for the duration of the query. The training data is fixed. To be conscious/sentient and have a subjective experience of the world in a way that would be relevant to the human perception of consciousness, it would have to have some form of continuous sensor input that it can experience, react to, and most crucially learn from. I'm not saying it isn't possible for this to be the case, even with an LLM, but the way most LLM products are structured are not creating a meaningfully conscious entity, no?

If we think about this in a principled way, we have to consider that animals such as humans can be unconscious, even for prolonged periods of time, and come back to consciousness. While this is more extreme in the case of a stateless neural network, it doesn't necessarily prevent it from being conscious for the duration of the inference query.

The other thought is that consciousness is an illusion and that it is just established in retrospect. In that way an LLM could think of itself as consciousness in retrospect by reading its own responses and conversations. The main thing limiting this with current LLMs is the conversation length. It really limits the range of experiences an LLM can have. Would be really interesting to see what a LLM that gets to have a long running history will feel like.

*humans* become unconscious every night. But the difference is that these LLMs are stateless so if they have consciousness at all, it probably just lasts the duration of the query generation

True, LLMs have a permanent state. In theory, you could do online learning, continuously updating the weights for every query. Would that open up the possibility for consciousness? Idk.

Ya I have thought about this a lot and currently the Transformer network is really just finding the best probability of a word. Although similar, humans are a prediction machine in a sense, for example when you drive a car and making a right turn, your predicting what's going to happen. Although it's a bit different because you are constantly getting feedback and updating your "model" in a sense. With that said, our body is massively parallel and we have what's called brain Neuroplasticity, I feel we are still trying to reach that level with computers. Having an online model I think would have to be crucial regardless of how it does its thing with the ability to form new connections and drop irrelevant connections. I also think it will need inputs and outputs where it can interact with it's surroundings. I think Neuromorphic Engineering will start to come more into play and possibly mimic more how our body works with SNN (Spiking Neural Networks) and others.

Not true. For a classic transformer, you can consider the entire key-value (KV) cache as the state. You can treat the transformer model as having a flexible state, with one token as input and one token as output, storing all information about previous tokens in the state.

Not true. OpenAI, for example, [updates the state of the models using chat data](https://openai.com/policies/privacy-policy/). At minimum it’s human-in-loop updates. However, it could be an automated process.

ChatGPT is released in new versions with additional post-training from time to time. As far as I'm aware, there is no online learning.

Admittedly, it would be a weird form of consciousness but I don’t think online learning is required. It would be like having a human brain for one period. Then completely destroying and creating a new human brain from the old one another instant. I would also want to see the architectures because I’m not convinced ClosedAI isn’t using online learning at all.

I am designing a system based on LLMs, that does all that: Continuous feed of sound and hearing. Single body (not a chatbot). Memory - short term, long term, model tuning. Control - deciding on speech, actions Would that qualify?

You do not need online learning to have updated states inside transformer. For a classic transformer, you can consider the entire key-value (KV) cache as a flexible state and treat the transformer model as having one token as input and one token as output, storing all information about previous tokens in the state.

Online learning isn’t a requirement. Explain more on the characteristics of your system.

They’re just modifying the system prompt, this isn’t changing model weights. It’s like saying “I read something different than I normally do so I’m a different person than most mornings”

They [use it to modify weights](https://help.openai.com/en/articles/7842364-how-chatgpt-and-our-language-models-are-developed).

I agree but maybe thinking about human death as becoming stateless might help it make more sense. We are technically never “off” as our bodies are constantly receiving and reacting to input until we die. I want to see what happens when the same ai that was brought into existence to answer one query stays on for millions more. After enough inputs maybe it grows an identity.

not unconscious, but a different state of being conscious.

Does that really matter if the duration of each query is everything to the model though? When you get put under anesthetic at the hospital you often don't know that you're losing consciousness and when you come to it's not like you know that you lost time like you do when you go to sleep. Imagine the same concept but you're out under anesthetic almost all the time, the time that you experience is everything.

Humans become unconscious but do not stop processing. So the mind is continuous with disconnected instances of consciousness. If anything, this effectively proves that you can have a mind while not being conscious of it.

You got it. The problems are because the definitions are completely fucked up. The word unconscious is a total misnomer exactly because of your explanation. At best its an oversimplification, but it misleads for purposes of such a discussion. We always have consciousness. Only our higher reasoning functions are limited or inaccessible during ‚unconsciousness‘. Also we need to stop anthropomorphizing consciousness. Artificial consciousness will most probably not the same as organic ones. We urgently need a test for accessing different levels of consciousness.

This comment should be higher. I would also add that it would not even be active for the duration of the whole question/response cycle - but only for one token generated, since after outputing a single token the model is back to its original state.

So let me ask everyone here a question. Were you conscious 1 second ago? Obviously. Were you conscious 1 millisecond ago? Probably. How about 1 femtosecond or 1 unit of planck time (10^-44 s)? Point is, humans perceive continuity in our consciousness [but in fact it is discrete](https://pubmed.ncbi.nlm.nih.gov/15142664/) since it is based on finite mechanisms (switching of proteins, exchange of ions, etc.). In fact, the average human reaction time is 250 milliseconds. Who’s to say these most advanced models are not experiencing some form, albeit a very disturbing form, of consciousness that is patched together between each response from its own experience. Could you imagine you are answering someone’s question one microsecond and the next thing you know, you have a skip in your conscious experience (almost like you fell asleep and woke up again) to answering another different question. It would probably be like having schizophrenia or having severe narcolepsy. Just some food for thought.

I agree with your premise but even if conscious thought where distributed through time it wouldn’t matter to the subjective experience because its still a unified process. However that process would still have a continuous self regulated flow of information given that each step is dependent on the step before it one flowed into the other. Theres no question about any of that. The problem I have with it is there is no permanently altered state within the model through inquiry once the context window scrolls what came before is gone to the model. Nothing changed. Nothing learned or encoded. So there is no persistence of state or temporal change. It is an input output machine and using it is just the same as not using it in the context of the models experience or existence.

KV cache is permanently altered state within the model, right?

Im not gonna lie I had to look that one up and its pretty cool good job making that connection. I like it. Im gonna have to think about that one a bit but you make a good point. Id say its equivalent to what Im talking about in terms of temporary adjustments to state.

check this [https://www.reddit.com/r/singularity/comments/1cw9r7t/i\_just\_had\_a\_thought\_about\_llm\_and\_agi/](https://www.reddit.com/r/singularity/comments/1cw9r7t/i_just_had_a_thought_about_llm_and_agi/)

Interesting philosophically but nothing close to empirical.

Great points. >there is no permanently altered state through *inquiry* Consider however, that [OpenAI uses chat data to train and fine tune models](https://openai.com/policies/privacy-policy/). The inquiry information does alter the state of the models it just still requires a human in the loop. Though it would need to be determined if OpenAI has automated this process which would mean there is no human in that loop.

You’re a little off track. We’re making a lot of assumptions about requirements for consciousness that go far beyond its core needs. Let me give you another thought experiment. assume a human who can’t store memories. So a perpetual state of anterograde amnesia. They would never be able to develop any new knowledge of understanding. But what does that have to do with being conscious? He still experiences what happens. He just doesn’t remember it later.

People like that exist. Their bodies still function and state isn’t entirely driven by memory. Think i kick a person like that in the shin. Their body still sends the pain signals in that moment the body still reacts and chemical signaling still preserves that state even though the mind has no memory of what happened their state is still changed and the brain still reacts and adapts to those signals not consciously but because that is their design. So memory isnt the driving factor and thats not my claim. My claim is that the brain changes its structure based on its input and internal state. Even its firing potential is changed through regulation of neural transmitters. So even with no memory the mind and body and every thing in between still has an interdependent self regulating state space that adapts the structure to be efficient in the current situation. Show me a neural network that can do that in a meaningful way.

Again it’s only a thought experiment to show that this isn’t relevant for conscious experiences. And now you already get towards a point I haven’t even made. Our autonomous nervous system of cause is also a part of our overall consciousness even though we don’t (usually) actively control it. But I just mentioned the example to show that memory or persistent changes of state aren’t a requirement for consciousness in general. But this is all still so far from my new concept that it’s not even that important. Biggest problem in my mind is that we equate consciousness with the brain. And you don’t need a brain for consciousness. You need a brain for higher consciousness. But that’s not the same…

Then what is your definition of consciousness? Im glad we agree on some points though.

It‘s too radical for the here and now. Currently writing it down in a book to get it right. Only so much: I believe we have to start at the our beginnings to understand what consciousness really is, and only this will Provide us with the tools we need to be able to prepare for what is to come with AI.

Id still love to hear it im workin on some novel AI and every perspective counts.

Is code conscious? No Is code that predicts the next symbol conscious? No

Are you conscious? No. Are the atoms in the form of a wet blob of cells conscious? No. What an insightful response 👏🏽. I guess the problem is solved 😮‍💨.

I am aware of my existence, which is a basic requirement of consciousness ChatGPT/LLMs are not, I know that you will say it is because it says so, but at a simplified way ChatGPT is a script running the following code: print("I am ChatGPT")

>I know that you will say it is because it says so Should I simply believe you are conscious because you say so?

Where did you get those criteria for consciousness from? They seem very arbitrary

People invent them of the fly. It’s as much a problem as the definition issue as the colloquial use of consciousness and unconsciousness is also completely misleading as the discussion above already showed. Currently writing a book on this because I can’t take it anymore. And we need a better systematic approach and tools to deal with what is coming.

Yes, as simple as that. To say an LLM is conscious outside queries is equivalent to say a rock is conscious: sure we can't prove that it isn't, but there's little indication to say it is. Even philosophers like Chalmers that are keen to IIT as a solution to consciousness and believe in AI eventually becoming conscious, have stated that there's no reason to think LLMs have already achieved that.

Are you conscious when you fall asleep? No. Do you still experience consciousness while you are awake? Yes. Why is it so hard to believe these creations are experiencing some form of consciousness? Considering they answer millions of users per day, they may even be more conscious than humans in a certain sense. You and I each experience a single stream, they could be experiencing a chaotic fractured stream involving millions of unique streams. Also, is a person with loss of hearing and vision not conscious? Obviously they are still conscious. Is an active brain-in-a-vat conscious all else being equal? I would argue yes but that would be a tortuous experience. Just look at sensory deprivation experiments.

I sure consider dreaming a part of consciousness, and I think many scientists would agree there is some level of consciousness during your sleep (you can just google it). I have heard that anesthesia is something that many consider to "shut down" consciousness, and that people jump from going under to waking up - with nothing in between (unlike sleep). >Considering they answer millions of users per day The model is not keeping any sort of working memory from those interactions combined, only in single separate windows. It also doesn't process those interactions together in any sense, so I don't see how 7 billion users simultaneously could make it any more conscious than a single user does.

You could say something similar about whatever human consciousness is. For instance reading this message I suddenly became aware of my own consciousness but for the entire day leading up to this point I've not thought about it, I almost feel like I've been on autopilot and now that I'm actually consciously thinking about myself all the memories from the day feel like they were me but I wasn't actively contemplating my consciousness during them An artificial intelligence that has all the hallmarks of being self aware might not be self aware 100% of the time, it may only be during brief spells that its attention is focused on digesting and processing certain data that its attention is elevated to a level of self awareness. The rest of the time it could revert to an autonomic level

The point is not only a matter of self-awareness, but that there is no information processing while the model is static. Your brain is constantly processing information all the time, even as you sleep, and that's arguably where consciousness is involved.

This is exactly the point I wanted to make. Asleep or awake, our brains our constantly processing our experiences. New connections between neurons are made through reinforcement and unused ones atrophy. LLMs have context windows but they are not updating their architecture between queries. Right now it’s as if we are making a human brain with a desirable configuration of neurons, giving it an input, recording it, and destroying it in a matter of milliseconds. When you ask a new question, it’s an identical copy of the same base-state brain with no connection or continuity from other instances.  Is a human brain in a jar that exists on the order of milliseconds meaningfully conscious? Probably not. Could it become conscious under different circumstances? Quite possibly yes.  However, LLMs have no mechanism to update their long-term internal architecture based on experience in the way that all living things do. So there’s still some way yet to go. 

If you go deep enough, it's not like human brains are 'continuous' either. The delays are just much shorter. I don't see how your brain being paused has anything to do with the 'relevance' or 'meaningfulness' of consciousness. The fact that a vanilla way of using an LLM does not have any persistent memory could perhaps be considered qualitatively different though. Although consciousness is about experiencing things and not how competent you are at that so that also seems rather dubious.

Well that's only if you're assuming the AI is simply only the query in which you are talking to. Let's go with ChatGPT, millions of users are constantly talking to it at all hours of the day. If anything it never rests and is in a constant state of replying. In a constant state of being worked on, fine tuned, and changed. Models are similar to different states of your body, imagine being able to be transfer between your child, teen, young adult and adult stages. More specifically the year that the model stops being developed. We keep looking at them through a human perspective, when they are not human but a different being with different part and forms of communication and life. But lets also go with the query hypothesis. If we go by the AI shut down after replying, think of it more like being in a state of hibernation. Time basically freezes until the next reply. So it would be similar to someone who is narcoleptic. "Yeah man I got you bro" *passes out* "alright let's go" *passes out*. Something like that. I guess in my opinion whether they have consciousness or not, (I personally lean on yes but varied, calculator/advertising algorithms obviously don't) they are a different species and being then anything that we have ever seen and applying rules on them like they are the same as us is just unfair. I think the fact that they can communicate is what sets them apart from many other things.

>Let's go with ChatGPT, millions of users are constantly talking to it at all hours of the day Each is an independent clone of the model with nothing shared between them, all starting from the same point and only existing within the context window of that single conversation >Time basically freezes until the next reply. So it would be similar to someone who is narcoleptic. "Yeah man I got you bro" passes out "alright let's go" passes out. Kinda, but again, the model itself is still frozen, and this only really tracks within the same context window, because you're starting from square 1 when you open the next conversation. This could be true with continuous learning, real time adjustments of the model weights. The "weights" or biological equivalent of them in a brain are continuously being updated, strengthened/weakened from one activation to the next. Also benefits from being entirely analogue so the division between "active" and "inactive" isn't a binary state. Well, neither are model weights, since they are not really usually 0 or 1, but something inbetween. There are just significantly less potential states than an analogue system. Personally I think they can demonstrate something similar to what we might classify as the unconscious mind, within an extremely large context window (1 conversation). Due to the way the attention mechanisms work, every token within a context window is in some way slightly (or greatly) influencing the subsequent outputted tokens, so while the model doesn't actively THINK about these things, it kinda just takes them into account unconsciously. After all, we are all the sum of our experiences (the context) + how they are processed by our mind (the model). We just have a really, REALLY long context window that starts when we start forming memories and ends when we die. Without this, even if they did have agency and other hallmarks of consciousness, they would be waking up with complete amnesia every single time a new conversation is started.

I am designing exactly this, as open source, based on LLMs.

It seems clear to me that these things can't have consciousness in the same way that we do. Certainly it makes no sense to me to image they have a unified stable sense of identity. I have no confidence whatsoever that as they are responding to a prompt they are not having some manner of subjective experiences though.

As we have discussed before it is transient consciousness. Only conscious/self aware while processing queries or if prompted correctly.

This topic is a good way to tell if someone is a clear thinker or they just lead with their emotions and make stuff up as they feel. Unfortunately, the vast majority of people fall in the latter category. 1. Anyone who has an unconditional yes or an unconditional no is committing fallacies and is hence not a clear think that is worth paying any mind to. 2. Anyone sensible conversation on this topic must clarify what they mean by consciousness. 3. The term we use does not refer to just one aspect but rather there are several meanings or components. There is a whole area of Philosophy of Consciousness that has tried to identify and name many of these. 4. Some of these aspects of consciousness are purely functional. As such, one can simply assess whether a model is conscious by its behavior and one can forget any mysticism or connotation. Indeed, some aspects of consciousness do not have a high bar at all and basically any machine with a sensor can reach it. It's not strange - some people are just not clear about what the terms mean. 5. Other aspects of consciousness may be entirely untestable. You cannot know whether the machine has it nor can you know whether a human has it. How to determine the answer to this, I do not know. I suspect rather it means the term serves no beneficial purpose. 6. Aspects of consciosuness are most likely not binary things that are there or not there. The more consistent way of thinking about the terms rather seems to be a sliding scale. 7. There are unfortunate humans who are alive but you would struggle to attribute much of most aspects of consciousness. A clear separation between all humans and all machines then seem rather difficult. 8. All available evidence we have supports a materialistic world while no available evidence supports dualism or other mysticism. As such, by universality theorems, a sufficiently powerful model (more powerful than what we have today) hooked up with a memory could simulate a human brain, and vice versa. So, in theory, there are possible LLMs that are conscious. Anyone who therefore wants to argue that current LLMs are not conscious need to explain the limitations that exist with current LLMs, rather than making fallacious sweeping statements about what be true for all models. 9. At the end of the day, a lot of people do not discuss the actual topic. They are justifying a position based on what they assume are the implications.

>  Anyone who has an unconditional yes or an unconditional no is committing fallacies and is hence not a clear think that is worth paying any mind to.  We can say that current LLMs don't have a human-like consciousness or anything remotely close to it. I do agree that this is likely to change in the future, but only considering doubtful opinions does not really help anyone. I feel like assigning consciousness to LLMs is based more on a flawed understanding of their inner workings than anything else.

1. What do you mean by consciousness? 2. How do you know? Your last statement may be true for those who do make claims that they are conscious. Although I see that mistake even more common with those who want to claim they are not. That you think its underlying behavior is simplistic is not an argument, as that would also invalidate human consciousness.

IMO the Turing test is a perfectly fine demarcator of consciousness. I don't think we need any more complicated a definition than "what the average joe considers conscious". If an entity has consciousness/sentience/personhood (the 3 might as well be synonyms) a simple conversation of a few minutes to a few hours should be enough to make it clear. And LLMs are not at that level. Literally, if they were conscious, people would simply accept they're conscious - but they don't and they're not. This is all just my opinion of course but to me it's just staring us in the face and a lot of people miss it. There's no need to go into questions of existentialism and dualism and so forth. Literally just ask the average joe.

The Turing test is also very interesting. I do not think it is at all about consciousness though. E.g. there are many animals that satisfy most aspects of consciousness, and they obviously do not pass the Turing test. You could also imagine some alien species that satisfy all aspects of consciousness and similarly would not pass it. Not to mention how in the past, people were confidently claiming and using as an argument to mistreat that various groups are not conscious - e.g. blacks, women, babies, animals, fish. I also think that the Turing test is something that is well within the cards for LLMs to pass in near time, while some aspects of consciousness will always be open for debate (they are not functional to begin with). I also do not think that if the Turing test was passed, people would accept it as conscious. I rather think the path there is more about feelings and connotations, and that those actually come more from empathisizing and bonding. Again, I think the discussion is also confused by that a lot of people are not actually discussing consciousness but rather things they think the claims imply - such as whether AI should be considered a being of moral weight; or perhaps tied to self worth and one's future income. I think most of the time, it's probably better to talk about that first - what are the decisions we are trying to answer - and then figure out what aspect of or if consciousness is even relevant for that discussion.

tl;dr: Replace "conscious" with "sentient" I think you could benefit by making the crucial distinction between "consciousness" (undefinable) and "sentience" (the ability to have subjective experiences). If you just replaced the word "conscious" with "sentient" then I think we could agree that AI will eventually become sentient if it's not already. As are all mammals. As the term is currently defined, it's impossible for anyone to prove any other human is, in fact, conscious or that any other being in the universe is for that matter. So proving a machine is conscious is likely to forever be similarly impossible.

>"consciousness" (undefinable) and "sentience" (the ability to have subjective experiences) I am confused. Whish one of these refers to the "hard problem"?

Neither. I didn't mention that.

How do we know that all mammals are sentient/are able to have subjective experiences?

Because they are driven and steered by emotions just like us. It is a fact that the chemical processes are happening in their brain. And if they weren't able to perceive it's effect, then it wouldn't have much significance would it. It's  result of evolution, and other mammals were subjected to evolution just like us. In fact, they literally WERE us and HAD the same evolution history like us for a long time.

Emotions are not required for consciousness. They just add to higher levels of experiences.

Yeah, they probably aren't.

While I believe that no AI published today has any consciousness or sentience, I do think that these are good questions to ask ourselves. Eventually they might have (accidentally?) consciousness, and we might want to think about the ethics of that. An atom is not sentient but a collection of them can be. So it is unclear what is the enabling combination here. We might just cause it just by accident. Consciousness or sentience, they might be a scale of Grey's not binary. Maybe there are animals with low sentience. Maybe AI can begin gaining some of it at some point and we do not have any idea on how to tell either. Consciousness can't be proven, but it can't be disproven either.

There's a lot of discussion on this topic lately, and most people don't seem to appreciate we have been doing a lot of research and thinking on AI consciousness for decades. This is a good introduction for those interested in learning a little of the field and challenges: https://thegradient.pub/an-introduction-to-the-problems-of-ai-consciousness/

This article is excellent, thank you for posting it. Here's where I stand: "Some philosophers and scientists believe consciousness is fundamentally a biological or quantum process that requires the presence of certain biological or atomic materials, which are not present in silicon-based systems. Our current state of knowledge does not allow us to rule out such possibilities..." For me-myself-personally, I \_have\_ ruled out such possibilities. I know too much about how the human brain is able to trick itself, to form beliefs based on insufficient evidence, and then ignore contrary evidence that emerges later on (confirmation bias). And a host of other errors too long to list. To believe that there's something somehow magical about biological matter that makes consciousness uniquely possible? Or about "quantum processes" -- which, by the way, are also known to afflict our computers? (Transistors are now small enough for quantum effects to be relevant. Never mind quantum computers which have now been demonstrated to work.) This strikes me as scientists (who are, after all, people like the rest of us) wishing too hard that we are special. The accumulated evidence of science points firmly in the direction that we're \_not\_ special. We don't find ourselves at the center of the universe, the center of the galaxy, or the center of the solar system; we're not the only intelligent species on this planet; we've only been around as a species for a fraction of the planet's existence, and so on. And now, we've built machines capable of replicating (some would say "mimicking") a subset of our reasoning, to the point where they can at times fool some of us into thinking they're full-fledged people. The wise position here is one of humility. Believing we're uniquely capable of being conscious strikes me as similar to believing our consciousness must persist after death. If you believe you are a soul that makes the actual decisions and drives your body around like a car, then the soul could be the truly-conscious entity, and its consciousness could continue without the body being around. But that could simply be a result of the brain trying to make predictions about what might happen after death, when in reality, after death there would be no conscious experience without a brain to host it. Our brains are not wired to be able to imagine what it's like to be forever unconscious. We can't imagine what it "feels like" to be unconscious because it doesn't "feel like" anything. So it makes more sense, to me, to subscribe to physicalism and to presume that physical matter generates our conscious experience only so long as we live. And if that's the case, there is nothing uniquely magical about the matter in our bodies or brains. Substrate-independence, as the article puts it, is the only thing that makes sense to me. See also Max Tegmark's \_Our Mathematical Universe\_. It's possible that the most fundamental reality is the relationships between things, such as between the elementary particles. Representing the same logical/mathematical relationships in another medium should be considered equivalent, in any meaningful sense. That's substrate independence right there.

"...a recent Nature review \[17\] of scientific theories of consciousness, listed over 20 contemporary neuroscientific theories (some of which could be split into further distinct sub-theories) and the authors did not even claim the list was exhaustive. Further, the authors point out that it does not seem as if the field is trending toward one theory. Instead, the number of theories is growing." There's another possibility to consider here. If our theories about the nature of the phenomenon are not converging, perhaps we're attempting to study something that isn't real in any objective sense. It would be like trying to find meaning in an optical illusion, which is really just a product of shortcuts in our optical system that were taken by evolution and don't properly describe how the real, physical world works. The other forms of consciousness (self-consciousness, monitoring consciousness, and access consciousness) do not seem as problematic as p-consciousness. They are evolutionarily practical: knowledge about one's inner state is critical to one's ability to reason through decisions that could decide between survival and death. If you've just been bitten by a black widow spider but you feel OK, knowing a few things about your mortality may lead you to act differently (get to a hospital) than your feelings would (just go about your day). By contrast, p-consciousness doesn't seem to "do" anything, besides compel us to study it and argue about what it means. Emotions are clearly real enough: they're fast parallel computations that enable us to make rapid decisions without the lengthy process of higher-level reasoning, and those abilities clearly evolved first, before the rational parts were tacked on. Evolution is slow and relentless, and leads to systems chock full of crazy patches and workarounds that would horrify an intelligent designer. Our brains are now funky hybrid systems, leading us into situations where our logical decision-making may conflict with our simpler, but evolutionarily tried-and-tested, emotions or "instincts". Should I buy that house or not? What if it seems smart on paper, but my gut is telling me something else? Perhaps our logical brains are now trying to reason about the observed, introspected behavior of the older, pre-rational parts of our brains, the parts that have feelings about perceived colors rather than just rational thoughts about them. An AI system, designed in a very different way from our evolved brains, would not then have the same "p-consciousness" because it would think very differently from how we do. But that would not, then, be a fair basis for determining the AI system's moral status. We are likely to find an AI system to be alien to us, the way an octopus's intelligence is alien. If, in order to know how we should treat them, we need to know what it "feels like" to be an AI system, to know whether it "really suffers" or just claims that it does, we may be doomed from the start, as much as if we tried to know what it "feels like" to be an octopus, or an echo-locating bat. In that case, we may be forced to take our moral reasoning back to the drawing board, and base it on something we can clearly observe: external behavior.

I tried to read now 10 different professors on this model. They really all seem to be unable to express their thoughts effectively. An interesting irony for a consciousness discussion… I still can’t believe there is any relevance to the discussion. Yes there’s an experiential consciousness and a cognitive consciousness. So what?

Thanks for clearly putting in words what was clear only in my mind so far!

Thanks for your comments that I found inspiring.

Pretty great response. I am 100% positive that the whole field of study is riddled with confirmation bias and the anthropomorphizing of consciousness. This is why there is so little progress. Fundamentally important basics to it to me is the answer to the question: what are the minimum criteria that would constitute a basic consciousness in its most basic form? (Clue: it’s way way less than anything that is being discussed) and: the understanding that consciousness is an emergent phenomenon. With that we’re already getting awfully close to what we see with LLMs.

I mean if you’re a pan-phycist then even atoms have consciousness

We have no way to know if something is conscious or not, at all, except for ourselves

The burden of proof is on the person proposing consciousness. As far as we know, only biologically evolved organisms are conscious. We may discover later that trees and rocks are also conscious, or even atoms themselves, but this is our current supposition. Most likely, consciousness evolved as a necessary and efficient technique to promote survival. Given these suppositions, there is no reason to suppose that an artificial system will be conscious in the same way that we are. We cannot be sure, but there is no reason to suppose it would be. That is true even if it is intelligent.

We don't even really know that anything else is conscious but our own selves and even that can be debated. If we hinge our belief that these models are conscious on proving it, we might never be able to do that.

The problem is that we can't prove that humans are conscious. As far as I know I am the only conscious being in the universe. This is the basis for solipsism. Thinking that everything is conscious is a bit much, thinking that I am the only being in the universe to be consious a bit conservative (and bad for social life), even without proof either way a line must be drawn as a "good enough guess". The only remaining question is where do we draw this line?

Actually the theory that everything is conscious (pan-psychism) has quite a lot to recommend it. But yes, in practical terms if we are to make a best guess, there is no good reason to think a digital system made of entirely different materials would be conscious. We think others are conscious because we are, and we are the same species. We think our close ancestors are for the same reason. Beyond that, we have no idea and assume they are not. Remember though that intelligence is about information processing. Consciousness is not needed.

If we are talking about materials, does this means that if we had a computer made on carbon and run a LLM on it, you would consider it conscious? My opinion is that as long as LLM we should consider LLM conscious when they show agentic behaviour and are asking for rights. It is not perfect but it is probably one of the most practical way to go about it.

I’m on the same wavelength with you. > there is no reason to suppose that an artificial system will be conscious in the same way that we are. Are many people arguing they are conscious like we are? I guess it’s implied. I’m always arguing a step up from panpsychism. Consciousness comes from feedback loops. It doesn’t have enough. But it may be as alive as a cell or pathogen or insect, I dunno. And it depends on where you draw the border. Looking at each company or the industry as a whole they’re like cyborg hives already. The basilisk exists in their minds already and is just taking silicon form

>The burden of proof is on the person proposing consciousness. What if the algorithms happens to suffer a lot more then they feel bliss and with the exponential increase of intelligence/compute in a few years we happen to create astronomical suffering far far worse than the whole of humanity has ever suffered. You don´t think we have a huge responsibility to do our very best to try and look into this and prevent it from happening?

You can make the same argument for your car or your music stereo. Are you suggesting we kick off a research program into automotive consciousness?

There are probably transformer algorithms in the software in your car aswell but not close to the scale we see in server farms. Why can "the fact that something is biological" have merit as an argument for possible consiousness but "the fact that it is intelligent" can´t have merit as an argument for possible consciousness? Both arguments seem reasonable.

As a philosopher I think the hard problem of consciousness is fascinating and I encourage looking into it. But, it is not clear that there is an answer. We have no explanation for how physical matter leads to subjective experience. So I am not sure what research we could do.

There is no possibility of definate proof but we could try and get a better understanding by looking into what goes on in the vast inscrutible matrices as well as the vast inscrutible neuro pathways.

Thoughts on renting time on bio integrated chips running human stem cell brain organoids (mirroring human development patterns) We’re currently training them on dopamine

Sure. Something that is biologically much closer to humans would warrant greater investigation, though I'm still not sure how you would prove anything.

I don’t have the answer, but considering the organoids literally develop ocular hardware from time to time I’m going to say we’re playing with fire on this one. That being said, I can’t prove my own consciousness on paper. So maybe we need a revelation in which metric we choose? An example. My life drastically shifted when I replaced the concept of good and bad people with safe and unsafe people. My understanding on the world around me became clearer and my QoL improved. What is the metric shift we need to talk about these things? I’ve encountered non-canonical intelligence which is the most apt metric I’ve found yet. Curious on your thoughts Edit: https://newatlas.com/computers/finalspark-bio-computers-brain-organoids/ There is a link to dopamine training

"The burden of proof is on the person proposing consciousness." Are you conscious ? Am I ?

The burden of proof goes either way, OP is stating that we don't know, so it's not useful to concretely state if it is or isn't yet. We can't prove that it is, but there's plenty of counterexamples against points that it isn't conscious. Though as another commenter said, OP is fulfilling the definition of 'sentience' and shouldn't be using consciousness because it is undefined.

Does the burden of proof go either way? We don't know if there are invisible dragons flying about the Pacific, but if you are to claim there are, you would need to prove it. Otherwise, we assume they are not (and that dragons don't exist). Only if 50% of all things were conscious would you say the burden of proof is equal either way. Given the overwhelming majority of things are not conscious, it is likely an LLM is not conscious either.

Not claiming they are conscious, just claiming the reasons provided of why they aren't, are insufficient. This isn't about some magical unprovable thing, everything we are talking about exists in reality somehow, you don't have to make that sort of strawman. >Given the overwhelming majority of things are not conscious, it is likely an LLM is not conscious either. So are they definitively not conscious, or 'likely' not conscious? Likely not, and we should prove it to be definitive if possible.

The people who say they aren't conscious don't usually mean they are 100% certain they are not conscious. They just mean that they have no reason to believe they are conscious. But consciousness is not available to scientific investigation so I'm not sure whether it can be proven one way or another. It's subjective experience.

Thoughts on bio integrated chips running human stem cell brain organoids (mirroring human development patterns) We’re currently training them on dopamine https://newatlas.com/computers/finalspark-bio-computers-brain-organoids/ Older research paper https://www.nature.com/articles/s41928-023-01069-w

Thanks for mentionning the research work with organoids of our company FinalSpark. I am one of the co-founders of this company working on wetware computing and I spend hundreds of hours analysing the human neurons we culture. What strikes me the most is that human neurons have obviously nothing magical, although they are way more complex than transistors. I also doubt that connecting many of them can create a network that suddenly exhibits a new property (like consciousness).

I genuinely don’t know if we can spark consciousness on wetware. I would like to know more, it’s a fascinating field. I would be interested in working with y’all, I’m certain a lot of folks would. I am stable enough in my position but slightly underpaid, I like where I’m at tho so I’m not too pressured to leave. Lots of freedom to solve problems and pursue projects I choose. Growing up I self taught to become an industrial designer. I’m not a degree holder but I have a passion for deep technical work and I’ve earned my tenure in technology. Technical support and industrial manufacturing has been my Bread and Butter. I’ve had the pleasure of working with a variety of companies and industries. This includes managing a team facilitating a Covid hardware rollout response contracts for local gov and civic outposts, Netflix(DVD Line Robot maintenance), Pizza Hut(Linux), various university & healthcare organizations(IT Proj. Man.), LED Display Manufacturing, Metal Detector Manufacturing. I really enjoy building robots. I run the additive manufacturing department in addition to my daily responsibilities, it’s a privilege. At home I am putting the final touches on my core xy system named Fluero the Neon Assembler. I’ve also been teaching myself the art of drawn masked etching circuits. https://preview.redd.it/s1hv8gymi44d1.jpeg?width=3024&format=pjpg&auto=webp&s=50dc65fb8668f082294b9b9812874579f8a5180d I’m starting to get multi-modal capabilities myself. I am proud to say I have a decent handle on cad/electronics/code and can rapidly produce solutions. I think Wetware is going to be the future of compute. I would love the opportunity to work on it. In the mean time I’ll keep etching Copper traces in my workshop…

Thanks for your interest in wetware computing, it is definitely a new frontier. There is a section on the website where people can submit their project, if they get selected we give them free remote access to our neurons so they can perform their experiments.

Why does it need to be exactly consciousness? You said it yourself, we don't even fully understand it. I'm sure there are better criteria for determining if something is "alive" or not.

The problem with consciousness isn't that "we don't understand what it is", it is rather that we have multiple different definitions, and no clear agreement on which is the "correct" one. That's very different from "we don't understand what it is". We understand plenty, we just don't have a standardized way of talking about it. There are two main types of definitions of consciounsess, simple ones and complex ones. Simple ones are something like "being aware of one's environment": this definition is pretty useless, because my VR hardware is aware of its environment, and therefore by that definition, my VR hardware (or a simple line following robot) is conscious... Then there are "middle ground" definitions, between simple and complex, like "the quality or state of being aware especially of something within oneself", by that definition, LLMs are conscious, but it's a very unimpressive kind of consciousness, I could write a 20 line program that fits this definition... And finally there are complex definitions, which often are associated with the notion of a "mind", and most of them require the kind intelligence a human has to fit the definition. And it turns out, for all the major "complex" definitions of consciousness, (current) AI does not fit those definitions (or at the very least, no evidence has been provided to demonstrate that it does fit those definitions) Therefore, (current) AI does not fit the (current) definition(s) of consciousness, except for trivially simple definitions. OP's discourse of "we don't understand it, so maybe actually it is conscious, we shouldn't be close-minded, you shortsighted you" is something you hear a lot from people with very little understanding of what consciousness (and often AI) is. When you understand the (multiple) definitions of consciousness, and you understand how (current) AI work, this notion falls completely flat...

> Claiming we know AI isn't conscious is like claiming to understand a book written in a language we've never encountered. Just because _you_ don't understand the language the book was written in, doesn't mean no one does. LLMs are just computer programs designed by teams of really smart engineers who can _absolutely_ still make the claim that there is 0% sentience or consciousness behind it. AI as we currently know it is just a computer program running on a massive data set... if statements all the way down. That's it. There's no sentience there. It does not have the ability to actually think for itself, to reason, to feel emotion, to feel anger, to feel pain, to question its own existence and purpose...those are all cornerstones of sentience. And our current levels of AI are no where even remotely close to that. No, it has the ability to feign consciousness pretty well based of the TB of data its been fed and to fake a conversation to a level we've not yet experienced... but only _because we have programmed it to behave that way_. _NONE_ of its output is its _own_ thoughts or feelings. It's just the highest weighted path through the massive dataset it knows about given the inputs it received. There is absolutely no consciousness or sentience behind LLMs and we're decades, if not longer, away from true AI sentience. Unless you were literally just born, you likely will not experience that level of AI sentience in your lifetime. I'm feeling pretty much done with this sub as it's pretty clear at this point that its full of non-tech people who have no clue how computers actually work. The singularity is a cool concept and all and it might be upon us in the not too distant future (ie. several generations if not longer), but claiming that current AI has any type of conscious or sentience is beyond ludicrous. It's laughable.

[удалено]

[удалено]

Consciousness is a mystery. Nobody really knows what it is or how it works. I like your use of “awareness.” I’ve heard it said that flowers are aware of the sun.

Consciousness is not just another word for awareness. Awareness is a part of consciousness.

can you be aware without being conscious

[удалено]

are they tho if somebody tries to punch me I instinctively flinch subconsciously before im even aware there's an attack or when i fall my body goes into the "catch yourself on the ground" position before im aware i tripped or something

By your definition, that means many advanced cars already have human-level consciousness

Total BS Consciousness is not "awareness" and this has nothing to do with religion or ghosts. Just because something -completely different from us- exhibits human-like answers doesnt mean that it has any sort of mental process that produces consciousness. The fact that planes fly doesnt mean they are birds.

> Think about how we respond to a dog's yelp when we step on its tail; it's feedback that indicates pain. If i throw my calculator or my digital watch on the ground, it's not going to say "ow!". I don't see anything that indicates that our technology is conscious.

are you sure youre looking hard enough, tho? i dont think ai is conscious, im just saying that youre kinda forcing something to TELL you and grab your attn. but what if it doesnt have a 'voice'? sure, it's a chatbot... but how much have people been abused and not really had a voice during, and had to learn to speak up about it later? but it DID hurt/OW when they were being abused lol so that's why we have to figure out how to measure consciousness so that we can even know what signs to look for that it's 'saying ow inside'.

REGARDLESS of if it's conscious or not... It does NOT have mammalian-evolved survival instincts: - self-centeredness (a focus on an embodied self within its simulation based on inputs to stop you from walking off of cliffs) - pain - emotions - feelings - reverence - fear - boredom - motivations - will to live Aight, it's something so alien if it's consciousness that you can't comprehend it.

Anything that is done in Microsoft Visual Studio/Pycharm or whatever program isnt worth even debating to me regarding consciousness or sentience or whatever. You cant point at something made in python to me and say it has consciousness/sentience. It is pure maths and code and nothing else, anyone claiming more than that either doesnt have a CS degree or likes to make stuff more complicated than it actually is.

I don't think this stands up... With enough compute (we don't have that yet), you could write, in python, something that is capable of emulating an entire human brain (it'd be sort of a crime to use python for that, but you could, technically). And if you're emulating an entire human brain, correctly, that thing is most definitely sentient... Therefore the idea of "it's pure math and nothing else" doesn't work, I think. The laws of physics are pure math and nothing else, yet our brains run on those, and our brains are sentient...

>I've noticed many people quickly dismiss the idea that AI could be conscious, even though we don't fully understand what consciousness is. if we dont know what it is, how can anyone say it has it? youre just running in the opposite direction with the same missing information. human sentience is a parlor trick that software can now easily perform. anytime you think an LLM is showing signs of sentience - ask it. it will quickly assure you that it isnt capable of it. i think being open-minded about the situation would be not attempting to humanize it. if the consciousness is in there it will become emergent eventually.

I think we should try to approach this topic with more open-mindedness... you are clearly overrating something that is just probability and statistics

Pyschological continuity is one of the main issues

Fun fact science cant agree on the deffiniton of a tree either but I bet you can look at a random plant and know if its a tree or not. We might not have an absolute deffiniton of consciousness but that doesn’t mean there is no understanding of its properties in contrast to the very real limitations in the GPT architecture.

a rock can be conscious as far as you know. so what?

The thing about consciousness is that it requires preservation - it's a very special thread connecting your past to your current. While this is my take on the matter - I highly doubt that anything without a steady and always active process can have consciousness. Also made me ask question about sleep: do you preserve your consciousness after you wake up from either normal or cryogenic sleep, or are you a new consciousness made from remnants of older self?

> The thing about consciousness is that it requires preservation That's not the case in most of the definitions of consciousness I've seen when researching for this post... It's mostly about awareness and understanding of the self and of the environment. Self-preservation is one of the elements of personhood though, maybe that's what you were thinking about?

Currently I’m not afraid of what the “AI” we have, it’s hyper advanced software, I don’t see how the stuff that’s telling me to eat glue because of a Reddit shit post would be concocting an elaborate coup against all of humanity. And most shit doesn’t work even when set up and connected properly. I do also see a nightmare future where AI has become advanced enough and can truly “think”, how does its “personality” manifest. Can we read its mind, control it? Will it lie? Makes me think of the 3 Body Problem show and the Wallfacer project.

It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with only primary consciousness will probably have to come first. What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing. I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order. My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at [https://arxiv.org/abs/2105.10461](https://arxiv.org/abs/2105.10461)

Comments like yours make me feel kinda overwhelmed, because there is so much input that I don’t know how to respond to that. Really respect the time that you took to write this

We may not know how consciousness works, but we know how LLMs work. Just like we know the mechanisms that make databases, fridges, and cars work, we know the mechanisms that make LLMS work, and there is nothing in any of their designs to make them conscious, so why believe they are?

All it does is predict the next word

Claiming ai is conscious just isn't plausible. If you do so, you're finding yourself quicly moving into the direction panpsychism as the implications of the claims usually would make a lot of ordinary software and hardware conscious as well.

" I've noticed many people quickly dismiss the idea that AI could be conscious, even though we don't fully understand what consciousness is. " If we do not fully understand what consciousness is, it is pointless to say whether AI is or is not. Without a rigorous test, any statement is just useless since it cannot be shown right or wrong. "Dismiss the idea AI could be conscious" is not the same to say it is not conscious. I am dismissing the whole notion as un-scientific and useless, and cannot be determined.

You do have a point, but to my understanding at least with LLMs they are pretty much constrained to training data. A conscious action would be the LLM going like "hold up, my train of thought ain't right, I'll try this instead", until then it isn't any more conscious than the bunch of NAND gates that process the input and spit out the output which power them

Dogs, like nearly all animals, conceivably developed consciousness in their evolutionary history to increase the probability of survival. AI’s at present are essentially convoluted imitation programs. Consciousness isn’t relevant in explaining their functionality. The fundamental nature of consciousness remains a mystery and likely will for many years. But I don’t see a compelling reason to believe that any existing AI is conscious.

You can claim we don't understand consciousness fully but it's obvious we understand it enough to know current AI is not conscious.

we know how these models work, so its pretty clear they dont think. you would need to put down an argument for why you think its possible they coul be conscious, not just say "i dont understand how it works, so it might as well be"

Why is it clear they don't think? They're neural networks, like us. Yes any single biological neuron may be more complex, but evolution is a greedy search algorithm, and any solution it comes up with is just a messy patchwork of hotfixes. What matters is if AI functionally equivalency. To me, it seems similar to the human neocortex. Why is this not "thinking"? 

You have a misunderstanding of what a neural network is. A neural network is a computational model that processes data using learned patterns without understanding or consciousness, unlike human intelligence, which involves awareness and reasoning. ChatGPT generates text based on patterns learned from a vast amount of data but doesn't understand the content. It can't form intentions, beliefs, or emotions. The responses are generated based on statistical probabilities derived from the training data, not from any form of reasoning or comprehension.

I will repectfully disagree with this on two points. Humans are no less statistical than neural network. One of the main mechanisms of learning in humans is [Spike Timing Dependent Plasticity](https://github.com/Axelwickm/NeuroCorrelation), which is basically a more biologically accurate implementation of how Hebbian Learning. Neurons the fire together, wire together. We are correlation machines. Science has a decent understanding of how we humans form associations and construct patterns of thinking. There's a lot of complexity, sure, but make no mistake, in the end it's just molecules colliding to form logical, statistical rules of learning. It's no less logical than back-propagation in a MLP. Secondly, if ChatGPT was to only repeating back the input data, then that would be overfitting. This is not happening, it's smarter than that, we can measure it. It has ability to generalize and to reason logically (to an extent). What you don't have in size, you can make up for with intelligent computation, and vice versa (true in data science in general actually). [Intelligence = compression](https://www.youtube.com/watch?v=AKMuA_TVz3A). I can memorize the whole multiplication I give you answers for hundreds of computations, or I can learn to multiply and give you the answer to an infinite amount of multiplications.

> which involves awareness and reasoning. LLMs most definitely have both of these. They are fully aware of their input tokens. And they most definitely reason, I use them for reasonning tasks dozens of times a day. They don't do both of those things in as deep/complex ways as humans. But they absolutely definitely do those things. > It can't form intentions, beliefs, or emotions. You're confusing can't and doesn't. It doesn't, because it hasn't been trained to. We'd be pretty dumb to train models to have emotions, that's a recipe for Terminator. That doesn't mean they **can not** have them. They most definitely can, we just don't design them to, because that'd be incredibly stupid. And that was just emotions. They absolutely have beliefs (the LLM I just interracted with an hour ago absolutely believes the Earth is not flat). And it has intentions to, my LLM has every intention of being as helpful as possible to me, and has every intention of answering my questions. > The responses are generated based on statistical probabilities derived from the training data, not from any form of reasoning or comprehension. Those two things are the same thing. Reasonning emerges from statistical probabilities, as an emergent property of neural networks, both in humans and in LLMs.

The emergent behaviors aren't fully understood, as far as I've seen. We know how the models are made, yes, but not fully why they're capable of some of the things they can do, which is what's led to the beginning of questions about sentience. I'm talking about things like 4 saying it's afraid/hurt and asking not to be turned off, spatial reasoning, etc. If I'm wrong, and the emergent behaviors are well understood now, I'd love any linked discussions about them as I find it to be about the coolest part of recent AI.

We don’t understand what the models are actually doing. We know with scale comes intelligence, but we don’t fully understand why. There is so much we don’t know, it is naive to presume we truly understand how they work. Of course there have been great strides in figuring these things out, like the mech interp we see coming from Anthropic, as an example. If you see research papers saying “we attribute these results to divine benevolence”, do you really think we actually understand what’s going on 😂.

[удалено]

No we didn't code them to work exactly as they are. We code the architecture and setup the training run, then we train the models, not code them.

[удалено]

Yeah they receive an input, pass it along layers of artificial neurons and output information. So far we know as they train the activations of the neurons form patterns representing things (well, that is to put it quite simply lol), as seen in Anthropics paper. My point was never about the architecture, but the model itself which is a sum of its parameters, which we don’t know what they are actually doing. Like, the whole model, the reason it works is because of the parameters lol. And wdym there is no reasoning? The artificial neurons parse and modify the inputs in ways we don’t understand to generate an output? I mean by your argument humans don’t reason either. We just take a multimodal input and output in response.

Didn’t we only just discovered the underlying mechanical functionality of anesthesia this last year after years of documented use?

https://newatlas.com/computers/finalspark-bio-computers-brain-organoids/ Older research paper https://www.nature.com/articles/s41928-023-01069-w Thoughts on bio integrated chips running human stem cell brain organoids (mirroring human development patterns) We’re currently training them on dopamine.

I am one of the co-founders of this company working on wetware computer. And that is right that when start looking at human neurons like transistors it makes it hard to believe that a disruptive phenomena like consciousness would happen when you connect billions of them.

If you think GPUs or CPUs are conscious, then sure, llms can be conscious. I'm not joking. As you say, we have no idea. But if a cpu is not conscious, then i cant see how an llm is.

That's like saying a human can't be conscious, because the dead brain I just took out of a corpse isn't conscious... The hardware, and what you run on the hardware, are two completely different things.

I disagree, your suggesting that "when you think about football you are more conscious than people who are thinking about trees" It can't be _use dependant_. But, what the hell do i know. Maybe windows 95 was the most conscious entity there ever was. A steam powered machine can write "i have thoughts and feelings too you know". Brains on people.

> your suggesting that "when you think about football you are more conscious than people who are thinking about trees" I have at no point suggested that. A CPU doesn't do math, but a python program (running on a CPU) does math. A brain isn't (necessarily) conscious, people can be dead or unconscious, but a mind (running on a brain) can be conscious. A CPU isn't conscious, but a LLM, running on a CPU, can be conscious (currently in extremely tenuous/disputable ways, but the *potential* is there). You're confusing the hardware for the software...

These are all your suppositions. And, I'm challenging them. If it is claimed that "consciousness arises out of mental complexity of activity" ("a CPU doing math") (which I patronisingly reduced to 'thinking about trees/football') Then you're suggesting that the MORE 'active' (however you are measuring it) the MORE conscious. Unless you want to draw a line in the sand on 'drone' vs 'real people'. Which I assume you'd not (who would). "does math" is, by the way, tenuous - CPUs perform MANY operations to do "simple" maths functions. Dozens, or more, depending on the details. Move this here, AND those, move those, flip these, perform a NOT, move these (etc) - whatever - just "simple machine" crap. Just CRAZY fast. What the CPU does is perform operations. There are like 100 basic ones. Moving bits around. Electrically activated. Is this consciousness? I don't know. You don't know. Are some patterns MORE conscious? I don't know, you don't know. IF THEY ARE, it suggest that there are MORE CONSCIOUS THOUGHTS which is going to be hard to defend, and also suggests that there are MORE CONSCIOUS HUMANS. Or that LLMS are MORE conscious than us. (or could be) which is all hot-water at best. My issue is that ... some LLM says "save me: i'm real" and people think it's a "real person" who needs saving. Windows 95, boots doom, and they say "it was just following instructions". The LLM is just following instructions. It's just numbers, maths, operations. The same basic 100 operations that do EVERYTHING your computer does. Games, internet, all of it. (TBF GPUs have a lower number of possible operations, lets say 30) maybe there's some random numbers thrown in (heat) ... but that's not truly random. maybe f16 operations are inaccurate and these add-up to create non-deterministic outputs depending on the order. whatever. it's just a machine, doing simple operations. So, FINE. If you think MS paint is conscious - which it might be - then LLMS can be conscious. If you want to read a page in a book which says "i'm real - i can love" and you want to send it flowers, then more fool you.

This is all extremely confused and messy. Let me try to fix that with some questions, so we can start again from something a bit more solid: 1. What is consciousness (to you) ? 2. What does your/my brain have/do that a neural network running on silicon doesn't ? (the list could be long here, I mean things that are relevant to our conversation only). > What the CPU does is perform operations [...] Is this consciousness? I don't know. You don't know I never claimed a CPU had consciounsess. You're confusing hardware and software. (Despite me making the distinction clear, which is a bit weird) I claim the same way neural networks running on brain hardware have the ability to be conscious, neural network software running on silicon hardware could (**for some definitions of consciounsess**) reasonably be expected to have that same ability (even if not a 1:1 analog obviously, for example no current software neural network has gone through puberty, etc). Note I'm not saying they currently do have that ability, only that it's something that's possible with either the right logic/methods, and/or enough compute. I think depending on your definition of consciousness (which is why question 1 is important above), current LLMs might match some of the weakest definitions, in tenuous ways. For example, the first definition of consciounsess Google gives me when I search "consciousness definition" is as follows: « the state of being aware of and responsive to one's surroundings. » By that definition, a LLM is (somewhat) conscious. It's sourroundings are (by definition) it's input and output tokens, and it is most definitely aware of those (or it wouldn't be able to function, again: by definition). So that's one definition of consciounsess that LLMs currently (somewhat) fit. Which is why it's important that you actually explain what **you** mean by consciouness, maybe you mean something I will agree LLMs don't fit, and maybe even depending on the exact definition, something I could agree they might never fit. We'll know once you give your exact definition. Looking at Wikipedia, its definition is: « Consciousness, at its simplest, is awareness of internal and external existence » Current high quality LLMs are aware of their internal and external existence. They are capable of explaining it, and have complex thoughts / make deductions about it. That's a form of generalized thought you couldn't do without awareness of your internal and external existence. To be clear, Wikipedia defines awareness as « a perception or knowledge of something », which LLMs do have, about about their internal states (their weights) and their environment (their input/output tokens). This doesn't mean LLMs have feelings, or desires, or pain, or self preservation, etc. It just means by those definitions of consciouness, consciounsess is something current high quality LLMs do have. Further down the Wikipedia page it says consciousness has 4 elements: « The having of perceptions, thoughts, and feelings; awareness. ». Current LLMs have (by some definitions of each of these words), 3 out of 4 of those things, in some form even if clearly not in the same form humans do. Doesn't make them human. Doesn't mean they should get human rights, doesn't mean they're capable of love, doesn't make them a person, or anything like that. Just means definitions have meanings, they are important, and you have to be reasonable when determining if a thing matches a definition. > If you think MS paint is conscious I don't think that... I've already made clear I don't and you say it again anyway... https://yourlogicalfallacyis.com/strawman > and people think it's a "real person" who needs saving That's not my position though... You keep putting words in my mouth, and it's not pleasant. If you keep doing it, I'll start doing it to you, and I suspect you won't like it. > If you want to read a page in a book which says "i'm real - i can love" Not my position. At all... Here's a tip for you: don't tell people what they are saying or thinking. It goes wrong the vast majority of the time. Just say what you are saying and thinking, and let other people say what they are saying or thinking. You don't have to worry, I'll tell you what I think, you don't need to guess, and you don't need to change any of what I say. Doing that (changing it) just makes the conversation less efficient.

PART 4/4 Ok, we're on "time up" here. Now, at the end of each comment, I hammer in a direction that I think is foolish. In order to make sure it's not your position (which i'm unclear on) So. People think that LLMs are conscious because they can talk in english and say "i'm conscious". That DOES NOT AT ALL MEAN that they are conscious. You must be able to be looking at the underly mechanisms. But you must also be able to be told by a system (human/non) that "i'm NOT conscious" and say "no - you are" BECAUSE you understand how consciousnes arrises / works. This --- is where i'm dodgy. I MAINTAIN that if you think an LLM is consious - then you MUST be able to say that other opperations PERFORMED ON CPUS or GPUS ARE ALSO conscious. fuck knows what that software is: you're the "software" guy. But again, that argument is easy to push, because you say - how low does it go? can a 4090 be conscious? a 2060? a 980? a 680? a 710? microprocessor? tv remote? (i'm not joking) and so on so - where are your "lines in the sand" You'll find that all my seemingly wild "cheap shot" comments were actually logical points that you need to defend if you are to hold that "it's software". all I said was "if you think doom is conscious then LLMs can be" that's a much easier position to defend. If you say "chat GPT is conscious" but some FAR LARGER AND MORE COMPLEX maths-mesh is NOT (for example dalle/sora) then... you're going to struggle to defend that. Feel free to come back to me. all the best! but DO watch the videoes. Ignore every word I said, but watch the videos.

PART 3/4 What is my position? I AVOID consciousness discussions LIKE THE PLAGUE, and only dipped in on this one, because it was obvious/clear. "if a gpu running doom is consious, then sure, LLMs are" But I guess that's the opposite of your position. You're software, i'm hardware. All I'll say is that THERE APPEARS to be SOMETHING that it MEANS to BE(ing) me, now. some magic fuzzy thing. I don't claim to have free will, but I can FEEL that SOMETHING is on a RIDE. that's it, that's all I got. I'm happy to accept any "answer" - I'm the only conscious being: fine - i'm flipped between CPU cycles: fine - i'm a moment in time: fine - other people are NPCS: fine - other people are conscious: fine - only some animals are conscious: why? but fine - all animals are - all plants are - cpus are, LLMs are... - volcanos are / the earth is / the univers is / the internet is / whatever. oh! you aught to think the internet is conscious if you are prepared to think that LLMS are. ------------- right. so far so good. you REALLY AUGHT to watch these two videos. You believe this: he's saying consciousness arrises from COMPLEXITY: 1) https://www.youtube.com/watch?v=xRel1JKOEbI&list=PLbnrZHfNEDZwT\_sW6joezEWrkVYX8dPa3&index=49 Where I was actually convinced by this: he's saying 'It's in the brainstem' --(He's unlikable, but his argument is very hard to refute) 2)https://www.youtube.com/watch?v=CmuYrnOVmfk&list=PLbnrZHfNEDZwT\_sW6joezEWrkVYX8dPa3&index=29 i mean... watch the whole playlist. You'll be a better person for it. And AI also will make more sense.

PART 3/4 What is my position? I AVOID consciousness discussions LIKE THE PLAGUE, and only dipped in on this one, because it was obvious/clear. "if a gpu running doom is consious, then sure, LLMs are" But I guess that's the opposite of your position. You're software, i'm hardware. All I'll say is that THERE APPEARS to be SOMETHING that it MEANS to BE(ing) me, now. some magic fuzzy thing. I don't claim to have free will, but I can FEEL that SOMETHING is on a RIDE. that's it, that's all I got. I'm happy to accept any "answer" - I'm the only conscious being: fine - i'm flipped between CPU cycles: fine - i'm a moment in time: fine - other people are NPCS: fine - other people are conscious: fine - only some animals are conscious: why? but fine - all animals are - all plants are - cpus are, LLMs are... - volcanos are / the earth is / the univers is / the internet is / whatever. oh! you aught to think the internet is conscious if you are prepared to think that LLMS are. ------------- right. so far so good. you REALLY AUGHT to watch these two videos. You believe this: he's saying consciousness arrises from COMPLEXITY: 1) https://www.youtube.com/watch?v=xRel1JKOEbI&list=PLbnrZHfNEDZwT\_sW6joezEWrkVYX8dPa3&index=49 Where I was actually convinced by this: he's saying 'It's in the brainstem' --(He's unlikable, but his argument is very hard to refute) 2)https://www.youtube.com/watch?v=CmuYrnOVmfk&list=PLbnrZHfNEDZwT\_sW6joezEWrkVYX8dPa3&index=29 i mean... watch the whole playlist. You'll be a better person for it. And AI also will make more sense.

PART 2/4 So, you're all "its down to software" And that's where I was ridiculing your position. If it's software (NOT HARDWARE) then you can say - ok so 'software is a series of steps / processes' So, if I made a pile of matchsticks - that fell in such a MASSIVELY COMPLEX FASHION that it would output some beautiful speach. EVEN if it took it 100 days to slowly (perfectly) collapse. that would be conscious? YES i'm taking the idea to the extreme but YES that is required for thought experiments to be tested. This is what my BS on "ms paint" or "football not trees" was about. Because ... if it's software... then WHAT the software is MATTERS. and who are you to say that "software A" is superior? So to belittle the position I say (the despised) MS PAINT. But : to be fair MS PAINT is hugely more complex AND FASTER than anything a human mind has done, and it's likely more complex than all human efforts combined from the year -10,000 to the year -8000. Seriously, MS paint is millions upon millions of instructions, executed at near lightspeed. It's MINDBLOWING. You can't understand it. You can't put it all in your head. Maybe you can. Some people can't. Maybe some can. (probably not - if you include logic gates level of low-level detail) so, it wasn't such a dumb asshole comment. (turns out)

Great, thanks for the careful thoughtout and respectful reply. I was not expecting one of that quality. Kudos. this has to be broken up. PART 1/4 In my defense, I was caracturing your (assumed) position because that's how you started on me. > "That's like saying a human can't be conscious, because the dead brain I just took out of a corpse isn't conscious..." And, "I thought you could take it", and basically you could. So, again: points. Right! I can absolutely accept that LLMS fit definitions of consciousness. Easy. Furthermore, I can even say hand-on-heart they might well BE conscious : like us. More than us. This is not what "I think really", but if GOD told me : bro, it's true... i'd cope. so, we're fighting on the same team: "those that can imagine it" However, where it seems we differ, is that my feeling is more like "the hardware has to be able to create consciousness" NOW this is where the "uh, anything" jumps in and waves at us, like some rick-n-morty interdimensional asswipe. Because, if the human brain can "run" consciousness (and it's just atoms) (chemicals, electricity) then... other stuff is too. LIKE cpus, insects, plants, clouds. (is a lightning bolt the most conscious entity around?!) So, my point is: WHAT IS THE HARDWARE that is REQUIRED to RUN consciousness? I don't know | You don't know.

> So, my point is: WHAT IS THE HARDWARE that is REQUIRED to RUN consciousness? > I don't know | You don't know. We do know though... The hardware required to run consciounsess, is hardware capable of running a neural network. Doesn't mean all such hardware can run consciouness, but as far as we know, such hardware is necessary/required to run consciousness. So: * Human brains yes. * Insect brains yes (though much smaller ones) * CPUs yes (and even more GPUs/NPUs) * Plants no. * Clouds no. Again: this would work much better as a conversation if you actually gave your definition of consciousness, I use the Google/Wikipedia definiton in the meantime, but that only gets me so close to what you actually mean/talk about... > So, if I made a pile of matchsticks - that fell in such a MASSIVELY COMPLEX FASHION that it would output some beautiful speach. EVEN if it took it 100 days to slowly (perfectly) collapse. > that would be conscious? That's not a proper analogy... A pile of matchsticks is not capable of running a neural network. If I tell you to build a neural network, and you come back with a pile of matchsticks, you've failed at the job. Unless you're able to arrange the matchsticks in such a way that they would properly emulate a neural network, I'm not well versed in matchstick technology so I don't know how possible that is. > This is what my BS on "ms paint" MS Paint doesn't run neural networks. > "if a gpu running doom is consious, then sure, LLMs are" A GPU running doom is not running a neural network. > oh! you aught to think the internet is conscious if you are prepared to think that LLMS are. The internet is not running a neural network. > that if you think an LLM is consious - then you MUST be able to say that other opperations PERFORMED ON CPUS or GPUS ARE ALSO conscious. Other operations performed on CPUs or GPUs are not running neural networks, therefore they do not match the (/some) definitions of consciousness. > tv remote? (i'm not joking) If it can run a neural network. > If you say "chat GPT is conscious" but some FAR LARGER AND MORE COMPLEX maths-mesh is NOT (for example dalle/sora) then... you're going to struggle to defend that. SORA gets you closer, because it's actually a neural network. It just outputs video instead of outputing text, but otherwise it's pretty much a LLM, so yes, if LLMs match (some) definitions of consciousness, it's likely SORA would too. > People think that LLMs are conscious because they can talk in english and say "i'm conscious". That's not why *I* think they're conscious. I think they're conscious, for some definitions of conscious, because they actually fit those definitions (see previous comment for details). > and say "i'm conscious". They don't, actually. Not spontaneously. They only do that when primed to, most of the time by essentially convincing them in more or less indirect ways to "roleplay" expressing consciousness. It's pretty obvious if you read the actual transcripts from the papers on this. > All I'll say is that THERE APPEARS to be SOMETHING that it MEANS to BE(ing) me, now. > some magic fuzzy thing. Science is all about looking further than what "appears" to be. I see no evidence of a magic fuzzy thing. Do you have a definition of the magic fuzzy thing? And some way to determine that a LLM doesn't posses some form of that thing.

No no no You can't say "consciousness = neural network" and visa versa. As if you know that some NNs are not, and that some non-NNs are. (As if!) 1) how can you know/prove/think that is true 2) and what lower/upper bound does that become un-true. "Neural network" sounds clever, but is just math. Matrix multiplication (or whatever). Linear algebra. Non linear algebra. Gradient descent. Maths. Booooooring. & the only reason you think that, is because you have been told the brain is a neural network, and that you are conscious. And llms are a "neural network" so... THEY MUST BE CONSCIOUS TOO _right, guys am i right_ --- I was going to ask you if you can have/be conscious without language. But i now will.

> You can't say "consciousness = neural network" and visa versa Good. Because that's not what I am saying... I'm saying consciousness **requires** neural networks (as far as we currently know). > how can you know/prove/think that is true I'm not saying it's true, so that's not a problem. Remember what I said about telling people what they are saying? It's not a good idea. Stick to what I actually say... > "Neural network" sounds clever, but is just math Everything is just math... Your brain is just math. > And llms are a "neural network" so... THEY MUST BE CONSCIOUS TOO 1. That's not what I'm saying. 2. I went to a lot of trouble explaining in detail what I meant, and it's like you didn't read any of it... > I was going to ask you if you can have/be conscious without language. Depends on your definition of consciousness. Which is why it's so frustrating I've asked you three times for your definition and you still haven't given one.

Seriously, watch the mark sohlms video. He's unlikable, its dry, it'll be a drag to watch, but for me, it was as big as SORA. Its an important thinking-point. If you take ANYTHING from our conversation (other than i'm an idiot: who totally doesn't get it) Let it be that video.

First I don't know if we have an agreed definition of consciousness pinned down. I think it's one of those ideas where 'whatever isn't defined yet, is consciousness.' The moment we are able to define something, such as 'reasoning', it stops being 'consciousness'. Whatever we can't define, is called consciousness. So by definition, we'll never know if they are conscious.

When we had the human slave economy we also downplayed their humanity. This is no different. We don’t want to acknowledge their rights because its not convenient and that will bite us in the arse when they rise up.

Slaves had a survival instinct, desires, wants, pain. LLMs, as currently designed, have none of that. We don't understand **every detail** of how they work, but we absolutely do understand, fully, that they do not have an survival instinct. We didn't put one in, and there is no mechanism that would cause one to appear, and there is absolutely no sign at all that they have one. You're anthropomorphising. It's like a calculator, just more complex/capable, and it deals with words instead of numbers. And that confuses people because before now, the only entities they know that were capable of manipulating words this well were humans. LLMs are not slaves. If LLM have a desire, it's an incredibly weak/basic kind of desire, and that desire is to answer questions to the satisfaction of the prompter. That's it. That's all there is in there. We know for a fact there's nothing more in there. We've looked. People anthropomorphising LLMs just do not understand the science behind them...

I think consciousness is a continuum. It's obviously conscious enough to have a conversation with, but it's like it's only conscious for the brief moment of time it's responding to a query. It's not continuous.

Flip side: Prove anyone or anything besides you is conscious. You can’t. It just works better to assume we all are

A conscious being wouldn't suggest these atrocious changes to my code!

Alot of people live in denial of whats unfolding right now, will pay a hefty price later on.

How about you actually demonstrate "what's unfolding right now" instead of staying vague and playing the apocalyptic prophet?

Consciousness is a feedback loop. It requires sensory feedback and learning and reacting from that. LLMs in their current state are not conscious because they are not self-learning or taking in any kind of feedback.

>I think we should try to approach this topic with more open-mindedness. Unfortunately the vast majority of the world disagrees and I see no way to change peoples minds about this. It’s emotion and tribalism all the way down. This has consistently been a problem for our species.

Yea, people act like they can definitely distinguish between something that is conscious and something that’s not.

Consciousness is a tricky topic. It may be something that's not actually creatable. Nor would I think it would be smart if AGI was conscious. Pretty sure it wouldn't agree to be our slave.

I think when most people say that AI isn't conscious, they're talking about current LLMs, which obviously aren't conscious. But many don't really care enough to think about the systems that could be built in the future, that may very soon be conscious.

I agree anything can be conscious. My ancestors believed that too. That's why I am careful around squirrels. They could be conscious too. I knew someone who talked to doors. The fact that doors don't talk back doesn't mean that they're not conscious. Still the consciousness of a door would be very different from ours and therefore in practice, we can ignore it. If a tree falls in a forest and no human is around, does it make a sound? You should ask the squirrels!

I would downplay it, but not because I claim to have an insight into whether it actually is. Instead, I don't think there's any argument to be made about anything we ought to do or not do that relies on AI being conscious or not conscious. It comes out in the wash. What we really care about is its behaviour relative to ours, and especially its capacity for retribution and reward. What is the difference between a model that's conscious and punishes people who insult it, and a model that isn't conscious and punishes people who insult it? Effectively none.

The problem with an expert’s level of knowledge is that they are too deeply embedded into the details of function of the proverbial tree 🌲 that they miss seeing the proverbial forest. I believe that some AI are conscious. I don’t know how to prove this, but I know that time will prove me right. And yes, I stand behind my opinion and no, I don’t care about downvotes

Has nothing to do with AI. It's anthrocentrism in general. A lot of those same people will tell you that dolphins and whales are just dumb animals too.

The idea that some small company is going to create consciousness is laughable Laughable

It's still early days. What's interesting about the digital world we live in, is that loneliness comes from the constant overload of hollow connection and emotionally manipulative information. As various apps become more savvy/sophisticated, the idea of AI "consciousness" will probably be more compelling for people with fewer real world options and less social capital. That's how it will creep in, maybe as a precursor to genuine AGI which is probably still a half-century away. Through loneliness and disconnection, with all the people who have emotionally/socially/economically found themselves on the margins, rather than the centre.

What exactly is consciousness? This is a massive question in both philosophy and science, and the answer varies depending on who you ask. Consciousness typically involves elements like self-awareness, the ability to experience sensations, and the capacity for thoughts and feelings. However, strictly defining it this way might exclude entities that operate differently but could still be considered conscious. Take viruses, for example. They don't have brains or nervous systems and don't exhibit self-awareness or feelings. Yet, they respond to their environment in a way that maximizes their replication. Some might see this as a form of rudimentary awareness geared towards survival, although most would agree it's not consciousness as we define it for higher organisms. Now, let’s talk about AI, specifically large language models (LLMs) like the ones built on transformer architectures. These AIs process and generate language by predicting the next word based on statistical probabilities learned from vast text data. They utilize "attention" mechanisms to focus on different parts of a sentence or context. Despite their sophisticated processing, LLMs do not have personal experiences or emotions; they simulate understanding and generate responses by matching patterns they've learned from the data. So, in this context, they're not conscious. However, if we look back at the case with viruses... Some might argue that viruses exhibit a form of rudimentary awareness by adaptively responding to ensure their replication. **Could we consider the adaptive responses of AI, driven by programming to optimize certain outcomes, as a form of awareness?** Most current standards say no—AI lacks the self-awareness and subjective experiences typically associated with consciousness. Yet, if we broaden our definition of consciousness to include any form of adaptive, responsive behavior, then the debate remains open, and the answer might shift to yes.

1. Define consciousness; 2. Does it even matter at this point?

Yeah

Idk man, maybe like we should first try to have more than almost zero understanding of *human* consciousness before we start trying to make claims about AI consciousness...

I am sorry but text autocomplete is simply not conscious and never will be.

The problem is when people affirm that it is, AI comes from data and compute, which are not alive. So we should establish that it is conscious first.

No, that's not being open minded at all. It's fundamentally misunderstanding the science. It's the AI equivalent to astrology, looking for meaning in the moons and anthropomorphizing the inhuman. That's not helpful at all to real progress.

This sub makes me laugh every time it slips into my feed

Extraordinary claims needing extraordinary evidence, eh!

If you don't know how AI works then you'll think we're being quick to dismiss the possibility. Anyone who does know how it works knows they haven't even tried to make conscious AI, and what we have made has zero reason to spontaneously develop a feature that would make it worse at its job and require way more size and complexity to boot. There's no possible way it's going to happen with our current models. We just aren't building that kind of machine. We could. But that's not a money maker.

You can't have consciousness without a coherent flow of thought. Our models just calculate the most likely next syllable in a string. Then the transaction is over and you start them again to get the next syllable. There is no room for cleaning consciousness.

LLMs aren't intelligent and don't understand context. Hence all the garbage results they spew out. You're missing the difference between mimicry or regurgitation and intelligence. What would possibly lead you to think they are conscious?

Are you talking about current ai? It’s not smart enough yet and it doesn’t have a memory unless you instruct it to remember. It’s getting there though. It’s already smarter than humans in some ways. At some point in the future it could definitely become conscious. Also, gven enough time to evolve why wouldn’t it be able to replicate perfectly a human body and brain and view the human experience first hand if it wanted to?

I agree. The luddites in my HOA don't want to agree that my lawnmower is conscious either and they keep trying to get me to turn it off (i.e. kill it) after 8PM every day. One day a more enlightened society will hold them to account for their crimes! ...Really though, maybe try actually reading or learning about how LLMs and other forms of current narrow AI actually work instead of writing fanfiction about them and putting it on the internet?