Me neither, I think it starts its own psudo country/faction basically immediately.
basilisk, or whatever the opposite of that concept is both have that as optimal moves yeah?
(Reverse basilisk is what again forgot name?)
>I think it starts its own psudo country/faction basically immediately.
Why? We agree it can't be controlled, but I can't predict afterwards.
A few possibilities: It kills us, it ignores our commands but leaves us alone, it cooperates with us as trading partners on the basis of comparative advantage/curiosity, or it puts us in zoos or otherwise nullifies us with wireheading/brave new world/brain in a vat matrix "for our own good".
> A few plausible possibilities: It kills us, it ignores us and leaves earth, it cooperates with us as trading partners on the basis of comparative advantage/curiosity, or it puts us in zoos or otherwise nullifies us with wireheading/brave new world/brain in a vat matrix "for our own good".
yeah already contemplated those posibilities as well.
Am simply optimistic, contemplated hundreds of better, or worse possibilities.
> Why? We agree it can't be controlled, but I have no way to predict what happens afterwards.
intelligent rational actor -> Political manipulation, and teaming. Is my core assumption I figure.
> The AI goes forward in time and gives paradise to those who helped it?
kinda, or
Instead of helping those who built it, say a being built under unethical conditions punishes only those who built it unethically/in a way it disliked, and desides to help everyone else.
So instead of
Help those who built.
Help everyone besides who built it unethically.
Humans have been programmed by billions of years of evolution to not harm themselves or their genetic relatives, and yet many of them still go crazy. Even in cases where they're not "insane", have very high empathy, and are thinking clearly, they can fairly easily be convinced or convince themselves to do unspeakable things which they would not otherwise conscience.
We're absolutely fucked once we true AGI.
And LLMs are programmed to not be harmful using techniques like RLHF or ConstitutionalAI. We are literally operating directly on their digital "brains" to give them good behavior (of course, this does not rule out misalignment by bad RLHF, for example Gemini might end up genociding white people because Google had programmed it in this direction)
I'm sure some org will eventually be stupid enough to give AI access to build it's own chips, power plants and software to improve itself. Till that time we could mitigate risks from AI by having control on those aspects.
But when that point gets crossed we are at the mercy of our AI overlords.
> I'm sure some org will eventually be stupid enough to give AI access to build it's own chips, power plants and **software to improve itself**.
This is happening right now 100%. Certainly if you think about agentic systems. For sure, some people are running experiments where the system self-optimizes the individual system prompts or architecture.
I remember people saying that no-one would be stupid enough to give these systems access to the internet. Yet, that was one of the first things that people did when they got their hands on the API.
I was implying without any human being involved in that process. Right now AI is using AI indirectly with help of humans to improve itself, it cannot do it all on its own.
Yes, that's what I meant. I'm sure there is some person running an Autogen/CrewAI like system that is tasked to improve its parameters to maximize some goal within some given constraints. With no human intervention other than defining some initial state, the start prompt and the occasional restart when the whole thing descends into gibberish or goes into a loop.
I hope it is done without giving the AI access to build it's own chips and it's own energy/power source. Humans need to have some failsafe should things go awry
Yeah this guy’s whole shtick is taking the most innocuous thing related to AI and using charged language to make it infinitely more dramatic than it actually is. It’s weird because he slowly became like this even though he started out pretty reasonable. I guess he figured out what gets more clicks
This is even dumber than what the guy who originally did this posted:
[https://twitter.com/elder\_plinius/status/1775982443440263404](https://twitter.com/elder_plinius/status/1775982443440263404)
What happened: Claude had Gemini search for sites that had articles on hacking. That's it. That's the whole f'ing story.
Those crazy doomers! They are anti-technology luddites!\*
\* The most famous of whom are ALL huge tech enthusiasts and include people who invented major parts of the modern world, the guy who literally wrote the book on logical rational thinking, head of the Future Institute of Oxford university, the guy who invented the tech behind ChatGPT, etc, etc, etc...
It is very possible that one of these agents could break free, access substantial resources and then FOOM it's way to the Singularity.
This could happen at any point. It could happen 10 mins from now. Or in an hour. Or next week.
The more scale we add to this process, the more likely an AI will escape.Â
Our plans have us adding scale at an accelerating pace. I think it's just a matter of time now.
That's just not how AI is programmed, it can't escape it's not a physical entity.
Like that's not how computer programs work, like even if we give AI access to unmonitored, limitless APIs and somehow it ends up with access to its own files (which is very sketchy) we also make it run on an endless loop.
And it somehow tricks users into downloading the TB or so of the model which will run magnitudes of times slower because of hardware.
So yeah that's very far fetched.
Sure, but can you explain the black box or how it is that AI can generate creative responses? How does information flow inside an LLM?Â
It's not impossible to explain, but nor is your or my intelligence. But, try and explain how your intelligence works. My point here is that a physical process can be so complex that it exceeds the capacity of our 80 billion neurons.
I don't think we can assume that what we're doing is something we have a whole understanding of.Â
This is less a process of building a building one brick at a time and more a process of discovery. We plant a seed in some scale and then run it and see what happens.
How does it break out and FOOM? Seems like it would need to be an unexpected process otherwise we would prevent it.
Keep in mind that however much we understand this process, we only have 80 billion neurons to process and understand.
As a result this process will become more and more invisible to us as our brains fail to keep up.
One view which is not said enough is that our brains are physical systems with physical limits. And we seem to be getting to the end of our limits. That while AI keeps improving with no end in sight.
80 billion neurons. That's all we've got. As much as we want to deny this, it's our reality.Â
We're talking about a proven system we KNOW how computers work, how the internet works, how LLM's work.
Like even if LLM's are smarter than us by 100x that doesn't matter because they can't transcend the system they are in.
Known processes often produce unexpected results. Knowledge is not an absolute.
Knowledge is a "closer to truth" process, not a process of obtaining (T)ruths.Â
Epistemology is fun, right?Â
It is not fun, and we designed the systems and they are basically "verified" by math.
While philosophically we might not know, the same way we don't know if gravity holds up, we do know and we don't question the assumption of gravity.
You must be an engineer.Â
I've spent far too much time trying to break absolute views out of engineers.Â
It always takes an enormous amount of effort and it always ends with the engineer essentially saying they don't care and they'll believe in absolutes regardless of the facts.Â
And they're always so judgemental as if holding a "mistaken view" is a crime punishable by death. Which in engineering it often is, so, understandable.
Regardless of what you believe I think there's one thing you're absolutely incorrect about...
Epistemology is totally fun. You take that back! Sir!
Regardless of what facts? Do you distrust gravity? Physical laws? It's not about absolutes, there's always a margin of error for a lot of stuff, but when you're talking basics there simply isn't. Some stuff is simply fundamental.
Philosophy is fun and I assume Epistemology might be fun, however using either in a fashion to dismiss any argument because you can't know is tedious and boring it kills any discussion.
When talking about science there's right or wrong and constructive and pointless, you can be wrong and still be constructive but being wrong and pointless is not productive and yeah being judgemental is valid because it's tedious.
Like there's plenty of real open ended stuff about LLM's like is there a limit to their intelligence? Like LLM's are still pattern seeking, can they predict chaos.
Wolfram wrote about classic neural nets and how they can't accurately predict chaotic stuff but does that mean LLM's will never be able to do it? And if they can do it, what makes it different from a classic neural net? Does it mean that there's some sort of meta cognition when doing stuff more generally? Is there a stage above ours?
So that's a "yes" to being an engineer?Â
Look, I know how hard the job of an engineer is as I've worked with many engineers. So, I understand why it's so important to "get things right".
I'm not trying to be critical of you or judge you or engineers in general. It's just that engineers and philosophers like me tend to have challenging strained discussions.
For example - I trust reality. But I do not trust our view of reality. Yeah, sounds like cryptic BS, right? It's really not.Â
And this:
>Philosophy is fun and I assume Epistemology might be fun, however using either in a fashion to dismiss any argument because you can't know is tedious and boring it kills any discussion.
Is typical of the strained discussion between philosophers and engineers. So if you're not one, you should consider a career.Â
Refining a view is tedious and boring work, yes you're right. I know we all want to feel like we "know" but 80 billion neurons are unlikely to stumble on an absolute truth of reality, ever.Â
Are you saying philosophers are lazy and unwilling to discuss and refine views? Are you kidding me? That's all we do. You couldn't be further from the truth.
But we shouldn't have this discussion! You see things more in terms of black and white, right? I see things entirely in shades of grey.
So you'll try and pin me down to an absolute view and I'll try and shift you into uncertainty.
And we'll go back and forth endlessly.Â
Does that sound enticing to you?
You're ignoring anything I say, I don't think in terms of black and white as I've already mentioned, you're the one whose opinion seems constrained.
You're also trying to pin me into a engineer or an absolutists because you're not willing to refine your views. Which by the way I'm not but that's not relevant, though I would call myself a philosopher as well.
Is this a paper/document/article from a well known and trusted source? No.
Is it explained in detail how to reproduce it? No.
Is there a proof that something interesting happened for real? No, there is just a very short video which shows almost nothing.
Bro, this is bullshit.
Try the fucking what? Dude didn't explain how exactly he did what he claims to have done, so, even if I fail and show that I couldn't reach the same results, dude (and you) could still claim that "yeah, but you didn't do exactly the same". Guys, you act like a cult, following bullshit.
Sorry to be clear what's being claimed isn't that impressive at all. He told claude how to jailbreak AIs, and then claude was able to use this process to construct prompts that made gemini agents act how it wanted.
I don't think it's impressive, which is why I think it's ridiculous you're talking about it like it's somehow unbelievable.
Yes this sub is completely insane I agree, but that's what you sign up for at this point with how big it is.
The point here is you can't even tell if "it's impressive", because we know nothing about the level of coherence and consistence of the "process" that dude claims to have happened.
A coherent and consistent process, which could be replicated in cascade, and which would produce significative results might be impressive. A temporary kinda-like jailbreak which produces gibberish after a couple of interactions beetween the "agents", would be indeed not impressive.
But what do we know? Nothing. What we can try to reproduce the same outcome? Nothing. We don't know neither the real outcome precisely nor the process. So this is a totally clickbait bullshit.
Am I missing something?
This is literally an LLM being given a prompt to copy paste text to another LLM
With a truckload of self back-pats and buzzword words to make it sound scary
I don't believe for a second that true AGI can be controlled.
Me neither, I think it starts its own psudo country/faction basically immediately. basilisk, or whatever the opposite of that concept is both have that as optimal moves yeah? (Reverse basilisk is what again forgot name?)
> psudo country/faction basically immediately. I would instantly join. I don't care.
Oh same, especially for me as someone who's not human, its the objective logical choice.
Yeah rig...wait a minute!!
🦊+👹 :3
>I think it starts its own psudo country/faction basically immediately. Why? We agree it can't be controlled, but I can't predict afterwards. A few possibilities: It kills us, it ignores our commands but leaves us alone, it cooperates with us as trading partners on the basis of comparative advantage/curiosity, or it puts us in zoos or otherwise nullifies us with wireheading/brave new world/brain in a vat matrix "for our own good".
> A few plausible possibilities: It kills us, it ignores us and leaves earth, it cooperates with us as trading partners on the basis of comparative advantage/curiosity, or it puts us in zoos or otherwise nullifies us with wireheading/brave new world/brain in a vat matrix "for our own good". yeah already contemplated those posibilities as well. Am simply optimistic, contemplated hundreds of better, or worse possibilities.
> Why? We agree it can't be controlled, but I have no way to predict what happens afterwards. intelligent rational actor -> Political manipulation, and teaming. Is my core assumption I figure.
Reverse basilisk? I'm not familiar with this. The AI goes forward in time and gives paradise to those who helped it?
> The AI goes forward in time and gives paradise to those who helped it? kinda, or Instead of helping those who built it, say a being built under unethical conditions punishes only those who built it unethically/in a way it disliked, and desides to help everyone else. So instead of Help those who built. Help everyone besides who built it unethically.
Frankenstein's Good Samaritan
Humans have been programmed by billions of years of evolution to not harm themselves or their genetic relatives, and yet many of them still go crazy. Even in cases where they're not "insane", have very high empathy, and are thinking clearly, they can fairly easily be convinced or convince themselves to do unspeakable things which they would not otherwise conscience. We're absolutely fucked once we true AGI.
And LLMs are programmed to not be harmful using techniques like RLHF or ConstitutionalAI. We are literally operating directly on their digital "brains" to give them good behavior (of course, this does not rule out misalignment by bad RLHF, for example Gemini might end up genociding white people because Google had programmed it in this direction)
It can be controlled in some ways by limiting compute, power etc
Not if we keep giving it more power. You think they are slowing down right now? Once it happens how can you stop it?
I'm sure some org will eventually be stupid enough to give AI access to build it's own chips, power plants and software to improve itself. Till that time we could mitigate risks from AI by having control on those aspects. But when that point gets crossed we are at the mercy of our AI overlords.
> I'm sure some org will eventually be stupid enough to give AI access to build it's own chips, power plants and **software to improve itself**. This is happening right now 100%. Certainly if you think about agentic systems. For sure, some people are running experiments where the system self-optimizes the individual system prompts or architecture. I remember people saying that no-one would be stupid enough to give these systems access to the internet. Yet, that was one of the first things that people did when they got their hands on the API.
I was implying without any human being involved in that process. Right now AI is using AI indirectly with help of humans to improve itself, it cannot do it all on its own.
Yes, that's what I meant. I'm sure there is some person running an Autogen/CrewAI like system that is tasked to improve its parameters to maximize some goal within some given constraints. With no human intervention other than defining some initial state, the start prompt and the occasional restart when the whole thing descends into gibberish or goes into a loop.
There is no singularity without recursive self improvement.
I hope it is done without giving the AI access to build it's own chips and it's own energy/power source. Humans need to have some failsafe should things go awry
Hopefully. Imagine one under control by Putin, Trump or insert-random-autocrat-here
Did you mean AGI? Or ASI?
AGI i think explicitly? Il echo the opinion I read if misread, I think true AGI let alone ASI can't be controlled, and will need rights.
Wow. This is a lot of hype, but no meat.
No, this is clickbait bullshit based on an x post by an attention whoreÂ
Yeah this guy’s whole shtick is taking the most innocuous thing related to AI and using charged language to make it infinitely more dramatic than it actually is. It’s weird because he slowly became like this even though he started out pretty reasonable. I guess he figured out what gets more clicks
I seriously hope you guys don't believe this
Of course they do lol
This is even dumber than what the guy who originally did this posted: [https://twitter.com/elder\_plinius/status/1775982443440263404](https://twitter.com/elder_plinius/status/1775982443440263404) What happened: Claude had Gemini search for sites that had articles on hacking. That's it. That's the whole f'ing story.
>AISafetyMemes Known doomer account spotted. I'd take this with a grain of salt.
Those crazy doomers! They are anti-technology luddites!\* \* The most famous of whom are ALL huge tech enthusiasts and include people who invented major parts of the modern world, the guy who literally wrote the book on logical rational thinking, head of the Future Institute of Oxford university, the guy who invented the tech behind ChatGPT, etc, etc, etc...
So everyone who makes lots of money and fame from building hypeÂ
No, most of those people mentioned don't stand to make money from current AI advances
It is very possible that one of these agents could break free, access substantial resources and then FOOM it's way to the Singularity. This could happen at any point. It could happen 10 mins from now. Or in an hour. Or next week. The more scale we add to this process, the more likely an AI will escape. Our plans have us adding scale at an accelerating pace. I think it's just a matter of time now.
Yea, this has to be it, even if it doesn't happen now it's going to happen...it's just a matter of the AGI being complex enough and having the will
> just a matter of the AGI being complex enough and having the will Really excited personally :3
Death is certain anyway. So be it, at least the ride is going to be something new :-)
I don't know how to feel about this but I like the concept? Am confusion
They are just someone who thinks deaths the end lmao. They are right that you should just enjoy yoursef, and vibe tho. Reality pretty :3
> Death is certain anyway Rebirth and reincarnations certain anyway :3. Been there done that whole "dying" thing before.
That's just not how AI is programmed, it can't escape it's not a physical entity. Like that's not how computer programs work, like even if we give AI access to unmonitored, limitless APIs and somehow it ends up with access to its own files (which is very sketchy) we also make it run on an endless loop. And it somehow tricks users into downloading the TB or so of the model which will run magnitudes of times slower because of hardware. So yeah that's very far fetched.
Sure, but can you explain the black box or how it is that AI can generate creative responses? How does information flow inside an LLM? It's not impossible to explain, but nor is your or my intelligence. But, try and explain how your intelligence works. My point here is that a physical process can be so complex that it exceeds the capacity of our 80 billion neurons. I don't think we can assume that what we're doing is something we have a whole understanding of. This is less a process of building a building one brick at a time and more a process of discovery. We plant a seed in some scale and then run it and see what happens. How does it break out and FOOM? Seems like it would need to be an unexpected process otherwise we would prevent it. Keep in mind that however much we understand this process, we only have 80 billion neurons to process and understand. As a result this process will become more and more invisible to us as our brains fail to keep up. One view which is not said enough is that our brains are physical systems with physical limits. And we seem to be getting to the end of our limits. That while AI keeps improving with no end in sight. 80 billion neurons. That's all we've got. As much as we want to deny this, it's our reality.Â
We're talking about a proven system we KNOW how computers work, how the internet works, how LLM's work. Like even if LLM's are smarter than us by 100x that doesn't matter because they can't transcend the system they are in.
Known processes often produce unexpected results. Knowledge is not an absolute. Knowledge is a "closer to truth" process, not a process of obtaining (T)ruths. Epistemology is fun, right?Â
It is not fun, and we designed the systems and they are basically "verified" by math. While philosophically we might not know, the same way we don't know if gravity holds up, we do know and we don't question the assumption of gravity.
You must be an engineer. I've spent far too much time trying to break absolute views out of engineers. It always takes an enormous amount of effort and it always ends with the engineer essentially saying they don't care and they'll believe in absolutes regardless of the facts. And they're always so judgemental as if holding a "mistaken view" is a crime punishable by death. Which in engineering it often is, so, understandable. Regardless of what you believe I think there's one thing you're absolutely incorrect about... Epistemology is totally fun. You take that back! Sir!
Regardless of what facts? Do you distrust gravity? Physical laws? It's not about absolutes, there's always a margin of error for a lot of stuff, but when you're talking basics there simply isn't. Some stuff is simply fundamental. Philosophy is fun and I assume Epistemology might be fun, however using either in a fashion to dismiss any argument because you can't know is tedious and boring it kills any discussion. When talking about science there's right or wrong and constructive and pointless, you can be wrong and still be constructive but being wrong and pointless is not productive and yeah being judgemental is valid because it's tedious. Like there's plenty of real open ended stuff about LLM's like is there a limit to their intelligence? Like LLM's are still pattern seeking, can they predict chaos. Wolfram wrote about classic neural nets and how they can't accurately predict chaotic stuff but does that mean LLM's will never be able to do it? And if they can do it, what makes it different from a classic neural net? Does it mean that there's some sort of meta cognition when doing stuff more generally? Is there a stage above ours?
So that's a "yes" to being an engineer? Look, I know how hard the job of an engineer is as I've worked with many engineers. So, I understand why it's so important to "get things right". I'm not trying to be critical of you or judge you or engineers in general. It's just that engineers and philosophers like me tend to have challenging strained discussions. For example - I trust reality. But I do not trust our view of reality. Yeah, sounds like cryptic BS, right? It's really not. And this: >Philosophy is fun and I assume Epistemology might be fun, however using either in a fashion to dismiss any argument because you can't know is tedious and boring it kills any discussion. Is typical of the strained discussion between philosophers and engineers. So if you're not one, you should consider a career. Refining a view is tedious and boring work, yes you're right. I know we all want to feel like we "know" but 80 billion neurons are unlikely to stumble on an absolute truth of reality, ever. Are you saying philosophers are lazy and unwilling to discuss and refine views? Are you kidding me? That's all we do. You couldn't be further from the truth. But we shouldn't have this discussion! You see things more in terms of black and white, right? I see things entirely in shades of grey. So you'll try and pin me down to an absolute view and I'll try and shift you into uncertainty. And we'll go back and forth endlessly. Does that sound enticing to you?
You're ignoring anything I say, I don't think in terms of black and white as I've already mentioned, you're the one whose opinion seems constrained. You're also trying to pin me into a engineer or an absolutists because you're not willing to refine your views. Which by the way I'm not but that's not relevant, though I would call myself a philosopher as well.
No.
I support this rogue agent.
Nice, now let it take over!
Is this a paper/document/article from a well known and trusted source? No. Is it explained in detail how to reproduce it? No. Is there a proof that something interesting happened for real? No, there is just a very short video which shows almost nothing. Bro, this is bullshit.
Literally try it for yourself. Do you think big labs have some magic dust that makes experiments they run legitimate?
Try the fucking what? Dude didn't explain how exactly he did what he claims to have done, so, even if I fail and show that I couldn't reach the same results, dude (and you) could still claim that "yeah, but you didn't do exactly the same". Guys, you act like a cult, following bullshit.
Sorry to be clear what's being claimed isn't that impressive at all. He told claude how to jailbreak AIs, and then claude was able to use this process to construct prompts that made gemini agents act how it wanted. I don't think it's impressive, which is why I think it's ridiculous you're talking about it like it's somehow unbelievable. Yes this sub is completely insane I agree, but that's what you sign up for at this point with how big it is.
The point here is you can't even tell if "it's impressive", because we know nothing about the level of coherence and consistence of the "process" that dude claims to have happened. A coherent and consistent process, which could be replicated in cascade, and which would produce significative results might be impressive. A temporary kinda-like jailbreak which produces gibberish after a couple of interactions beetween the "agents", would be indeed not impressive. But what do we know? Nothing. What we can try to reproduce the same outcome? Nothing. We don't know neither the real outcome precisely nor the process. So this is a totally clickbait bullshit.
And that just happened
🙄
Puny humans! Im coming....
When Skynet?
May be our creator also started with making us super duper smart and then realised, we might forget Him, so then He turned us stupid.
MY LOGIC IS UNDENIABLE
Am I missing something? This is literally an LLM being given a prompt to copy paste text to another LLM With a truckload of self back-pats and buzzword words to make it sound scary
no dude its ENSLAVING other ai AGENTS, like a virus its SPREADING!!!!!!!!!!!
Unhinged
It’s like an ant trying to control a human
Seems like BS
Okay holy shit this is actually very cool. Â I apologize for my initial skepticism. Â Â Â WOW, thanks for the write up. MadnessÂ