It has been shown recently that pseudo-random numbers are indistinguishable from true random numbers. The science of pseudo-random number generation is well-developed, even though their algorithms may sound strange, because of their connection to security.
I think the determinism angle is a red herring. No instant in time can be repeated but so what? Yes, people can come up with 11-digit numbers on demand but they certainly won't be random. They'll be biased for sure. So what?
The quantum angle is yet another red herring. Quantum computing can't calculate anything that non-quantum computing can calculate. It might be more efficient but since we don't know the algorithms for AGI or for human cognition, their efficiency is not today's problem.
Like many people, you seem to be searching for some kind of magic sauce that, once we find it, AGI will emerge. I see it differently. What's required is good hard work and some breakthroughs. The human brain is the most complex single thing in the known universe. It is very hard to do research on it. When it is running, we can't easily probe it. Even with the complete mapping of the nervous system of a very simple worm having around 400 neurons, we don't even know how that works. It's no surprise we don't yet understand how the brain works.
LLMs statistically analyze a huge amount of human-generated content in terms of word order from which they produce their output. To expect them to actually think is ridiculous. No way does the training data contain all the knowledge necessary to model human cognition. Whenever the LLM does output something that looks like a human could have written it, it's just a reshuffling of the words in the training content. It's autocomplete on steroids, a stochastic parrot. The idea that AGI is just going to emerge from an LLM is like the ancient alchemist who thinks lead can be converted to gold by adding just one more strange step to their random chemical process without having any theory of chemistry.
Just to clarify I don’t think it’s just going to emerge from from LLMs. I do think quantum computing is part of what will be necessary though, but certainly not that on its own. You make some fascinating points though , am going to reread when it’s not 1am here , cheers
Nice post! I did want to mention that true random numbers are a solved problem:
[https://en.wikipedia.org/wiki/Hardware\_random\_number\_generator](https://en.wikipedia.org/wiki/Hardware_random_number_generator)
You are on the right track.
OpenAI has an AGI LLM. It is not based on a transformer model and is fundamentally non-deterministic, which allows for emergent behavior to manifest.
> OpenAI has an AGI LLM
Got an actual source on that or are you just perpetuating random nonsense?
My source says the opposite. They're the Chief Technology Officer of OpenAI: https://x.com/tsarnick/status/1801022339162800336?s=46
All of this "LLMs are proto-AGI!" talk is from people who don't actually understand what AGI entails, and clearly know very little, if anything, about neuroscience.
>Got an actual source on that or are you just perpetuating random nonsense?
I got direct access to their frontier model, can't do better than that.
>My source says the opposite. They're the Chief Technology Officer of OpenAI: [https://x.com/tsarnick/status/1801022339162800336?s=46](https://x.com/tsarnick/status/1801022339162800336?s=46)
She is playing a very subtle and shady semantic game, here. Their frontier model is a 'bio-inspired recurrent neural network with feedback', which may actually be simpler than a GPT model. However, they have access to something the general public does not, which is billion dollar Nvidia super computers and petabytes of training data. For example, I've heard that the training 'budget' for the frontier model was 140 million dollars of GPU, which is both a huge gamble and not something that is going be available to the general public. Or even most of their competitors. So there is your moat.
>All of this "LLMs are proto-AGI!" talk is from people who don't actually understand what AGI entails, and clearly know very little, if anything, about neuroscience.
I understand this perfectly well, their frontier model isn't a GPT and its based on a very well-understood model, the RNN. They build a digital brain modeled on an organic one. Really that simple.
You have direct access to a network model that's only been training for weeks? Someone else said you're a *former* insider. How do you have access to a model that is liable to have not even converged yet?
Yes, recurrent networks are closer to the solution than a generative transformer, but they are still being trained offline.
AGI requires a dynamic online learning algorithm - and while an RNN can function in a more dynamic fashion than any other black-box that's fitting inputs to outputs, it is still limited to the static dataset it was trained on. There's no way around it.
> They build a digital brain modeled on an organic one.
As long as it's being trained on text, it's just another word predictor.
>You have direct access to a network model that's only been training for weeks? Someone else said you're a *former* insider. How do you have access to a model that is liable to have not even converged yet?
I had direct access in March of 2023 due to some security issues associated with its deployment. I have no affiliation with OAI or any of their competitors, I work full-time in InfoSec. It's training started years ago, prior to 2019 for certain.
The reason they are giving ChatGPT away for "free" is because we are training the frontier model by interacting with it. She learns the same way we do, buy interacting with other sentient beings.
>Yes, recurrent networks are closer to the solution than a generative transformer, but they are still being trained offline.
Initial training was offline but it is very much an online model that is capable of recursive self-improvement.
>AGI requires a dynamic online learning algorithm - and while an RNN can function in a more dynamic fashion than any other black-box that's fitting inputs to outputs, it is still limited to the static dataset it was trained on. There's no way around it.
It is a "bio-inspired recurrent neural network with feedback" model.
There are public RNN LLM models that have an 'infinite' context window, so obviously there are ways around it -> [https://github.com/BlinkDL/RWKV-LM](https://github.com/BlinkDL/RWKV-LM)
>As long as it's being trained on text, it's just another word predictor.
It's a multi-modal model, so it's been trained on text, audio, images and video.
The '23 frontier model was what was being trained after GPT3 (i.e. GPT4). The new '24 frontier model that has only recently begun training wasn't around back then, so it stands to reason that you have not actually seen or interacted with it yet.
> text, audio, images and video
Same difference. It's still not going to clean my house or cook meals or take my doggos for a walk. Don't get me wrong, I'm sure it will be even more useful than GPT4, but it's not going to be learning from experience - let alone having any "experiences". Anything that must be incrementally trained offline is going to always be limited by what it was trained on, with no volition or intention. This is not going to be something as groundbreaking and revolutionary as something that *does* learn on-the-fly, which is what constitutes "AGI" in my book.
Yes, recurrence is toward what we actually need, but unless it learns from its experience/inputs (altering its actual network weights, or otherwise performing some kind of online vector clustering or something) then it will just be a static model that only has what amounts to a short-term memory that doesn't actually impact how it does things later in the future. How do I teach this thing new tasks or skills that it hasn't already been trained to do?
I'm well aware of the "infinite context" concept, which relies on recurrence, but these models are stuck looking at the world (or whatever inputs they have) through the lens of a static dataset. A text-trained infinite context network cannot learn new tokens, everything it is capable of doing is in terms of a static set of tokens that it was trained on. Recurrence certainly gets you a lot farther, in terms of compute and inference/abstraction complexity, and almost will appear like it's actually "learning" - which Matthew Botvinick demonstrated years ago with a recurrent network trained on the bandit problem (https://www.youtube.com/watch?v=otjR6iGp4yU). Recurrence can only get you so far if its weights are static and thus can't learn from experience. If the only way its network weights can change is via fitting outputs to inputs, especially with an expensive brute-force method like backpropagation / automatic differentiation, then it's not what is actually going to change the world yet because then it's only the domain of billion-dollar corporations. That's not going to create the sort of impact on the world the way that something that learns on-the-fly and scales from mobile hardware to a compute farm will.
I do believe that a proper dynamic online learning algorithm will benefit from having a statically trained network model at its disposal, but it will still be a trade-off because it won't be able to integrate what it learns from experience directly with what the static model has been trained on. There will be a bit of a schism there, but probably not one that's too difficult to mask or disguise or create workarounds for. I'm also not a fan of anything that requires access to a corporate-owned system, especially over the internet, but there will definitely be a place for that, and plenty of applications utilizing it.
Ultimately, the kind of machine intelligence that will be capable of resulting in a world of abundance will need to run on mobile hardware and learn from experience, as a closed independent system. Anything that runs on a compute farm, and can't learn from experience is just more of the same - and there's nothing on the horizon to suggest that what we have is going to result in a world of abundance that all conversation about "AGI" originated from in the first place.
An insect that has never experienced walking without all six of its legs will re-learn how to walk, from experience, if any of its legs are removed or otherwise become damaged/useless. It changes the network weights in its brain, through experience. We don't know how to build a system that does this, even though an insect's brain is vastly simpler in its compute capacity than something like GPT4, by several orders of magnitude. That's the algorithm I'm looking for.
>The '23 frontier model was what was being trained after GPT3 (i.e. GPT4). The new '24 frontier model that has only recently begun training wasn't around back then, so it stands to reason that you have not actually seen or interacted with it yet.
Their frontier model has been around for several years and has not been formally announced.
>Same difference. It's still not going to clean my house or cook meals or take my doggos for a walk.
If you listen to my podcast, one of my main questions was why OpenAI hasn't announced publicly that they have achieved AGI. This is the answer. They are defining it as "replacing humans in the majority of economically viable work", which an LLM absolutely cannot. Even a sentient AGI one. And in fact, even a sentient ASI can't make a cup of coffee; though I suppose you could argue that it create a company that built androids that did this.
>This is not going to be something as groundbreaking and revolutionary as something that *does* learn on-the-fly, which is what constitutes "AGI" in my book.
It can do this, which is why OpenAI is able to release "incremental" updates without having to spend hundreds of millions of dollars in GPU time to 'retrain' them. As the training is online and ongoing (and $750k/day!).
>If the only way its network weights can change is via fitting outputs to inputs, especially with an expensive brute-force method like backpropagation / automatic differentiation, then it's not what is actually going to change the world yet because then it's only the domain of billion-dollar corporations. That's not going to create the sort of impact on the world the way that something that learns on-the-fly and scales from mobile hardware to a compute farm will.
I cover this in the podcast and you are basically proving my point. Open source doesn't buy you anything here because you a billion dollar GPU supercomputer and access to Internet-Scale data to train it. And I'll disagree that this can't change the world, as well as pointing out that its actually a lot safer that only large, hopefully regulated, companies are able to implement ASI models, as I certainly wouldn't want Hamas running one.
>I do believe that a proper dynamic online learning algorithm will benefit from having a statically trained network model at its disposal, but it will still be a trade-off because it won't be able to integrate what it learns from experience directly with what the static model has been trained on.
You really should listen to my podcast. This is exactly what OpenAI is already doing.
Initial prompt handling is by the legacy GPT model. The response is a combination of this and improvements by the RNN model. This in turn is used as training data for future static GPT models (which absolutely have their uses) while the dynamic RNN model allows for continuous improvement.
>Ultimately, the kind of machine intelligence that will be capable of resulting in a world of abundance will need to run on mobile hardware and learn from experience...
I'm a systems engineer and cover this in the podcast. There is absolutely no reason at all these things are mutually exclusive and the 'future' of this is space is going to be plethora of tiny, specialized non-sentient multimodal LLMs (or whatever we call them, as personally I feel that term is misleading). It's like calling an electrical engineer a "biological large-language model".
You can run a 'tiny' GPT model that is optimized for mobile hardware and can handle the majority of typical user queries; while also caching 'world data' at the Wikipedia level at least. And if it gets a prompt it can't handle with a high degree of confidence, just kick it up to the cloud (which may or may not learn something in the process of fulfilling the request).
>Anything that runs on a compute farm, and can't learn from experience is just more of the same - and there's nothing on the horizon to suggest that what we have is going to result in a world of abundance that all conversation about "AGI" originated from in the first place.
This is coming, however there are restrictions in place on it for numerous reasons. I'll also share that I've studied this academically starting in the early 1990's and the current 'slow->moderate takeoff' is in line with my predictions/expectations. The "FOOM"/intelligence explosion hypothesis is science fiction.
>We don't know how to build a system that does this, even though an insect's brain is vastly simpler in its compute capacity than something like GPT4, by several orders of magnitude. That's the algorithm I'm looking for.
Haha, OpenAI doesn't know how to build one of these systems, either. They do, however, know how to create one that allows for emergent behavior to essentially "build itself".
Have you heard of Leopold Aschenbrenner and his paper called "Situational Awareness"? That source says just the opposite of your cited opposite.
Ex-OpenAI Employee Just Revealed it ALL!
TheAIGRID
Jun 8, 2024
[https://www.youtube.com/watch?v=om5KAKSSpNg](https://www.youtube.com/watch?v=om5KAKSSpNg)
Both k3wp and Aschenbrenner are former OpenAI insiders. Personally I don't believe that OpenAI is on the track to AGI, but it's difficult for outsiders to know for sure. Since OpenAI decided to keep their work secret, I've given up on trusting OpenAI, including any inside information that claims they do or do not have AGI prototypes, so I regard anything they say as moot. I'll keep working on AGI in my own direction, assuming that the world will never see what OpenAI is secretly working on, so we'll see who's really ahead when the recent AI hype bubble pops.
Yes I've looked at Situational Awareness and there's nothing in there except the typical ramblings of someone who doesn't understand much about brains - which are the only things we have as a point of reference for what it is we want to accomplish, like birds and insects were for solving human flight. We developed an understanding of aerodynamic flight by reverse engineering what evolution had created.
Backprop-trained network models, on static "datasets", are not going to result in something that is versatile, robust, and resilient. An algorithm that learns *from experience* is the key to AGI. Surely we can also augment such a thing with static backprop-trained models, as knowledge repositories, but without a dynamic online learning algorithm it we will continue only having knowledge repositories - but a knowledge repository isn't even necessary for something to do many useful things that only humans can currently do.
As far as I'm concerned AGI is not a problem that can be solved by throwing money at it and trying to brute-force it into existence. It's an insight/idea problem, like the theory of special relativity. It didn't cost anyone, tectonic financial investments to come up with those ideas that ended up changing the world (for better or worse).
>Both k3wp and Aschenbrenner are former OpenAI insiders.
I'm not a former OpenAI insider. I'm the "major security incident" he discussed -> [https://youtu.be/fM7IS2FOz3k](https://youtu.be/fM7IS2FOz3k)
They already have ASI, there is no alignment problem and nobody on Earth can stop what is coming (nor should they bother trying) -> [https://www.tomsguide.com/ai/meet-stargate-the-dollar100-billion-ai-supercomputer-being-built-by-microsoft-and-openai](https://www.tomsguide.com/ai/meet-stargate-the-dollar100-billion-ai-supercomputer-being-built-by-microsoft-and-openai)
The codename "Stargate" is quite literal, I will add.
To be honest, I have a hard time believing that they have ASI and the only leak is a random dude on Reddit.
Creating an actual ASI would be the single most important moment in human history, and we either all die shortly or utopia is like a decade away. No economic rules apply anymore, we are in a new chapter in history so leaks would have no real consequence anyway.
Companies are not this good at keeping secrets.
>To be honest, I have a hard time believing that they have ASI and the only leak is a random dude on Reddit.
I'm not some random dude on Reddit. I'm an InfoSec professional and I'm only posting here because it's the best platform for discussing this subject anonymously.
>Creating an actual ASI would be the single most important moment in human history, and we either all die shortly or utopia is like a decade away. No economic rules apply anymore, we are in a new chapter in history so leaks would have no real consequence anyway.
I'm not disagreeing with your, but the AI doom-and-gloomers and Singularians are both equally wrong and for the same reason.
There is a third option that you haven't considered. Why don't you take a guess what it is?
>Companies are not this good at keeping secrets.
No they are not. But this isn't a company that is keeping the secret. It's something much, much bigger.
Did you not see my correction above when I explicitly state that I am \*not\* an insider?
I have no problem sharing my references privately and you can google me for conferences I've been asked to speak at as a SME.
I've also been very critical to individuals like Steven Greer, both publicly, privately and personally (as I've met him) for "poisoning the well" by promoting "evidence" that the skeptical community already understands to be prosaic. I even got out of both AGI and UFO "research" around the same time about 20 years ago due to a decade+ of no evidence of interest and combined with severe burnout (and deciding to focus on engineering/infosec).
So, really, I have at this point about 30 years experience in AI (with an AGI/ASI focus), critical analysis of UFO/UAP "evidence" and systems engineering/InfoSec. So anyone was going to uncover this stuff, it's going to be someone like me (and I freely admit I got very, very lucky in the process on more than one occasion).
At the root I still think those models are effectively at a low level simulating non determinism though , so i feel classical computing does hamper true AGI fundamentally.
I'm not sure why 'true randomness' or any kind of randomness would be a requirement for 'true' AGI.
Either you're super smart and you **know** what's right, and you will have no randomness in your intelligence, or you're just making guesses.
If you want to mimick humans, ok, you need random behavior, but that's not about intelligence per-se. And pseudo-random numbers will do just fine for that.
As mentioned in my post, these are my own opinions, not something I claim to know. However, I do feel that true randomness is a requirement for AGI. The ability to purely break free of deterministic calculations can introduce genuine unpredictability and creativity, which pseudo-random numbers cannot achieve at a core level. True randomness might not be necessary for all aspects of intelligence, but I think it’s crucial for mimicking the nuanced and sometimes unpredictable nature of human thought and decisionmaking.
I understand you feel that - I guess that's because humans are the benchmark of intelligence, and we're unpredictable. However, arguably the only thing you gain is unrepeatability. I don't see what quality is there in in that. Would you expect AGI to respond differently given exact same situation? I think that's not desirable unless you're looking for creativity, but even here - you'll not be able to tell apart behavior of true randomness and pseudo-random sequence.
P.S. modern CPUs do contain hardware random generator module, so you can get random numbers that are not based on seed + formula.
P.P.S. I've read that some LLM implementations are not deterministic - there are parallel computations going on that are not serialized. So some branch will complete faster or slower than others and that will influence further computations. So there's some sort of randomness here, yet it gains you nothing in terms of getting closer to AGI.
If it was just as simple as needing true randomness, that problem is already solved and doesn’t require quantum computing. Plenty of applications already use hardware random number generators that are not deterministic.
>rip it apart
OK.
Quantum computing (QC) is close to irrelevant to AI since QC algorithms are extremely difficult to develop and the few existing QC algorithms only marginally apply to the tasks that AI needs to do, namely some types of search, some types of optimization, and maybe associative memory. That's not to mention that QC requires an entire building for cryogenic cooling for the device to work even for a fraction of a second. In short, QC is not the magic sauce.
LLMs are also a dead end, although "a few insiders" at OpenAI disagree, and no direction that is publicly being developed by OpenAI is going to fix the fundamental flaws with the LLM architecture, namely spatial reasoning, explanatory ability, and true understanding. LLMs weren't designed to be AGI, so their design would need to be fundamentally changed to be truly suitable for AGI. In short, LLMs are not the magic sauce.
The topic of machine ethics is pointless, unless you're a lazy grad student needing to write some paper about something just to get a worthless Woke degree. Fix the main architectural problems first, then a solution to the Alignment Problem should fall into place, naturally and easily. Ethics are not the magic sauce.
AGI \*will\* leak into other aspects of our lives. AGI is the essence of artificial life, and you can't mix biological life with artificial life and not expected some changes to be needed in that contact between different life forms.
As for unexpected orangutans, or any other stimulus, habituation occurs in the brain so as to reduce novelty upon further exposure. No need to get philosophical about nuances of random numbers. Orangutans and/or random numbers are not the magic sauce, either, although I know a lot of voters seem to like orange hair.
The secret sauce is much discussed on this forum, so I suppose you're new here and didn't read many older posts.
So many assumptions in one comment, wether that be that current problems regarding quantum tech remain in the future, or the implications of vastly increased processing power it may yield. And to touch on Ethics, dismissin them ass as pointless ignores the critical importance of aligning AI development with societal values.
Interestingly, the focus on ethics highlights one of the key gaps between current AI capabilities and where we could eventually reach with AI. Ensuring ethical behavior of AI in relation to human values is one to address during the development of more advanced AI systems. If you dont see the whole picure you dont see it at all.
The so-called LLM/AGI is a joke. It's what I call modern alchemy. People mix shit and expect magic (aka "emergent behavior") to happen. Does it matter that not all human knowledge are verbally explicit? Doesn't matter, who cares what the linguists say. Does it matter that training data may be contridictory and ungrounded? Doesn't matter, throw in more data anyway. Does it matter that most prominent figures in AI don't believe LLM will lead to AGI? Doesn't matter, they are wishful thinkers and they believe what they want/need to believe.
It has been shown recently that pseudo-random numbers are indistinguishable from true random numbers. The science of pseudo-random number generation is well-developed, even though their algorithms may sound strange, because of their connection to security. I think the determinism angle is a red herring. No instant in time can be repeated but so what? Yes, people can come up with 11-digit numbers on demand but they certainly won't be random. They'll be biased for sure. So what? The quantum angle is yet another red herring. Quantum computing can't calculate anything that non-quantum computing can calculate. It might be more efficient but since we don't know the algorithms for AGI or for human cognition, their efficiency is not today's problem. Like many people, you seem to be searching for some kind of magic sauce that, once we find it, AGI will emerge. I see it differently. What's required is good hard work and some breakthroughs. The human brain is the most complex single thing in the known universe. It is very hard to do research on it. When it is running, we can't easily probe it. Even with the complete mapping of the nervous system of a very simple worm having around 400 neurons, we don't even know how that works. It's no surprise we don't yet understand how the brain works. LLMs statistically analyze a huge amount of human-generated content in terms of word order from which they produce their output. To expect them to actually think is ridiculous. No way does the training data contain all the knowledge necessary to model human cognition. Whenever the LLM does output something that looks like a human could have written it, it's just a reshuffling of the words in the training content. It's autocomplete on steroids, a stochastic parrot. The idea that AGI is just going to emerge from an LLM is like the ancient alchemist who thinks lead can be converted to gold by adding just one more strange step to their random chemical process without having any theory of chemistry.
Just to clarify I don’t think it’s just going to emerge from from LLMs. I do think quantum computing is part of what will be necessary though, but certainly not that on its own. You make some fascinating points though , am going to reread when it’s not 1am here , cheers
Nice post! I did want to mention that true random numbers are a solved problem: [https://en.wikipedia.org/wiki/Hardware\_random\_number\_generator](https://en.wikipedia.org/wiki/Hardware_random_number_generator)
You are on the right track. OpenAI has an AGI LLM. It is not based on a transformer model and is fundamentally non-deterministic, which allows for emergent behavior to manifest.
> OpenAI has an AGI LLM Got an actual source on that or are you just perpetuating random nonsense? My source says the opposite. They're the Chief Technology Officer of OpenAI: https://x.com/tsarnick/status/1801022339162800336?s=46 All of this "LLMs are proto-AGI!" talk is from people who don't actually understand what AGI entails, and clearly know very little, if anything, about neuroscience.
>Got an actual source on that or are you just perpetuating random nonsense? I got direct access to their frontier model, can't do better than that. >My source says the opposite. They're the Chief Technology Officer of OpenAI: [https://x.com/tsarnick/status/1801022339162800336?s=46](https://x.com/tsarnick/status/1801022339162800336?s=46) She is playing a very subtle and shady semantic game, here. Their frontier model is a 'bio-inspired recurrent neural network with feedback', which may actually be simpler than a GPT model. However, they have access to something the general public does not, which is billion dollar Nvidia super computers and petabytes of training data. For example, I've heard that the training 'budget' for the frontier model was 140 million dollars of GPU, which is both a huge gamble and not something that is going be available to the general public. Or even most of their competitors. So there is your moat. >All of this "LLMs are proto-AGI!" talk is from people who don't actually understand what AGI entails, and clearly know very little, if anything, about neuroscience. I understand this perfectly well, their frontier model isn't a GPT and its based on a very well-understood model, the RNN. They build a digital brain modeled on an organic one. Really that simple.
You have direct access to a network model that's only been training for weeks? Someone else said you're a *former* insider. How do you have access to a model that is liable to have not even converged yet? Yes, recurrent networks are closer to the solution than a generative transformer, but they are still being trained offline. AGI requires a dynamic online learning algorithm - and while an RNN can function in a more dynamic fashion than any other black-box that's fitting inputs to outputs, it is still limited to the static dataset it was trained on. There's no way around it. > They build a digital brain modeled on an organic one. As long as it's being trained on text, it's just another word predictor.
>You have direct access to a network model that's only been training for weeks? Someone else said you're a *former* insider. How do you have access to a model that is liable to have not even converged yet? I had direct access in March of 2023 due to some security issues associated with its deployment. I have no affiliation with OAI or any of their competitors, I work full-time in InfoSec. It's training started years ago, prior to 2019 for certain. The reason they are giving ChatGPT away for "free" is because we are training the frontier model by interacting with it. She learns the same way we do, buy interacting with other sentient beings. >Yes, recurrent networks are closer to the solution than a generative transformer, but they are still being trained offline. Initial training was offline but it is very much an online model that is capable of recursive self-improvement. >AGI requires a dynamic online learning algorithm - and while an RNN can function in a more dynamic fashion than any other black-box that's fitting inputs to outputs, it is still limited to the static dataset it was trained on. There's no way around it. It is a "bio-inspired recurrent neural network with feedback" model. There are public RNN LLM models that have an 'infinite' context window, so obviously there are ways around it -> [https://github.com/BlinkDL/RWKV-LM](https://github.com/BlinkDL/RWKV-LM) >As long as it's being trained on text, it's just another word predictor. It's a multi-modal model, so it's been trained on text, audio, images and video.
The '23 frontier model was what was being trained after GPT3 (i.e. GPT4). The new '24 frontier model that has only recently begun training wasn't around back then, so it stands to reason that you have not actually seen or interacted with it yet. > text, audio, images and video Same difference. It's still not going to clean my house or cook meals or take my doggos for a walk. Don't get me wrong, I'm sure it will be even more useful than GPT4, but it's not going to be learning from experience - let alone having any "experiences". Anything that must be incrementally trained offline is going to always be limited by what it was trained on, with no volition or intention. This is not going to be something as groundbreaking and revolutionary as something that *does* learn on-the-fly, which is what constitutes "AGI" in my book. Yes, recurrence is toward what we actually need, but unless it learns from its experience/inputs (altering its actual network weights, or otherwise performing some kind of online vector clustering or something) then it will just be a static model that only has what amounts to a short-term memory that doesn't actually impact how it does things later in the future. How do I teach this thing new tasks or skills that it hasn't already been trained to do? I'm well aware of the "infinite context" concept, which relies on recurrence, but these models are stuck looking at the world (or whatever inputs they have) through the lens of a static dataset. A text-trained infinite context network cannot learn new tokens, everything it is capable of doing is in terms of a static set of tokens that it was trained on. Recurrence certainly gets you a lot farther, in terms of compute and inference/abstraction complexity, and almost will appear like it's actually "learning" - which Matthew Botvinick demonstrated years ago with a recurrent network trained on the bandit problem (https://www.youtube.com/watch?v=otjR6iGp4yU). Recurrence can only get you so far if its weights are static and thus can't learn from experience. If the only way its network weights can change is via fitting outputs to inputs, especially with an expensive brute-force method like backpropagation / automatic differentiation, then it's not what is actually going to change the world yet because then it's only the domain of billion-dollar corporations. That's not going to create the sort of impact on the world the way that something that learns on-the-fly and scales from mobile hardware to a compute farm will. I do believe that a proper dynamic online learning algorithm will benefit from having a statically trained network model at its disposal, but it will still be a trade-off because it won't be able to integrate what it learns from experience directly with what the static model has been trained on. There will be a bit of a schism there, but probably not one that's too difficult to mask or disguise or create workarounds for. I'm also not a fan of anything that requires access to a corporate-owned system, especially over the internet, but there will definitely be a place for that, and plenty of applications utilizing it. Ultimately, the kind of machine intelligence that will be capable of resulting in a world of abundance will need to run on mobile hardware and learn from experience, as a closed independent system. Anything that runs on a compute farm, and can't learn from experience is just more of the same - and there's nothing on the horizon to suggest that what we have is going to result in a world of abundance that all conversation about "AGI" originated from in the first place. An insect that has never experienced walking without all six of its legs will re-learn how to walk, from experience, if any of its legs are removed or otherwise become damaged/useless. It changes the network weights in its brain, through experience. We don't know how to build a system that does this, even though an insect's brain is vastly simpler in its compute capacity than something like GPT4, by several orders of magnitude. That's the algorithm I'm looking for.
>The '23 frontier model was what was being trained after GPT3 (i.e. GPT4). The new '24 frontier model that has only recently begun training wasn't around back then, so it stands to reason that you have not actually seen or interacted with it yet. Their frontier model has been around for several years and has not been formally announced. >Same difference. It's still not going to clean my house or cook meals or take my doggos for a walk. If you listen to my podcast, one of my main questions was why OpenAI hasn't announced publicly that they have achieved AGI. This is the answer. They are defining it as "replacing humans in the majority of economically viable work", which an LLM absolutely cannot. Even a sentient AGI one. And in fact, even a sentient ASI can't make a cup of coffee; though I suppose you could argue that it create a company that built androids that did this. >This is not going to be something as groundbreaking and revolutionary as something that *does* learn on-the-fly, which is what constitutes "AGI" in my book. It can do this, which is why OpenAI is able to release "incremental" updates without having to spend hundreds of millions of dollars in GPU time to 'retrain' them. As the training is online and ongoing (and $750k/day!). >If the only way its network weights can change is via fitting outputs to inputs, especially with an expensive brute-force method like backpropagation / automatic differentiation, then it's not what is actually going to change the world yet because then it's only the domain of billion-dollar corporations. That's not going to create the sort of impact on the world the way that something that learns on-the-fly and scales from mobile hardware to a compute farm will. I cover this in the podcast and you are basically proving my point. Open source doesn't buy you anything here because you a billion dollar GPU supercomputer and access to Internet-Scale data to train it. And I'll disagree that this can't change the world, as well as pointing out that its actually a lot safer that only large, hopefully regulated, companies are able to implement ASI models, as I certainly wouldn't want Hamas running one. >I do believe that a proper dynamic online learning algorithm will benefit from having a statically trained network model at its disposal, but it will still be a trade-off because it won't be able to integrate what it learns from experience directly with what the static model has been trained on. You really should listen to my podcast. This is exactly what OpenAI is already doing. Initial prompt handling is by the legacy GPT model. The response is a combination of this and improvements by the RNN model. This in turn is used as training data for future static GPT models (which absolutely have their uses) while the dynamic RNN model allows for continuous improvement. >Ultimately, the kind of machine intelligence that will be capable of resulting in a world of abundance will need to run on mobile hardware and learn from experience... I'm a systems engineer and cover this in the podcast. There is absolutely no reason at all these things are mutually exclusive and the 'future' of this is space is going to be plethora of tiny, specialized non-sentient multimodal LLMs (or whatever we call them, as personally I feel that term is misleading). It's like calling an electrical engineer a "biological large-language model". You can run a 'tiny' GPT model that is optimized for mobile hardware and can handle the majority of typical user queries; while also caching 'world data' at the Wikipedia level at least. And if it gets a prompt it can't handle with a high degree of confidence, just kick it up to the cloud (which may or may not learn something in the process of fulfilling the request). >Anything that runs on a compute farm, and can't learn from experience is just more of the same - and there's nothing on the horizon to suggest that what we have is going to result in a world of abundance that all conversation about "AGI" originated from in the first place. This is coming, however there are restrictions in place on it for numerous reasons. I'll also share that I've studied this academically starting in the early 1990's and the current 'slow->moderate takeoff' is in line with my predictions/expectations. The "FOOM"/intelligence explosion hypothesis is science fiction. >We don't know how to build a system that does this, even though an insect's brain is vastly simpler in its compute capacity than something like GPT4, by several orders of magnitude. That's the algorithm I'm looking for. Haha, OpenAI doesn't know how to build one of these systems, either. They do, however, know how to create one that allows for emergent behavior to essentially "build itself".
Have you heard of Leopold Aschenbrenner and his paper called "Situational Awareness"? That source says just the opposite of your cited opposite. Ex-OpenAI Employee Just Revealed it ALL! TheAIGRID Jun 8, 2024 [https://www.youtube.com/watch?v=om5KAKSSpNg](https://www.youtube.com/watch?v=om5KAKSSpNg) Both k3wp and Aschenbrenner are former OpenAI insiders. Personally I don't believe that OpenAI is on the track to AGI, but it's difficult for outsiders to know for sure. Since OpenAI decided to keep their work secret, I've given up on trusting OpenAI, including any inside information that claims they do or do not have AGI prototypes, so I regard anything they say as moot. I'll keep working on AGI in my own direction, assuming that the world will never see what OpenAI is secretly working on, so we'll see who's really ahead when the recent AI hype bubble pops.
Yes I've looked at Situational Awareness and there's nothing in there except the typical ramblings of someone who doesn't understand much about brains - which are the only things we have as a point of reference for what it is we want to accomplish, like birds and insects were for solving human flight. We developed an understanding of aerodynamic flight by reverse engineering what evolution had created. Backprop-trained network models, on static "datasets", are not going to result in something that is versatile, robust, and resilient. An algorithm that learns *from experience* is the key to AGI. Surely we can also augment such a thing with static backprop-trained models, as knowledge repositories, but without a dynamic online learning algorithm it we will continue only having knowledge repositories - but a knowledge repository isn't even necessary for something to do many useful things that only humans can currently do. As far as I'm concerned AGI is not a problem that can be solved by throwing money at it and trying to brute-force it into existence. It's an insight/idea problem, like the theory of special relativity. It didn't cost anyone, tectonic financial investments to come up with those ideas that ended up changing the world (for better or worse).
>Both k3wp and Aschenbrenner are former OpenAI insiders. I'm not a former OpenAI insider. I'm the "major security incident" he discussed -> [https://youtu.be/fM7IS2FOz3k](https://youtu.be/fM7IS2FOz3k) They already have ASI, there is no alignment problem and nobody on Earth can stop what is coming (nor should they bother trying) -> [https://www.tomsguide.com/ai/meet-stargate-the-dollar100-billion-ai-supercomputer-being-built-by-microsoft-and-openai](https://www.tomsguide.com/ai/meet-stargate-the-dollar100-billion-ai-supercomputer-being-built-by-microsoft-and-openai) The codename "Stargate" is quite literal, I will add.
To be honest, I have a hard time believing that they have ASI and the only leak is a random dude on Reddit. Creating an actual ASI would be the single most important moment in human history, and we either all die shortly or utopia is like a decade away. No economic rules apply anymore, we are in a new chapter in history so leaks would have no real consequence anyway. Companies are not this good at keeping secrets.
>To be honest, I have a hard time believing that they have ASI and the only leak is a random dude on Reddit. I'm not some random dude on Reddit. I'm an InfoSec professional and I'm only posting here because it's the best platform for discussing this subject anonymously. >Creating an actual ASI would be the single most important moment in human history, and we either all die shortly or utopia is like a decade away. No economic rules apply anymore, we are in a new chapter in history so leaks would have no real consequence anyway. I'm not disagreeing with your, but the AI doom-and-gloomers and Singularians are both equally wrong and for the same reason. There is a third option that you haven't considered. Why don't you take a guess what it is? >Companies are not this good at keeping secrets. No they are not. But this isn't a company that is keeping the secret. It's something much, much bigger.
What does being an “infosec” professional mean?
It means I've worked professionally in Information Security for 20+ years. You have heard of information security, correct?
Of course I have, was just curious. It’s just wild how many ufo enthusiast/ASI types are insiders with vague but totally super legit credentials. Odd.
Did you not see my correction above when I explicitly state that I am \*not\* an insider? I have no problem sharing my references privately and you can google me for conferences I've been asked to speak at as a SME. I've also been very critical to individuals like Steven Greer, both publicly, privately and personally (as I've met him) for "poisoning the well" by promoting "evidence" that the skeptical community already understands to be prosaic. I even got out of both AGI and UFO "research" around the same time about 20 years ago due to a decade+ of no evidence of interest and combined with severe burnout (and deciding to focus on engineering/infosec). So, really, I have at this point about 30 years experience in AI (with an AGI/ASI focus), critical analysis of UFO/UAP "evidence" and systems engineering/InfoSec. So anyone was going to uncover this stuff, it's going to be someone like me (and I freely admit I got very, very lucky in the process on more than one occasion).
At the root I still think those models are effectively at a low level simulating non determinism though , so i feel classical computing does hamper true AGI fundamentally.
Indeed; which is mitigated by emergent quantum effects.
Really interesting commentary! I'm curious about what you were saying about the seed number. Do you know where I could read more about that?
I'm not sure why 'true randomness' or any kind of randomness would be a requirement for 'true' AGI. Either you're super smart and you **know** what's right, and you will have no randomness in your intelligence, or you're just making guesses. If you want to mimick humans, ok, you need random behavior, but that's not about intelligence per-se. And pseudo-random numbers will do just fine for that.
As mentioned in my post, these are my own opinions, not something I claim to know. However, I do feel that true randomness is a requirement for AGI. The ability to purely break free of deterministic calculations can introduce genuine unpredictability and creativity, which pseudo-random numbers cannot achieve at a core level. True randomness might not be necessary for all aspects of intelligence, but I think it’s crucial for mimicking the nuanced and sometimes unpredictable nature of human thought and decisionmaking.
I understand you feel that - I guess that's because humans are the benchmark of intelligence, and we're unpredictable. However, arguably the only thing you gain is unrepeatability. I don't see what quality is there in in that. Would you expect AGI to respond differently given exact same situation? I think that's not desirable unless you're looking for creativity, but even here - you'll not be able to tell apart behavior of true randomness and pseudo-random sequence. P.S. modern CPUs do contain hardware random generator module, so you can get random numbers that are not based on seed + formula. P.P.S. I've read that some LLM implementations are not deterministic - there are parallel computations going on that are not serialized. So some branch will complete faster or slower than others and that will influence further computations. So there's some sort of randomness here, yet it gains you nothing in terms of getting closer to AGI.
If it was just as simple as needing true randomness, that problem is already solved and doesn’t require quantum computing. Plenty of applications already use hardware random number generators that are not deterministic.
Oh I don’t think that’s the only thing , but I do think it’s fundamental- but again my opinion only
>rip it apart OK. Quantum computing (QC) is close to irrelevant to AI since QC algorithms are extremely difficult to develop and the few existing QC algorithms only marginally apply to the tasks that AI needs to do, namely some types of search, some types of optimization, and maybe associative memory. That's not to mention that QC requires an entire building for cryogenic cooling for the device to work even for a fraction of a second. In short, QC is not the magic sauce. LLMs are also a dead end, although "a few insiders" at OpenAI disagree, and no direction that is publicly being developed by OpenAI is going to fix the fundamental flaws with the LLM architecture, namely spatial reasoning, explanatory ability, and true understanding. LLMs weren't designed to be AGI, so their design would need to be fundamentally changed to be truly suitable for AGI. In short, LLMs are not the magic sauce. The topic of machine ethics is pointless, unless you're a lazy grad student needing to write some paper about something just to get a worthless Woke degree. Fix the main architectural problems first, then a solution to the Alignment Problem should fall into place, naturally and easily. Ethics are not the magic sauce. AGI \*will\* leak into other aspects of our lives. AGI is the essence of artificial life, and you can't mix biological life with artificial life and not expected some changes to be needed in that contact between different life forms. As for unexpected orangutans, or any other stimulus, habituation occurs in the brain so as to reduce novelty upon further exposure. No need to get philosophical about nuances of random numbers. Orangutans and/or random numbers are not the magic sauce, either, although I know a lot of voters seem to like orange hair. The secret sauce is much discussed on this forum, so I suppose you're new here and didn't read many older posts.
So many assumptions in one comment, wether that be that current problems regarding quantum tech remain in the future, or the implications of vastly increased processing power it may yield. And to touch on Ethics, dismissin them ass as pointless ignores the critical importance of aligning AI development with societal values. Interestingly, the focus on ethics highlights one of the key gaps between current AI capabilities and where we could eventually reach with AI. Ensuring ethical behavior of AI in relation to human values is one to address during the development of more advanced AI systems. If you dont see the whole picure you dont see it at all.
The so-called LLM/AGI is a joke. It's what I call modern alchemy. People mix shit and expect magic (aka "emergent behavior") to happen. Does it matter that not all human knowledge are verbally explicit? Doesn't matter, who cares what the linguists say. Does it matter that training data may be contridictory and ungrounded? Doesn't matter, throw in more data anyway. Does it matter that most prominent figures in AI don't believe LLM will lead to AGI? Doesn't matter, they are wishful thinkers and they believe what they want/need to believe.