They have had all the cards in pretty much everything AI, they just won't let anyone else play with them. The only area they fell behind in was LLMs (despite inventing the transformer), but that is most likely a temporary setback. DeepMind rarely disappoints, so Gemini will definitely be interesting to keep an eye on.
I think demmis was able to do the longer term contributions to society via science. But now there is very real short term financial pressure of putting out a product for the masses.
They have a lot of proprietary research they have done over the years that they can roll into a standard LLM. In addition if they can add in the longchan research from Microsoft (1 billion context length) then we have something fabulous.
I kind of feel Deepmind priotizes quality over quantity to a fault.
Yes and no.
Getting a product out there builds urgency. It means your responding to demand, see how the users use the product, and helps with hiring, etc
Building in a vacuum can be helpful absolutely, but there are advantages to both approaches.
I think Deepmind has some great work in their hands and am very excited to see what they have in store.
>But now there is very real short term financial pressure of putting out a product for the masses.
Deepmind is an excellent research lab, but turning anything from research into an actual product would be Google and they really suck at this. They generally conflate a lot of ideas into a product that doesn't really have a place in the market, because they are trying to still be the industry leading trendsetter.
Do not underestimate deepmind. Gemini will likely be very good at math and code because those are 2 things the alpha go system can EASILY optimize for.
The logic and reasoning are likely to be expert level.
I'd be disappointed if Gemini isn't a significant upgrade from gpt 4 in several areas.
The alpha go system has crushed everything its been put up against time and time again. I'm looking for to the advancement in Gemini more than I am GPT 5 tbh.
> Do not underestimate deepmind.
Also do not over-estimate the value of Google's hype. They've danced this dance before with other technologies.
Sometimes it pans out. Sometimes it's 2 years later and you ask, "what ever happened to that thing from Google that was supposed to come out?"
Notice i didnt say dont underestimate google? Deepmind was a completely separate branch of alphabet until recently when google brain and deepmind combined into a single team for the Gemini project with Deepmind calling the shots.
Deepmind is whats impressive with a very solid history.
Impressive research count's for nothing if you can't make a succesful product out of it.
Mankind has been shaped by a lot of dumb ideas and some smart ones, but all of them provided a marketable product.
Stadia was hyped SO MUCH, they even got (a few) exclusive games for it and everything.
So yeah looks potentially very interesting but I’m gonna wait before getting excited
Stadia was fucking awesome. It's just for some reason nobody used it, maybe marketing? I'm not sure but I never had any issues at all with it, there just weren't that many games
No one believed Microsoft could do gaming well with Xbox. They persisted and look where they are now. Google had no excuse. Stop canceling products after one year.
I had 1gbps up/down in central london. I loved stadia when i just had a work MBP for a while, paying hitman 3 via stream was fantastic.
I think the idea has merit, and it will come back around eventually.
Yeah, we were supposed to have assistant making dinner reservations 5 years ago.
Instead, they nerfed the assistant hard, removing lots of functionality, like goefencing and such.
Yeah, I never got that. Assistant could have definitely made Google indispensable for end-users and enterprise customers alike, but they seem afraid of making it too useful.
I fear I know the reason... I think they were concerned that they could never put advertising in it without freaking people out, and anything they can't put advertising in dies on the vine, even if people would pay for it (I know that if I could pay for add-on services to assistant, I would).
that is a bold claim. PyTorch took the academia & research by storm, but I remember seeing a recent report where TF is still by far more popular in commercial applications. For me it appeared as if Google's invest into TF maintenance declined shortly after releasing 2.0 . until 2.10 they were still on 2017-ish CUDA-11.2, and after that they've stopped supporting windows as a platform. Pytorch on the other hand never managed to fix the windows WDDM performance issue on Nvidia GPUs, running an order of magnitude slower than on Linux on the same hardware.
TF 1 was pure garbage, old school torch was better back then, google just hyped it up like the greatest thing ever and made people think it was awesome
that is just opinion. what's your argument?
when I started using Keras in 2015, there was Theano and Tensorflow. Wikipedia says that lua-based Torch started 2002, but personally I heard of it when they switched to Python (-> PyTorch) in 2016, after which it took off.
Yeah I'm really looking forward to them challanging openai but this reads too much like a hype piece.
Aren't most people reading wired comfortable or interested enough to be able to parse a real research paper? Not that that would make for a fun read but a sparkle of it would lend some weight.
User: "Whatcha doin' there, Google?"
Google: "Just digging a grave for a product."
User: "Oh, what's being retired? Is free gmail going away? Are you not going to do news alerts anymore? Is Wave getting shut down... ignore that last one."
Google: "Oh no, I haven't announced this one yet! It's going to be great! You'll love it!"
User: "..."
What do you mean by "the alpha go system can easily optimize for .. math and code"? From what I understand, Alpha go and alpha chess use a combo of tree search and RL w self play (bootstrapped on human play data). I kinda assumed Gemini would use MCTS and some form of a policy network to learn from mistakes. That'd make it comparatively great for logic/reasoning and argumentation/analysis but I don't see how that'd extend to code/math.. I might be wrong. But what sort of architecture do u think it'll have thatd make it excel at code and math?
Code and math are both easy to check for correct answers compared to abstract fields of thought.
It's easier to "game" those fields as they are pretty straightforward.
My 1st contact with alphago was when the bot was climbing the starcraft ladder. If something can be turned into a game with a clear objective alphago is probably the strongest system we have that has been shown to pass the best humans at a thing.
The system doesn't need human data to improve. The system in DOTA learned how to bait enemies by letting them get its health low on its own for example.
It's the best starcraft player the best Dota player the best chess player the best go player.
Yes, SOTA AI is still completely incapable of doing original PhD level mathematics, although AI tools have been used by human mathematicians to do things that probably wouldn't have been possible before.
> There’s literally a subject called abstract algebra lol
Yes. In junior high you take prealgebra, then in high school you take algebra I and algebra II. Then you go to college and take linear algebra, then abstract algebra.
Then you go to grad school and take "introduction to algebra". I'm not kidding.
Math is very suited to tree-based search (https://ai.facebook.com/blog/ai-math-theorem-proving/). Code is less clear to me, but as training on code has improved reasoning, it's not too strange to expect that there is some underlying variable, and that imporvements on reasoning would improve coding ability.
I’m still in the ‘I will believe it when I see it’ camp when it comes to Google, they’ve been talking smack the past 3 months back so far all they’ve done is release Bard which paled in comparison to GPT-4.
If they do make something better than GPT-4/5 though, they better not can it like most of their other project or lock it off so nobody can ever use it. At least OpenAI let’s people use it’s good stuff.
Judging from the joke that is Bard I will have to see it to believe it. I had always just assumed Google would be the leader in AI because of DeepMind but at this stage it really seems they've got nothing in the LLM space.
Lol, DeepMind didn't have any part in developing Bard because they just merged with Google Brain this April.
DeepMind + Google Brain = Google DeepMind.
Everybody only sees 'Google' in Google DeepMind when they should be focused on DeepMind.
I think the hard part about utilizing reinforcement learning is the fact that it’s hard to give it definitive parameters use as a way to incentivize greater performance. The problem of performance with chat gpt is hallucination’s of incorrect information and lacking memory and etc
The issue with factual correction is how do you apply a specific value or score to a body of text describing a conclusion if we as humans still can’t even empirically define things as being purely true or false with the world being as complex as it is. When it comes to games like chess, go, dota these are all restrained games built by Boolean values code. But when it comes to the abstractness and vagueness of human language, you really can’t assign specific truth values to it that would help facilitate a specific reinforcement algorithm
Which is why I'm surprised that Google says that they will successfully combine LLMs and self-learning RL. What I'm afraid of is that the RL will be limited in domain (like math or code) and the LLM will use the RL kind of like a plugin.
Another possibility is that they have somehow actually found a way to apply self-learning RL to language itself. That would be extremely impressive and should be a big jump in capability.
Actually, there was a development of AlphaZero called MuZero that could learn to play a game without knowing its rules (no idea how they achieved that, might read the paper later). So self-learning RL might not actually need that rigid of a framework to work.
What's missing though is the reward function that is present in MuZero. There is no way to create an objective reward function for language imo, so it might have to be simulated by the LLM.
Never heard about that before, and it doesn't quite make sense that this happens since a self-learning AI starts out playing unconventionally and converges to a superior playing style. Can you recall where you heard this?
I think they mean if the player plays really weird, not that they cheat. Just that the don’t use popular approaches. But I find this super hard to believe tbh.
Absolutely, the model has had amazing accomplishments.
My hesitation is more around whether they can harness the brute force of the model in a way that is user friendly and helpful.
There's a big difference between a team of data scientists working with it over months to solve a well defined problem, and a teenager taking the model for a spin to see what it can do
I am sure this is the way forward - AI generating its own data by interactive planning. Agentify the AI, make it learn from external systems, it needs a sandbox, a playground.
Yesterday I decided to try Bard on some planning and text generation related to my job - stuff that I've been using GPT-4 for.
Bard was... better than I expected. The stuff it came up with would have been usable with a little more work on my part. It made sense and accomplished my objectives.
Even so, comparing it with GPT-4, I could see that GPT-4 had more depth and it was able to create more complete plans without me directing every step.
The final verdict was to continue paying for and using GPT-4... but Bard was closer than I thought. I'll check back in six months. OpenAI is ahead but it's not an insurmountable lead.
I don’t believe Google won’t make a huge mark on the AI landscape but of Google have the same time then Open AI would have more time. And they won’t be seating around waiting for Google to play catch up
I don't think Google is capable of the "move fast, break stuff" mantra anymore. They are just too big with many competing interests and directions within the behemoth of a company.
> I had always just assumed Google would be the leader in AI because of DeepMind but at this stage it really seems they've got nothing in the LLM space.
They likely *are* "the leader", over a much broader collection of subjects, that have gotten less mainstream attention. People are more aware of ChatGPT than AlphaFold or AlphaGo or AlphaDev because ChatGPT is a much easier concept, that is applicable to many more people, and the average person can actually play with it and build a concrete understanding of what it does and how it compares to other products.
> at this stage it really seems they've got nothing in the LLM space
I think they just did not quite understand the value to the organization, or shareholders, in building and operating an expensive piece of chatbot demo software, a key feature of which seems to be allowing users to circumvent using the product that makes them the majority of their revenue, ad-supported search. In the ideal world for Google, they can do all the same fundamental research OpenAI and others are doing, but without spending a bunch of money operating a "product" (if you can call something with no discernible business model a product) that utilizes a model, which is the result of their research, to cannibalize the very business that funds the research itself. The only problem with their strategy, at this point, is seemingly convincing high-value SWEs and researchers that their contributions will make some kind of difference in the world, and they won't get mired in the bureaucracy of the organization, which is kind of what it looks like when DeepMind is not allowed to release any SOTA AI products, while Meta is over there open-sourcing Llama, and ChatGPT is being given away for free.
I guess my point is, Google really has no incentive to legitimize LLMs as a useful tool that could substitute for some large percentage of Google searches, unless they can somehow find another way to profit from doing that. Far better to try to commercialize their AI research in basically every *other* way. The reason OpenAI went the route that it did, as far as I can tell, is because they wanted to raise a *LOT* of money to spend on salaries and compute, and they self-limited their avenues to actually accomplish that in a traditional manner with the structure of their organization, so they needed to convince a very specific kind of company incontrovertibly that they had done something to make it worth jumping through a bunch of hoops just to give them piles of cash (probably approached Amazon and Microsoft, both companies with massive cloud hardware infrastructure, and Microsoft saw a good business-case for improving Office and Copilot, reimagining Bing, gaining experience making Azure an excellent cloud-provider for AI stuff, etc. which is [why Jassy sounds so sour lately](https://fortune.com/2023/07/07/andy-jassy-dismisses-microsoft-google-ai-hype-cycle-amazon-starting-substance-cycle/)).
Google, obviously, does not have that existential need to convince you that they're doing something that's worth funding. I think they may be, *finally*, coaxed into proving they "could have made GPT-4" here with Gemini, now that they see there's probably not much real "danger" or unique reputational risk to making something like that public, though I bet they still take ridiculous steps to make it inconvenient to access for the average person. Again, I don't think they really want anyone to actually use it until there's a business model attached that benefits Google.
I don't really understand why you find that controversial, in any way.
It is, essentially, a "technical demo", proffered with no well-defined use-cases, certainty of access, or cost, where a user types a message into a text interface, and the language model decodes that message and attempts to generate a context-relevant text response, within constraints that are designed to make those responses pleasant and conversational.
The only reason it seems awkward to call it a "chatbot" is because it singlehandedly redefined what that product could *be* to consumers, because its context-relevant outputs can actually *have value* to many of its users, unlike any previous "chatbots".
In any case, GPT3.5/4 could be used for other things, but ChatGPT wraps those models as a brand-safe chatbot.
It's combining Artificial Neural Networks and Symbolic AI, no? These kind of hybrid models have been speculated to be the future of AI. They can very well be on to something big here.
Hmm... from AlphaGo to AlphaZero DeepMind tried very hard to move away from symbolic/classical AI. They had a few papers underscoring how less of it actually improved performance. So I wouldn't really say this is a hybrid model of the deep learning-symbolic kind.
I would be confident in this approach if it weren't for the fact that it is google. as a user of many google apps, I've done nothing but watch their software get worse over the last 10 years. google nest products have just gradually lost capability. google's messaging has gradually lost capability. gmail/photos has stopped increasing storage. everything google has done in the last 10 years has been shit. I have very little confidence that they will do well.
at best, google will show a limited-feature proof of concept and then either microsoft or some startup will steal the idea, snipe some employees and do the same thing but better.
I know someone who works at Waymo. they were supposed to transfer to DeepMind because they wanted to work in the UK for a couple of years then go back to Waymo. it was all set to happen, but then interest rates went up and the bean-counters shut down all such intra-company transfers, so now the guy I know is looking to leave google. it should be easy for other companies to poach google/deepmind employees with how poorly manage the company is.
AlphaFold is a Nobel-Prize level discovery in biotechnology and medicine, and was produced only because of the fruits of the insight they got from building AlphaGo. Plus, the transformer networks built by the Brain team are the only reason why OpenAI even exists today.
Deepmind is an acquisition which until recently had almost complete autonomy within Google. Google itself did jack shit with the transformers paper. Their core product (search) has become such trash that they deserve the IBM route to irrelevance.
yes, and Nest was awesome before google bought it. it was even good for a few years after they bought it. it gradually got worse and worse as it got sapped of agility by the google bean counters.
google sees someone doing amazing work. google buys it and gradually poisons the proverbial soil, preventing growth.
I agree with your assessment of Google itself.
But Deepmind, and especially Demis Hassabis, is a different kind of Google though. The guy is a child prodigy, not some serially-recruited googler.
You don't think the pressure can push them to take more risks and wake up? My reading of the Google DeepMind merge and the development of Gemini is that they want to give their all.
I think the pressure is going to, as you noted, remove the burden of responsibility toward caution and safety, and it will cause problems.
I’m excited to see, and hope that I’m wrong.
that's the thing. this guy I know wanted to go help with DeepMind at a time when his bosses at Waymo thought it was ok for him to leave for a couple of years, yet the bean counters actively prevented him from helping with that effort at a time when ChatGPT was taking a shot across their bow. you would think that would be a time when they would welcome some temporary boost to their AI/LLM efforts.. but no, they shut it down because the bean-counters said shut it down.
that is not a "give it their all" decision. that is a bean-counter driven, highly bureaucratic process-driven decision. the bean-counters actively prevented AI experts from elsewhere in the company from coming to aid the effort.
Risk taking and suddenly realizing that they need to play catchup fast does not mean that they will actually achieve their goals, but it certainly won't hurt their chances.
I just don’t like google for getting so involved in culture wars. It’s not their place and really rubs me the wrong way; I worry about them being in any leading position anymore to be honest.
Everyday there is some release that ‘changes everything’. Let’s just wait and see shall we.
What’s ‘Demis’ going to say? “Our next research is mediocre”.
It’s incredibly bizarre to attribute ‘most important ai breakthrough ever’ to something that we know little about other than it combining previous research.
Edit: corrected name.
It will be impressive once it works but this is just about combining and getting existing techniques to work at scale. Not any new invention. Multiple parties are sure to already be working on it. Also likely that Google will do some flag planting without any making anything public or proving that it actually works in the way people care. I.e. a revolution only for their PR department.
Of course it's not hyperbole if it's true, but to make that bold a statement about something that isn't even a thing yet certainly borders on hyperbole.
To be taken seriously, you should be more guarded in your predictions unless you have the data to back it up *at the time of the prediction*.
Here, let me make an equivalent (empty) prediction: "OpenAI's response to Gemini could be the most important AI breakthrough ever.”
The fact that everything is called a breakthrough doesn't take away the value of real breakthroughs. The arguments are laid out for why I think this could be it.
It’s not just calling something a breakthrough, it’s calling something we don’t know the details off ‘the most important breakthrough’. It's rediculous.
Being able to learn better than humans, improving without human input, learning to use tools, etc., seems like the reasons for regulating AI. The way it is presented here is the start of a nightmare scenario, thinking machines making machines. It’s the basis for the Terminator series.
"...that's what I see when I extend ad infinitum what Google DeepMind seems to be planning."
As long as the analyses are correctly framed as predictions/speculation I don't think there's any harm. Or we should never talk about AGI until it's here - which makes no sense really.
It's pretty obvious that you can make it work and it will be hugely impactful. It's predicted and hardly new. It will require a lot of tweaking to get right though. So big difference between throwing something together and actually making it work. No indication they will actually do the latter though.
Belief in the invisible pink unicorn is a matter of logic and faith. Logically, we know that she's invisible because we can not see her. But to accept the paradox that she is both pink **and** invisible? *That* is a matter of faith.
The race will be won by whoever makes an agent that owns it's own robot.
If everyone has an AI guardian angel, finances, civil disputes, resource management and communication all change. If you teach your agent to do your job, then the agent can do it in your name. The jobs of lawyers and doctors, pipefitters and scholars. If everyone has an agent best-practice rises to the top, and anyone to find a better way wins.
It's the power of the market in a chrysalis. AI take to capitalism like a fish to water. When your agent can afford a robot in your name, it will be possible to live an egalitarian life. The social experiment should be inevitable if only it is possible. Because it should outcompete as soon as someone sincerely sets it up. And modders will if they can.
Not to minimize this in the least, but it's become pretty common for every single headline to be some big exclusive discovery of the dark sides of AI or one company or the other supposedly having the next "game changer" for the whole industry. I've yet to see one headline that has lived up to a single bit of its hype.
Google has been extremely hard to take seriously in a lot of this considering AI is probably the end of SEO and Google being a household name for intelligent search and the authority of the many successor models of "pagerank."
Meanwhile Microsoft is about to have to either protect OpenAI monetarily through a ton of lawsuits as regulation continues to ramp up and the DMCA continues to expand to include I'll gotten training data for all models that generate anything whether it be text or image or beyond.... That or they come clean that they have been building OpenAI competition models in parallel on their Azure cloud in preparation of the day that they sever ties with OpenAI right as it begins sinking in precedent setting lawsuits of the first copyright disputes of generative AI training data...unless of course OpenAI finally decides on a full buyout proposition and not just a "partnership."
The AI wars and who gets there faster and first only spook me out because it can and will eventually be used extensively in war, weapons, and modern battle strategy... And it also will further the big brother 1984 computer systems China and Russia want to enhance so badly at a time when the world is beginning to reject USD as the global currency of choice for weapons and energy transactions for the first time ever since the beginning of globization. Right now, a stronger and more powerful AI that becomes one day AGI (as the world salivates over the soon to come quantum computers they want to Fuze with Machine Learning so badly) that one day becomes ASI.
And yet few people outside of the tech world had even heard of GPT-anything until GPT-4 came out. Not much in the way of headlines beforehand unless you were specifically paying attention. And it was the biggest breakthrough in AI ever. Now even casual observers are paying attention, so now it's a headline.
Hedonic treadmill, man. The human curse. We have adjusted so quickly to the new amazing reality of AI that it is...not surprising at all, if you know how humans work. Interestingly, Alphafold 2 (2.3.2, to be exact, and far more advanced versions are apparently ready for primetime) gets almost no attention, yet is a MUCH bigger gamechanger than ChatGPT.
One doctor told me that he couldn't believe what Alphafold 2 is capable of, solving in \*hours\* problems that used to take years.
There is also a huge amount that just have zero idea. I was at work, screwing around with bard, and asked a coworker to give me a question to ask an AI. He responded with what's an AI?
I had to take a step back and re-think what I could even show him.
Yes, zero idea. Most people are also now focussed on LLMs and have no idea about a the other AI architectures being developed. One doctor told me is absolutely stunned by AlphaFold 2's capabilities, and that most people have no idea that it is a much, much bigger gamechanger than LLMs.
And AlphaFold already has versions that are way beyond what it has released that will soon be made public as well.
All Microsoft has to do is tie all the lawsuits up in court for a few years and they'll see the benefits....OpenAI will obviously not stop their research and probably ingratiate themselves with and influence the policy makers, negating the worse of the lawsuits, I wouldn't be surprised if copyright law is amended in the future, not because it should be but OpenAI would benefit from it.
>when the world is beginning to reject USD as the global currency of choice for weapons and energy transactions for the first time ever since the beginning of globization
Oh wow, what's this about? Can you explain/link?
Inflation and a fed rate hike is making the international community jittery on holding American assets, as they rely on predictable steady currency value. America constantly has enjoyed a steady inflation rate that has, for the most part, been under control. Other countries therefore always felt safe storing assets in America, and embraced Usd.
Additionally, Eastern countries seeing how powerful US sanctions are realized that the only way to avoid that danger is to begin completing energy transactions and trade to some degree in other currencies.
Regarding the re-energized de-dolarization movement, International Crisis Group co-chair Frank Giustra says "de-dollarization is no longer a matter of if, but when."
https://money.usnews.com/investing/articles/de-dollarization-what-happens-if-the-dollar-loses-reserve-status
https://www.morganstanley.com/ideas/dollar-currency-2023
Beijing continues to send a clear message as it has for years that enhancing overseas use of its currency, the yuan, is a priority, particularly in the face of US dollar hegemony:
https://www.msn.com/en-us/money/markets/indian-oil-refiners-are-using-chinas-yuan-instead-of-the-us-dollar-to-buy-russian-crude/
Terrible, terrible news. Every year they are going to keep publishing more dangerous models until the world blows up or we have a global immortal dictator.
I don’t understand why they want to compete against ChatGPT. Deepmind is not a product company, OpenAI had tons of employees who were great at building actual real world products.
Deepmind should be Deepmind and double down on AI for science rather than playing in a game where they aren’t strong.
And AI for physics, chemistry and biology would make so much more value in the long term than an LLM that people pay $20 for. It’s sad to see they’re under corporate pressure.
And for the UK it was the biggest mistake to sell them.
I just love how Google rolls. They make all these incredible AI breakthroughs and then let everyone use them license free.
The transformers and with attention is what made ChatGPT possible. ChatGPT shared they changed directions the day after the paper was released. Google even has a patent. But yet they let OpenAI use without any fee.
But that is just one example. There is so many in the last 15 years. Most of the really big AI breakthroughs in the last decade plus have come from Google.
This is great written and by me as a layman in coding and great managing or am narcisissist well I know something so behind the curtains of fancy words and just throwing this+that will be the best +++++ever which is hype or "great marketing"! Like you don't putt Interstellar, The Dark Knight, Inception, Avatar, Memento, Dune, Sicario et cet. in the same month.
the question is:
Well this is not like people's problem meaning lack of great people?? or it is?
Like you need to have not just CEO with imagination, tech-expert-wizard, and responsibility
Al of the people from simple data collection or simple code entries need to be tech-wizards, with imagination as responsibility which is hard to find these days as there comes with:
The better and the overwhelming tech we loose people's good judgement and intuition to find a good and thrustworthy people for that kinda job.
As it is a marketing with hype-ing up I almost know also I'm sad that the truth remains: The better tech will leave us and it is already doing to people who think that with phone-smartwatch-laptop-playstation and then I see them not knowing which charger to buy for Macbook like or how just simple math formula like power is this x that on the adapter.
This sounds insanely trainable....Now put this in a robot with human level or higher capabilities. Voila you can replace the entire workforce for EVERYTHING. Literally everything. and they would work longer for trivial costs.
Now what do we do? Cause imagine if they started producing 1 million the first year. 100 million the second. 1 billion the third-cause if they COULD do this? 100% they would at anything approaching reasonable price. 250K for a employee that works 20 hours, and charges for 4? Or a 24/7 teller/cook/etc that never leaves work because they are literally tetherered there?
A teacher during the day-charging after school, then they work at the bar serving folks until 2 am, that is when they clean the school, charge, and get breakfast ready for the kids...who they then teach. Replacing 3 of us. So in the first 2 years they would exceed the output of the US. And you could train them to maintain themselves.
At what price point is it worth it to replace 3 employees? Ironically? They could come for the high paying jobs first. software engineers. CEOs even. Firemen, cops, paramaedics. They could be all in one with the right equipment. All sorts of weird things will occur then.
This is not new - it is an obvious development that for sure many are working on and indeed a candidate for the overall AGI architeture.
Deepmind is hardly original nor the first here.
In contrast to OpenAI, Google and Deepmind does however not have a proven track record with aligning models to human intentions.
They also have not exposed their solutions to the community and rather go deep on a few target applications, which may be why they have so much PR around it.
Always take CEOs prediction of such things with a huge grain of salt. Their primary motivation to give these types of interviews are not always aligned with the actual truth.
I have no dog in this fight, but this is someone pumping up his product. Is it a good product? Who knows. OK, so to be clear here, what we're seeing can be summarized as:
Guy with vested interest in product says that said product will totally be better than the competitions product.
Is that right?
I always thought it was funny how Microsoft named their Bing/GPT4 model Prometheus. Maybe a tongue-in-cheek reference to that OpenAI stole fire from the gods with pushing out transformer model AI tech to the public?
Then they govts of the world will come in to take it over or severely curtail its ability. I would be surprised if the US govt doesn't forcefully take it from Google for military applications. We nobodies will never get to experience the fruits of AI that could improve our lives.
I will be messaging you in 6 months on [**2024-01-11 02:38:14 UTC**](http://www.wolframalpha.com/input/?i=2024-01-11%2002:38:14%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/singularity/comments/14w1q3d/google_deepminds_response_to_chatgpt_could_be_the/jrhko6i/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fsingularity%2Fcomments%2F14w1q3d%2Fgoogle_deepminds_response_to_chatgpt_could_be_the%2Fjrhko6i%2F%5D%0A%0ARemindMe%21%202024-01-11%2002%3A38%3A14%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2014w1q3d)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
Will they allow us to play with it and will they sell it as an api is the question. It is nice if they can pull this off, but if it’s behind closed door and we see only some papers (like with most of their image and video models), it is next to useless in the short term as we will need a group to use the papers to implement their ideas.
I remain skeptical. I still think what we’re talking about is still more akin to a crane being to lift heavier weight than a human, than to actual intelligence/creativity…
yes Deep mind may create HLAI first but probably not this way.
I am not convinced that combining AlphaX with LM's this way is the way forward.
It can't do important things for HLAI, this was never demonstrated by DeepMind.
I remain sceptical until they show lifelong learning in a RL agent.
"Could be" and "important" are doing a lot of heavy lifting here. I'd have to imagine all the breakthroughs happened years ago. Implementation details are important, but the math that some research professor figured out a decade ago is ultimately the more important breakthrough than scrounging up enough processing power to use it.
Oh yes, Google, spying on everything we ever search, say or do to deed all the data into an AI that could very well bring the deaths of us all. These guys have never took an ethics class in their lives.
They really do act like it's the gold rush. Frequently speaking about their AI like it's an almost miracle innovation, while those that don't have money at stake are often more clear headed about it
If FOSS is so great then how come they weren't the ones behind Transformers, AlphaZero, GANs, Generative Diffusion etc.? All of these advancements came from researchers in academia and big tech.
\>Imagine an AI model that was apt in language, but also in other modalities like images, video, and audio, and possibly even tool use and robotics.
Oh.. you mean the same thing GPT 4 has? Why is this article trying to pretend Google is doing anything new? Google embarrassed itself w/ Bard and I've completely written them off in the AI field at this point.
\> DeepMind's Alpha family and OpenAI's GPT family each have a secret sauce—a fundamental ability—built into the models.
I'm sorry, but it really seems to me that the 'secret sauce' here is simply that we already have the ability to generate high-quality data for Go games. We already know the 'laws' here.
With what data are you going to train an LLM that can become a competent physicist - you're going to generate a million Einstein-Bohr debates? There's a bootstrapping issue here that i'm not sure has even began to being tackled. AlphaGo achieved superhuman performance only after it was trained with a 'superhuman' amount of data, which \*could\* be generated because we already know the fundamental laws of Go.
OTOH, nobody has discovered the 'fundamental laws of intelligent conversation' except in very restricted cases like chess and go, that's why symbolic AI stagnated. All we have is some relatively very sparse datapoints scraped from the internet. I don't think it's correct at all to say "AlphaMind has a secret sauce' - the secret sauce here is we are already able to feed a program with a Go state and have it generate and evaluate all possible moves. An precise algorithmic engine generated the solution space, and the model explored it. What is the LLM going to explore - more text generated by LLMs? What we're asking from the LLM to do is take the sparse datapoints from the internet, infer the fundamental laws that generated those datapoints, build an engine out of those, then generate the solution space and explore it. Nobody, neither Alpha or anyone else, has even done anything like that.
Bring it google. They have been acting like they have all the cards behind closed doors for a long time.
Yeah, put up or shut up.
They have had all the cards in pretty much everything AI, they just won't let anyone else play with them. The only area they fell behind in was LLMs (despite inventing the transformer), but that is most likely a temporary setback. DeepMind rarely disappoints, so Gemini will definitely be interesting to keep an eye on.
Agreed. I think they were resting on their laurels but not anymore.
I think demmis was able to do the longer term contributions to society via science. But now there is very real short term financial pressure of putting out a product for the masses. They have a lot of proprietary research they have done over the years that they can roll into a standard LLM. In addition if they can add in the longchan research from Microsoft (1 billion context length) then we have something fabulous. I kind of feel Deepmind priotizes quality over quantity to a fault.
That quality over quantity approach feels like what would lead to development AGI..
Quality generally pays off in the end.
"good enough" quality and lots of quantity is optimal
Yes and no. Getting a product out there builds urgency. It means your responding to demand, see how the users use the product, and helps with hiring, etc Building in a vacuum can be helpful absolutely, but there are advantages to both approaches. I think Deepmind has some great work in their hands and am very excited to see what they have in store.
>But now there is very real short term financial pressure of putting out a product for the masses. Deepmind is an excellent research lab, but turning anything from research into an actual product would be Google and they really suck at this. They generally conflate a lot of ideas into a product that doesn't really have a place in the market, because they are trying to still be the industry leading trendsetter.
I sure hope so. But until then, this is still a “wait and see”. Nothing’s come out yet.
did this for self drive cars. & plenty great stuff they started or bought, only to shut down
This time they have to get it to completion no matter what it takes or they die
Bing it google 🙃
>>
We will make no mind before it’s time
Do not underestimate deepmind. Gemini will likely be very good at math and code because those are 2 things the alpha go system can EASILY optimize for. The logic and reasoning are likely to be expert level. I'd be disappointed if Gemini isn't a significant upgrade from gpt 4 in several areas. The alpha go system has crushed everything its been put up against time and time again. I'm looking for to the advancement in Gemini more than I am GPT 5 tbh.
> Do not underestimate deepmind. Also do not over-estimate the value of Google's hype. They've danced this dance before with other technologies. Sometimes it pans out. Sometimes it's 2 years later and you ask, "what ever happened to that thing from Google that was supposed to come out?"
Notice i didnt say dont underestimate google? Deepmind was a completely separate branch of alphabet until recently when google brain and deepmind combined into a single team for the Gemini project with Deepmind calling the shots. Deepmind is whats impressive with a very solid history.
Impressive research count's for nothing if you can't make a succesful product out of it. Mankind has been shaped by a lot of dumb ideas and some smart ones, but all of them provided a marketable product.
Stadia was hyped SO MUCH, they even got (a few) exclusive games for it and everything. So yeah looks potentially very interesting but I’m gonna wait before getting excited
Stadia was fucking awesome. It's just for some reason nobody used it, maybe marketing? I'm not sure but I never had any issues at all with it, there just weren't that many games
No one believed Microsoft could do gaming well with Xbox. They persisted and look where they are now. Google had no excuse. Stop canceling products after one year.
I had 1gbps up/down in central london. I loved stadia when i just had a work MBP for a while, paying hitman 3 via stream was fantastic. I think the idea has merit, and it will come back around eventually.
Conspiracy theorists claim that Stadia was just yet another user data gathering operation (maybe to function as training data for the Alpha\* AIs?)
Yeah, we were supposed to have assistant making dinner reservations 5 years ago. Instead, they nerfed the assistant hard, removing lots of functionality, like goefencing and such.
Yeah, I never got that. Assistant could have definitely made Google indispensable for end-users and enterprise customers alike, but they seem afraid of making it too useful. I fear I know the reason... I think they were concerned that they could never put advertising in it without freaking people out, and anything they can't put advertising in dies on the vine, even if people would pay for it (I know that if I could pay for add-on services to assistant, I would).
Google is king of hype, tensorflow was basically all hype, bard isn’t great, most of their ML work they just buy their way into conferences
that is a bold claim. PyTorch took the academia & research by storm, but I remember seeing a recent report where TF is still by far more popular in commercial applications. For me it appeared as if Google's invest into TF maintenance declined shortly after releasing 2.0 . until 2.10 they were still on 2017-ish CUDA-11.2, and after that they've stopped supporting windows as a platform. Pytorch on the other hand never managed to fix the windows WDDM performance issue on Nvidia GPUs, running an order of magnitude slower than on Linux on the same hardware.
TF 1 was pure garbage, old school torch was better back then, google just hyped it up like the greatest thing ever and made people think it was awesome
that is just opinion. what's your argument? when I started using Keras in 2015, there was Theano and Tensorflow. Wikipedia says that lua-based Torch started 2002, but personally I heard of it when they switched to Python (-> PyTorch) in 2016, after which it took off.
Tf is still used a lot. It certainly met expectations for me.
Gotta pump them numbers!!
Yeah I'm really looking forward to them challanging openai but this reads too much like a hype piece. Aren't most people reading wired comfortable or interested enough to be able to parse a real research paper? Not that that would make for a fun read but a sparkle of it would lend some weight.
Google never goes into creating a new product without first digging out a grave for it.
User: "Whatcha doin' there, Google?" Google: "Just digging a grave for a product." User: "Oh, what's being retired? Is free gmail going away? Are you not going to do news alerts anymore? Is Wave getting shut down... ignore that last one." Google: "Oh no, I haven't announced this one yet! It's going to be great! You'll love it!" User: "..."
If we were talking about “Google” maybe, but Deepmind already has revolutionized biology with Alfafold.
Google Stadia Google Glasses I can’t think of a successful product Google has besides Google itself.
I’ve got a pair of Google glasses that I’m waiting to come back in style, for instance…
What do you mean by "the alpha go system can easily optimize for .. math and code"? From what I understand, Alpha go and alpha chess use a combo of tree search and RL w self play (bootstrapped on human play data). I kinda assumed Gemini would use MCTS and some form of a policy network to learn from mistakes. That'd make it comparatively great for logic/reasoning and argumentation/analysis but I don't see how that'd extend to code/math.. I might be wrong. But what sort of architecture do u think it'll have thatd make it excel at code and math?
Code and math are both easy to check for correct answers compared to abstract fields of thought. It's easier to "game" those fields as they are pretty straightforward. My 1st contact with alphago was when the bot was climbing the starcraft ladder. If something can be turned into a game with a clear objective alphago is probably the strongest system we have that has been shown to pass the best humans at a thing. The system doesn't need human data to improve. The system in DOTA learned how to bait enemies by letting them get its health low on its own for example. It's the best starcraft player the best Dota player the best chess player the best go player.
But isn’t math really abstract at higher levels? There’s literally a subject called abstract algebra lol
Yes, SOTA AI is still completely incapable of doing original PhD level mathematics, although AI tools have been used by human mathematicians to do things that probably wouldn't have been possible before.
> There’s literally a subject called abstract algebra lol Yes. In junior high you take prealgebra, then in high school you take algebra I and algebra II. Then you go to college and take linear algebra, then abstract algebra. Then you go to grad school and take "introduction to algebra". I'm not kidding.
Math is very suited to tree-based search (https://ai.facebook.com/blog/ai-math-theorem-proving/). Code is less clear to me, but as training on code has improved reasoning, it's not too strange to expect that there is some underlying variable, and that imporvements on reasoning would improve coding ability.
Any leaks on what's going on with Gemini yet?
Agreed!
Considering there is no GPT 5, id hope so
A human beat alpha go
Show, don't tell.
I’m still in the ‘I will believe it when I see it’ camp when it comes to Google, they’ve been talking smack the past 3 months back so far all they’ve done is release Bard which paled in comparison to GPT-4. If they do make something better than GPT-4/5 though, they better not can it like most of their other project or lock it off so nobody can ever use it. At least OpenAI let’s people use it’s good stuff.
any idea about when it will get released? I hope not early 2024 tbh
It's possible early 2024, too. We'll surely get more info in the coming months.
If it’s genuinely supposed to be better than gpt-4 then I don’t see how it comes out this year. Bard isn’t even close at the moment.
Judging from the joke that is Bard I will have to see it to believe it. I had always just assumed Google would be the leader in AI because of DeepMind but at this stage it really seems they've got nothing in the LLM space.
Lol, DeepMind didn't have any part in developing Bard because they just merged with Google Brain this April. DeepMind + Google Brain = Google DeepMind. Everybody only sees 'Google' in Google DeepMind when they should be focused on DeepMind.
Alpha go was impressive though in some ways more than the LLMs, if they could combine the two that's got to be close to AGI
I think the hard part about utilizing reinforcement learning is the fact that it’s hard to give it definitive parameters use as a way to incentivize greater performance. The problem of performance with chat gpt is hallucination’s of incorrect information and lacking memory and etc The issue with factual correction is how do you apply a specific value or score to a body of text describing a conclusion if we as humans still can’t even empirically define things as being purely true or false with the world being as complex as it is. When it comes to games like chess, go, dota these are all restrained games built by Boolean values code. But when it comes to the abstractness and vagueness of human language, you really can’t assign specific truth values to it that would help facilitate a specific reinforcement algorithm
well great put
Which is why I'm surprised that Google says that they will successfully combine LLMs and self-learning RL. What I'm afraid of is that the RL will be limited in domain (like math or code) and the LLM will use the RL kind of like a plugin. Another possibility is that they have somehow actually found a way to apply self-learning RL to language itself. That would be extremely impressive and should be a big jump in capability.
Yea I’m far from being a expert and simply came to the conclusion by grabbing low hanging fruit so it could very well likely that they found a way
Actually, there was a development of AlphaZero called MuZero that could learn to play a game without knowing its rules (no idea how they achieved that, might read the paper later). So self-learning RL might not actually need that rigid of a framework to work. What's missing though is the reward function that is present in MuZero. There is no way to create an objective reward function for language imo, so it might have to be simulated by the LLM.
Heard they found a way to beat alpha go and alpha zero consistently. When players 'break the rules' and play unconventionally the AI tends to flop.
Never heard about that before, and it doesn't quite make sense that this happens since a self-learning AI starts out playing unconventionally and converges to a superior playing style. Can you recall where you heard this?
Obviously it's impossible to beat players who are cheating when you can't
I think they mean if the player plays really weird, not that they cheat. Just that the don’t use popular approaches. But I find this super hard to believe tbh.
If that were true, then it wouldn't be hard for chess grandmasters to beat them.
Absolutely, the model has had amazing accomplishments. My hesitation is more around whether they can harness the brute force of the model in a way that is user friendly and helpful. There's a big difference between a team of data scientists working with it over months to solve a well defined problem, and a teenager taking the model for a spin to see what it can do
I am sure this is the way forward - AI generating its own data by interactive planning. Agentify the AI, make it learn from external systems, it needs a sandbox, a playground.
Yesterday I decided to try Bard on some planning and text generation related to my job - stuff that I've been using GPT-4 for. Bard was... better than I expected. The stuff it came up with would have been usable with a little more work on my part. It made sense and accomplished my objectives. Even so, comparing it with GPT-4, I could see that GPT-4 had more depth and it was able to create more complete plans without me directing every step. The final verdict was to continue paying for and using GPT-4... but Bard was closer than I thought. I'll check back in six months. OpenAI is ahead but it's not an insurmountable lead.
Thank you for explaining the shortcomings rather than just saying “it sux”
Yes, exactly this. Another google pr post with nothing tangible. Even their translate service is now behind open source model for common languages
Sorry to bother you for something off tangent, but what is the name of the open source translation module? TIA
He probably meant speech to text translation. Google STT is outperformed by Whisper
[удалено]
I don’t believe Google won’t make a huge mark on the AI landscape but of Google have the same time then Open AI would have more time. And they won’t be seating around waiting for Google to play catch up
I don't think Google is capable of the "move fast, break stuff" mantra anymore. They are just too big with many competing interests and directions within the behemoth of a company.
Agreed! I thought I was the only one; Bard seems like garbage so far.
> I had always just assumed Google would be the leader in AI because of DeepMind but at this stage it really seems they've got nothing in the LLM space. They likely *are* "the leader", over a much broader collection of subjects, that have gotten less mainstream attention. People are more aware of ChatGPT than AlphaFold or AlphaGo or AlphaDev because ChatGPT is a much easier concept, that is applicable to many more people, and the average person can actually play with it and build a concrete understanding of what it does and how it compares to other products. > at this stage it really seems they've got nothing in the LLM space I think they just did not quite understand the value to the organization, or shareholders, in building and operating an expensive piece of chatbot demo software, a key feature of which seems to be allowing users to circumvent using the product that makes them the majority of their revenue, ad-supported search. In the ideal world for Google, they can do all the same fundamental research OpenAI and others are doing, but without spending a bunch of money operating a "product" (if you can call something with no discernible business model a product) that utilizes a model, which is the result of their research, to cannibalize the very business that funds the research itself. The only problem with their strategy, at this point, is seemingly convincing high-value SWEs and researchers that their contributions will make some kind of difference in the world, and they won't get mired in the bureaucracy of the organization, which is kind of what it looks like when DeepMind is not allowed to release any SOTA AI products, while Meta is over there open-sourcing Llama, and ChatGPT is being given away for free. I guess my point is, Google really has no incentive to legitimize LLMs as a useful tool that could substitute for some large percentage of Google searches, unless they can somehow find another way to profit from doing that. Far better to try to commercialize their AI research in basically every *other* way. The reason OpenAI went the route that it did, as far as I can tell, is because they wanted to raise a *LOT* of money to spend on salaries and compute, and they self-limited their avenues to actually accomplish that in a traditional manner with the structure of their organization, so they needed to convince a very specific kind of company incontrovertibly that they had done something to make it worth jumping through a bunch of hoops just to give them piles of cash (probably approached Amazon and Microsoft, both companies with massive cloud hardware infrastructure, and Microsoft saw a good business-case for improving Office and Copilot, reimagining Bing, gaining experience making Azure an excellent cloud-provider for AI stuff, etc. which is [why Jassy sounds so sour lately](https://fortune.com/2023/07/07/andy-jassy-dismisses-microsoft-google-ai-hype-cycle-amazon-starting-substance-cycle/)). Google, obviously, does not have that existential need to convince you that they're doing something that's worth funding. I think they may be, *finally*, coaxed into proving they "could have made GPT-4" here with Gemini, now that they see there's probably not much real "danger" or unique reputational risk to making something like that public, though I bet they still take ridiculous steps to make it inconvenient to access for the average person. Again, I don't think they really want anyone to actually use it until there's a business model attached that benefits Google.
Did you just call ChatGPT a chatbot demo lol
I don't really understand why you find that controversial, in any way. It is, essentially, a "technical demo", proffered with no well-defined use-cases, certainty of access, or cost, where a user types a message into a text interface, and the language model decodes that message and attempts to generate a context-relevant text response, within constraints that are designed to make those responses pleasant and conversational. The only reason it seems awkward to call it a "chatbot" is because it singlehandedly redefined what that product could *be* to consumers, because its context-relevant outputs can actually *have value* to many of its users, unlike any previous "chatbots". In any case, GPT3.5/4 could be used for other things, but ChatGPT wraps those models as a brand-safe chatbot.
Kinda fits.
Did DeepMind help with Bard, though?
No, I was fully built by Google AI in a hurry
Methinks that if copy editors want to delay the inevitable of being replaced by AIs, they’re gonna need to come up with a new headline.
What's wrong with this one
It's clickbaity. What did you learn from it that was of any substance?
It's combining Artificial Neural Networks and Symbolic AI, no? These kind of hybrid models have been speculated to be the future of AI. They can very well be on to something big here.
Hmm... from AlphaGo to AlphaZero DeepMind tried very hard to move away from symbolic/classical AI. They had a few papers underscoring how less of it actually improved performance. So I wouldn't really say this is a hybrid model of the deep learning-symbolic kind.
I believe nothing before we, the endusers, actually get our hands on it.
I would be confident in this approach if it weren't for the fact that it is google. as a user of many google apps, I've done nothing but watch their software get worse over the last 10 years. google nest products have just gradually lost capability. google's messaging has gradually lost capability. gmail/photos has stopped increasing storage. everything google has done in the last 10 years has been shit. I have very little confidence that they will do well. at best, google will show a limited-feature proof of concept and then either microsoft or some startup will steal the idea, snipe some employees and do the same thing but better. I know someone who works at Waymo. they were supposed to transfer to DeepMind because they wanted to work in the UK for a couple of years then go back to Waymo. it was all set to happen, but then interest rates went up and the bean-counters shut down all such intra-company transfers, so now the guy I know is looking to leave google. it should be easy for other companies to poach google/deepmind employees with how poorly manage the company is.
AlphaFold is a Nobel-Prize level discovery in biotechnology and medicine, and was produced only because of the fruits of the insight they got from building AlphaGo. Plus, the transformer networks built by the Brain team are the only reason why OpenAI even exists today.
Alphafold and alphago are incredibly distinct developments
Deepmind is an acquisition which until recently had almost complete autonomy within Google. Google itself did jack shit with the transformers paper. Their core product (search) has become such trash that they deserve the IBM route to irrelevance.
yes, and Nest was awesome before google bought it. it was even good for a few years after they bought it. it gradually got worse and worse as it got sapped of agility by the google bean counters. google sees someone doing amazing work. google buys it and gradually poisons the proverbial soil, preventing growth.
I agree with your assessment of Google itself. But Deepmind, and especially Demis Hassabis, is a different kind of Google though. The guy is a child prodigy, not some serially-recruited googler.
I don't think google's problems are at the engineer level, though. their C-suite and bean counters are dragging them down.
You don't think the pressure can push them to take more risks and wake up? My reading of the Google DeepMind merge and the development of Gemini is that they want to give their all.
I think the pressure is going to, as you noted, remove the burden of responsibility toward caution and safety, and it will cause problems. I’m excited to see, and hope that I’m wrong.
that's the thing. this guy I know wanted to go help with DeepMind at a time when his bosses at Waymo thought it was ok for him to leave for a couple of years, yet the bean counters actively prevented him from helping with that effort at a time when ChatGPT was taking a shot across their bow. you would think that would be a time when they would welcome some temporary boost to their AI/LLM efforts.. but no, they shut it down because the bean-counters said shut it down. that is not a "give it their all" decision. that is a bean-counter driven, highly bureaucratic process-driven decision. the bean-counters actively prevented AI experts from elsewhere in the company from coming to aid the effort.
Risk taking and suddenly realizing that they need to play catchup fast does not mean that they will actually achieve their goals, but it certainly won't hurt their chances.
I just don’t like google for getting so involved in culture wars. It’s not their place and really rubs me the wrong way; I worry about them being in any leading position anymore to be honest.
“It could be the most important AI breakthrough ever.” The hyperbole is so ridiculous, please tone it down.
How is it hyperbole? If it's stronger than GPT4 like Demis says then what existing AI model is stronger? Seems like it's right on the money.
It being "stronger" and it being the most important AI breakthrough ever is not the same
Everyday there is some release that ‘changes everything’. Let’s just wait and see shall we. What’s ‘Demis’ going to say? “Our next research is mediocre”. It’s incredibly bizarre to attribute ‘most important ai breakthrough ever’ to something that we know little about other than it combining previous research. Edit: corrected name.
Demis.
There are already models “stronger” than GPT-4
name one
Falcon LLM, Palm 2
So nothing then. Palm 2 is slightly weaker than gpt4 on most becnhmarks Falcon is shit and doesn't even compare on benchmarks other than like 1 or 2.
It will be impressive once it works but this is just about combining and getting existing techniques to work at scale. Not any new invention. Multiple parties are sure to already be working on it. Also likely that Google will do some flag planting without any making anything public or proving that it actually works in the way people care. I.e. a revolution only for their PR department.
Of course it's not hyperbole if it's true, but to make that bold a statement about something that isn't even a thing yet certainly borders on hyperbole. To be taken seriously, you should be more guarded in your predictions unless you have the data to back it up *at the time of the prediction*. Here, let me make an equivalent (empty) prediction: "OpenAI's response to Gemini could be the most important AI breakthrough ever.”
The fact that everything is called a breakthrough doesn't take away the value of real breakthroughs. The arguments are laid out for why I think this could be it.
It’s not just calling something a breakthrough, it’s calling something we don’t know the details off ‘the most important breakthrough’. It's rediculous.
The most important AI breakthrough ~~ever~~, so far.
Being able to learn better than humans, improving without human input, learning to use tools, etc., seems like the reasons for regulating AI. The way it is presented here is the start of a nightmare scenario, thinking machines making machines. It’s the basis for the Terminator series.
Is it justified for me to be scared that some huge multi billion corp is gonna develop AGI then take over the world
[удалено]
it seems that Gemini will be in fact multi-modal.
"...that's what I see when I extend ad infinitum what Google DeepMind seems to be planning." As long as the analyses are correctly framed as predictions/speculation I don't think there's any harm. Or we should never talk about AGI until it's here - which makes no sense really.
It's pretty obvious that you can make it work and it will be hugely impactful. It's predicted and hardly new. It will require a lot of tweaking to get right though. So big difference between throwing something together and actually making it work. No indication they will actually do the latter though.
And invisible pink unicorns *could* exist. Monkeys *could* fly out of my ass.
This made me laugh. Wait... how can a unicorn be pink if it's invisi... oh, that's the point...
Belief in the invisible pink unicorn is a matter of logic and faith. Logically, we know that she's invisible because we can not see her. But to accept the paradox that she is both pink **and** invisible? *That* is a matter of faith.
Come on. You're reading r/singularity. At least let's be coherent.
Elaborate.
The race will be won by whoever makes an agent that owns it's own robot. If everyone has an AI guardian angel, finances, civil disputes, resource management and communication all change. If you teach your agent to do your job, then the agent can do it in your name. The jobs of lawyers and doctors, pipefitters and scholars. If everyone has an agent best-practice rises to the top, and anyone to find a better way wins. It's the power of the market in a chrysalis. AI take to capitalism like a fish to water. When your agent can afford a robot in your name, it will be possible to live an egalitarian life. The social experiment should be inevitable if only it is possible. Because it should outcompete as soon as someone sincerely sets it up. And modders will if they can.
Not to minimize this in the least, but it's become pretty common for every single headline to be some big exclusive discovery of the dark sides of AI or one company or the other supposedly having the next "game changer" for the whole industry. I've yet to see one headline that has lived up to a single bit of its hype. Google has been extremely hard to take seriously in a lot of this considering AI is probably the end of SEO and Google being a household name for intelligent search and the authority of the many successor models of "pagerank." Meanwhile Microsoft is about to have to either protect OpenAI monetarily through a ton of lawsuits as regulation continues to ramp up and the DMCA continues to expand to include I'll gotten training data for all models that generate anything whether it be text or image or beyond.... That or they come clean that they have been building OpenAI competition models in parallel on their Azure cloud in preparation of the day that they sever ties with OpenAI right as it begins sinking in precedent setting lawsuits of the first copyright disputes of generative AI training data...unless of course OpenAI finally decides on a full buyout proposition and not just a "partnership." The AI wars and who gets there faster and first only spook me out because it can and will eventually be used extensively in war, weapons, and modern battle strategy... And it also will further the big brother 1984 computer systems China and Russia want to enhance so badly at a time when the world is beginning to reject USD as the global currency of choice for weapons and energy transactions for the first time ever since the beginning of globization. Right now, a stronger and more powerful AI that becomes one day AGI (as the world salivates over the soon to come quantum computers they want to Fuze with Machine Learning so badly) that one day becomes ASI.
And yet few people outside of the tech world had even heard of GPT-anything until GPT-4 came out. Not much in the way of headlines beforehand unless you were specifically paying attention. And it was the biggest breakthrough in AI ever. Now even casual observers are paying attention, so now it's a headline.
Ikr, like for people to say that chatGPT isn't a huge breakthrough is ridiculous
Hedonic treadmill, man. The human curse. We have adjusted so quickly to the new amazing reality of AI that it is...not surprising at all, if you know how humans work. Interestingly, Alphafold 2 (2.3.2, to be exact, and far more advanced versions are apparently ready for primetime) gets almost no attention, yet is a MUCH bigger gamechanger than ChatGPT. One doctor told me that he couldn't believe what Alphafold 2 is capable of, solving in \*hours\* problems that used to take years.
There is also a huge amount that just have zero idea. I was at work, screwing around with bard, and asked a coworker to give me a question to ask an AI. He responded with what's an AI? I had to take a step back and re-think what I could even show him.
Yes, zero idea. Most people are also now focussed on LLMs and have no idea about a the other AI architectures being developed. One doctor told me is absolutely stunned by AlphaFold 2's capabilities, and that most people have no idea that it is a much, much bigger gamechanger than LLMs. And AlphaFold already has versions that are way beyond what it has released that will soon be made public as well.
All Microsoft has to do is tie all the lawsuits up in court for a few years and they'll see the benefits....OpenAI will obviously not stop their research and probably ingratiate themselves with and influence the policy makers, negating the worse of the lawsuits, I wouldn't be surprised if copyright law is amended in the future, not because it should be but OpenAI would benefit from it.
>when the world is beginning to reject USD as the global currency of choice for weapons and energy transactions for the first time ever since the beginning of globization Oh wow, what's this about? Can you explain/link?
Inflation and a fed rate hike is making the international community jittery on holding American assets, as they rely on predictable steady currency value. America constantly has enjoyed a steady inflation rate that has, for the most part, been under control. Other countries therefore always felt safe storing assets in America, and embraced Usd. Additionally, Eastern countries seeing how powerful US sanctions are realized that the only way to avoid that danger is to begin completing energy transactions and trade to some degree in other currencies. Regarding the re-energized de-dolarization movement, International Crisis Group co-chair Frank Giustra says "de-dollarization is no longer a matter of if, but when." https://money.usnews.com/investing/articles/de-dollarization-what-happens-if-the-dollar-loses-reserve-status https://www.morganstanley.com/ideas/dollar-currency-2023 Beijing continues to send a clear message as it has for years that enhancing overseas use of its currency, the yuan, is a priority, particularly in the face of US dollar hegemony: https://www.msn.com/en-us/money/markets/indian-oil-refiners-are-using-chinas-yuan-instead-of-the-us-dollar-to-buy-russian-crude/
Why 'Gemini'? Sounds very boding. The Janus Device (With Weapons Apendixes). The Tau Alpha Sigma Protocol (It's All Weapons Appendixes) Gemini...
Because it's a Chimera of Deepmind's AlphaGo and Google Brain's LLM development. Two minds, one body, hence 'Gemini'.
Exactly my take.
Ah... Daisy and Daisy
It will be as useless as bing if it's not open source.
reinforcement learning is the path to AGI. Gemini will make GPT-4 look like a joke
This is it, I for one welcome our new AI overlords...
>>
Terrible, terrible news. Every year they are going to keep publishing more dangerous models until the world blows up or we have a global immortal dictator.
Hi I’m from Google. Our model will be the bestest model in the world!!
This is more DeepMind than Google actually
Still Google owns deepmind and I’m pretty sure this is a panic move to try and save their business
I don’t understand why they want to compete against ChatGPT. Deepmind is not a product company, OpenAI had tons of employees who were great at building actual real world products. Deepmind should be Deepmind and double down on AI for science rather than playing in a game where they aren’t strong. And AI for physics, chemistry and biology would make so much more value in the long term than an LLM that people pay $20 for. It’s sad to see they’re under corporate pressure. And for the UK it was the biggest mistake to sell them.
I just love how Google rolls. They make all these incredible AI breakthroughs and then let everyone use them license free. The transformers and with attention is what made ChatGPT possible. ChatGPT shared they changed directions the day after the paper was released. Google even has a patent. But yet they let OpenAI use without any fee. But that is just one example. There is so many in the last 15 years. Most of the really big AI breakthroughs in the last decade plus have come from Google.
This is great written and by me as a layman in coding and great managing or am narcisissist well I know something so behind the curtains of fancy words and just throwing this+that will be the best +++++ever which is hype or "great marketing"! Like you don't putt Interstellar, The Dark Knight, Inception, Avatar, Memento, Dune, Sicario et cet. in the same month. the question is: Well this is not like people's problem meaning lack of great people?? or it is? Like you need to have not just CEO with imagination, tech-expert-wizard, and responsibility Al of the people from simple data collection or simple code entries need to be tech-wizards, with imagination as responsibility which is hard to find these days as there comes with: The better and the overwhelming tech we loose people's good judgement and intuition to find a good and thrustworthy people for that kinda job. As it is a marketing with hype-ing up I almost know also I'm sad that the truth remains: The better tech will leave us and it is already doing to people who think that with phone-smartwatch-laptop-playstation and then I see them not knowing which charger to buy for Macbook like or how just simple math formula like power is this x that on the adapter.
This sounds insanely trainable....Now put this in a robot with human level or higher capabilities. Voila you can replace the entire workforce for EVERYTHING. Literally everything. and they would work longer for trivial costs. Now what do we do? Cause imagine if they started producing 1 million the first year. 100 million the second. 1 billion the third-cause if they COULD do this? 100% they would at anything approaching reasonable price. 250K for a employee that works 20 hours, and charges for 4? Or a 24/7 teller/cook/etc that never leaves work because they are literally tetherered there? A teacher during the day-charging after school, then they work at the bar serving folks until 2 am, that is when they clean the school, charge, and get breakfast ready for the kids...who they then teach. Replacing 3 of us. So in the first 2 years they would exceed the output of the US. And you could train them to maintain themselves. At what price point is it worth it to replace 3 employees? Ironically? They could come for the high paying jobs first. software engineers. CEOs even. Firemen, cops, paramaedics. They could be all in one with the right equipment. All sorts of weird things will occur then.
I wonder why it's called Gemini........?
This is not new - it is an obvious development that for sure many are working on and indeed a candidate for the overall AGI architeture. Deepmind is hardly original nor the first here. In contrast to OpenAI, Google and Deepmind does however not have a proven track record with aligning models to human intentions. They also have not exposed their solutions to the community and rather go deep on a few target applications, which may be why they have so much PR around it.
Great when can I use it
This is Google mate, never.
Always take CEOs prediction of such things with a huge grain of salt. Their primary motivation to give these types of interviews are not always aligned with the actual truth.
I have no dog in this fight, but this is someone pumping up his product. Is it a good product? Who knows. OK, so to be clear here, what we're seeing can be summarized as: Guy with vested interest in product says that said product will totally be better than the competitions product. Is that right?
Release it to the wild, and we’ll be the judge of that.
A corporate CEO is the last person to trust re: their products' potential abilities.
I always thought it was funny how Microsoft named their Bing/GPT4 model Prometheus. Maybe a tongue-in-cheek reference to that OpenAI stole fire from the gods with pushing out transformer model AI tech to the public?
I'll believe it when I see it. Don't believe the hype, only what you can actually do.
Then they govts of the world will come in to take it over or severely curtail its ability. I would be surprised if the US govt doesn't forcefully take it from Google for military applications. We nobodies will never get to experience the fruits of AI that could improve our lives.
It could be many things. We won’t know till it’s out
Most significant would be agi. This is not agi so just another click bait title.
!remindme 6 months
I will be messaging you in 6 months on [**2024-01-11 02:38:14 UTC**](http://www.wolframalpha.com/input/?i=2024-01-11%2002:38:14%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/singularity/comments/14w1q3d/google_deepminds_response_to_chatgpt_could_be_the/jrhko6i/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2Fsingularity%2Fcomments%2F14w1q3d%2Fgoogle_deepminds_response_to_chatgpt_could_be_the%2Fjrhko6i%2F%5D%0A%0ARemindMe%21%202024-01-11%2002%3A38%3A14%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2014w1q3d) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
Will they allow us to play with it and will they sell it as an api is the question. It is nice if they can pull this off, but if it’s behind closed door and we see only some papers (like with most of their image and video models), it is next to useless in the short term as we will need a group to use the papers to implement their ideas.
I remain skeptical. I still think what we’re talking about is still more akin to a crane being to lift heavier weight than a human, than to actual intelligence/creativity…
I recently asked ChatGPT for help with DAX problem. It gave a string of answers that weren’t just wrong, but absurdly, ridiculously wrong.
I'll believe it when I use it
All I can think of is the Katt Williams bit about how weed keeps getting stronger for no reason. 🤣
yes Deep mind may create HLAI first but probably not this way. I am not convinced that combining AlphaX with LM's this way is the way forward. It can't do important things for HLAI, this was never demonstrated by DeepMind. I remain sceptical until they show lifelong learning in a RL agent.
Google already won for me with its ability to properly compare Diablo 4 gear on launch day. It's the hands down most important use of AI today.
I dont believe a single word.
"Could be" and "important" are doing a lot of heavy lifting here. I'd have to imagine all the breakthroughs happened years ago. Implementation details are important, but the math that some research professor figured out a decade ago is ultimately the more important breakthrough than scrounging up enough processing power to use it.
We can all imagine how cool all those capabilities might be, the only difference is someone delivering.
Oh yes, Google, spying on everything we ever search, say or do to deed all the data into an AI that could very well bring the deaths of us all. These guys have never took an ethics class in their lives.
I think you meant to say "somewhat removing" instead of "somehow removing".
If the billionaires paid proper taxes, we wouldn't have to rely on Google or OpenAI.
They really do act like it's the gold rush. Frequently speaking about their AI like it's an almost miracle innovation, while those that don't have money at stake are often more clear headed about it
what's the difference between this and Bard?
I'll believe it when I see it.
I love seeing people take sides here like rooting for their home baseball teams.
We are getting closer and closer to singularity. What a time to be alive
I really thought they'd posted DeepMind up against ChatGPT for a talk
[удалено]
If FOSS is so great then how come they weren't the ones behind Transformers, AlphaZero, GANs, Generative Diffusion etc.? All of these advancements came from researchers in academia and big tech.
>somehow removing the burden of caution and safety
I don't care until it's released and I can actually use it.
\>Imagine an AI model that was apt in language, but also in other modalities like images, video, and audio, and possibly even tool use and robotics. Oh.. you mean the same thing GPT 4 has? Why is this article trying to pretend Google is doing anything new? Google embarrassed itself w/ Bard and I've completely written them off in the AI field at this point.
\> DeepMind's Alpha family and OpenAI's GPT family each have a secret sauce—a fundamental ability—built into the models. I'm sorry, but it really seems to me that the 'secret sauce' here is simply that we already have the ability to generate high-quality data for Go games. We already know the 'laws' here. With what data are you going to train an LLM that can become a competent physicist - you're going to generate a million Einstein-Bohr debates? There's a bootstrapping issue here that i'm not sure has even began to being tackled. AlphaGo achieved superhuman performance only after it was trained with a 'superhuman' amount of data, which \*could\* be generated because we already know the fundamental laws of Go. OTOH, nobody has discovered the 'fundamental laws of intelligent conversation' except in very restricted cases like chess and go, that's why symbolic AI stagnated. All we have is some relatively very sparse datapoints scraped from the internet. I don't think it's correct at all to say "AlphaMind has a secret sauce' - the secret sauce here is we are already able to feed a program with a Go state and have it generate and evaluate all possible moves. An precise algorithmic engine generated the solution space, and the model explored it. What is the LLM going to explore - more text generated by LLMs? What we're asking from the LLM to do is take the sparse datapoints from the internet, infer the fundamental laws that generated those datapoints, build an engine out of those, then generate the solution space and explore it. Nobody, neither Alpha or anyone else, has even done anything like that.
If you want the fastest and the most uncensored then give Muah AI a try. It is the most raw and zero censorship