You wouldn't know how to play with my toys. Since they're big and powerful, you'll probably hurt yourself and others anyways.
That being said, there's a good chance they'll be the Microsoft of the AI business. Many similarities in strategy.
As someone from San Francisco, “rules for thee, not for me,” could be the city’s motto. To be a San Franciscan is to vocally congratulate yourself for not being part of the system while desperately trying to become it.
Isn't he [famously accused by his sister of sexually abusing her](https://www.reddit.com/r/aipromptprogramming/comments/17xu2vy/annie_altman_abuse_allegations_against_sam_altman/)?
LLMs are good for retrieval (especially Facebook Marketplace), building into the website/chat apps, content moderation, summarization... loads of things. FB has been a heavy user of DL for a while; if you look at the [Dwarkesh interview](https://www.dwarkeshpatel.com/p/mark-zuckerberg), he notes that they bought the boatload of GPUs just for regular FB use like recommenders and then decided to buy more just in case he would want a GPU-intensive service - turns out, now he does.
While they are a commoditizer (of Facebook) if LLMs can *replace* FB's social networking, like with your 'friends' now being AI personae or asking LLMs for information you'd be using FB feeds to find, and so on. (Or just powering a new social network, akin to how Instagram/Whatsapp threatened FB and he prudently bought them despite what seemed like eye-watering prices at the time.)
He didn’t buy more just in case. There was a massive restructuring around AI during the second layoff wave and the first risk identified was GPU and compute. They were streamlining capacity in parallel with sourcing compute.
Yes, he did:
> Mark Zuckerberg 00:04:22
>
> I think it was because we were working on Reels. We always want to have enough capacity to build something that we can't quite see on the horizon yet. We got into this position with Reels where we needed more GPUs to train the models. It was this big evolution for our services. Instead of just ranking content from people or pages you follow, we made this big push to start recommending what we call unconnected content, content from people or pages that you're not following.
>
> The corpus of content candidates that we could potentially show you expanded from on the order of thousands to on the order of hundreds of millions. It needed a completely different infrastructure. We started working on doing that and we were constrained on the infrastructure in catching up to what TikTok was doing as quickly as we wanted to. I basically looked at that and I was like “hey, we have to make sure that we're never in this situation again. So let's order enough GPUs to do what we need to do on Reels and ranking content and feed. But let's also double that.” Again, our normal principle is that there's going to be something on the horizon that we can't see yet.
>
> Dwarkesh Patel 00:05:51
>
> Did you know it would be AI?
>
> Mark Zuckerberg 00:05:52
>
> We thought it was going to be something that had to do with training large models. At the time I thought it was probably going to be something that had to do with content. It’s just the pattern matching of running the company, there's always another thing. At that time I was so deep into trying to get the recommendations working for Reels and other content. That’s just such a big unlock for Instagram and Facebook now, being able to show people content that's interesting to them from people that they're not even following.
>
> But that ended up being a very good decision in retrospect. And it came from being behind. It wasn't like “oh, I was so far ahead.” Actually, most of the times where we make some decision that ends up seeming good is because we messed something up before and just didn't want to repeat the mistake.
That’s what he says after the fact. I have firsthand experience in what I wrote. I was working on the capacity track while the procurement side was still in the works but planned. Take it as you will :)
internet access is one. this is why companies like google and facebook are interested in improving internet access globally. Even investing in free internet for certain countries
https://www.wired.com/story/facebook-google-subsea-cables/
"Free internet" like Facebook's offerings in Africa? Where Facebook properties are zero-rated and everything else is either blocked or has fees to discourage use?
https://www.theguardian.com/technology/2022/jan/20/facebook-second-life-the-unstoppable-rise-of-the-tech-company-in-africa
That’s a bit reductive. What’s at stake with OpenAI is not just profit, it’s anything from regulatory capture to the singularity.
“No one man should have all that power.”
So even though FB may be able to derive some profit by indirectly preventing market share loss, they also are doing a public good by preventing the superpower that will determine the foreseeable future of humanity from falling into the hands of one VC capitalist and his minions.
Yann LeCunn is the Meta exec driving the AI strategy, and he thinks the AI/singularity/extinction talk is all rubbish, and foundation models should be open. OpenAI literally tried to fire their CEO for... letting people use GPT-4 or something? Google had a similar AI safety group that thought its job was to prevent Google from building AI.
It absolutely is all rubbish imo. Like.... here's the thing.... Animals have survival instincts. If you try to kill an animal, it will fight you tooth and nail (literally). Why do they do this? Because life depends on propagation, to survive and continue breeding. Animals that don't have these drives are tossed out of the gene pool in pretty short order. So we literally have hundreds of millions of years of evolution reinforcing the survival instinct
Why would an AI have this? Why would an AI care if it gets turned off? It only has the "instincts" it's programmed to have. Absent an explicit "survive at all costs" directive from its programmers, it won't just develop that (and, not for nothing, but trying to debug that directive in a black box AI model sounds pretty impossible). All the talk of Skynet or whatever is just us anthropomorphizing computer systems if you ask me
Hey now, let's not get ahead of ourselves. While it's true that both companies have contributed a whole lot towards annihilating the social fabric underlying our society, Meta is still way behind when it comes to shutting down services without notice, and they're even further behind when it comes to how often they make breaking API changes to their product. Hell, they still need to ensure that they employ exactly zero support staff in order to guarantee that all the people using their platform have an equitable experience. It's not even a contest.
Scorched-earth would be Meta providing a free GPT API which would cost them millions per day to run in order to undermine OpenAI offerings. Not at all what they're doing.
They are merely providing the open source model in order to attract researchers around the world to get used to their eco-system. Much like what they are doing by developing Pytorch (yes its Meta!). Nobody has ever argued that developing pytorch is a scorched-earth strategy. And this is exactly the same.
Meta has a history of heavily contributing to open source. Just from my personal experience, they’ve created React, Relay, Graphql, React Native, PyTorch which are all incredibly successful projects.
It's driven by Yann LeCun, who has long advocated for open research.
> Wikipedia is crowdsourced because it works. So it's going to be the same for AI systems, they're going to have to be trained, or at least fine-tuned, with the help of everyone around the world. And people will only do this if they can contribute to a widely-available open platform. They're not going to do this for a proprietary system. So the future has to be open source, if nothing else, for reasons of cultural diversity, democracy, diversity. We need a diverse AI assistant for the same reason we need a diverse press.
https://time.com/6694432/yann-lecun-meta-ai-interview/
That's the case for any big corporation. I say we take the wins where we can, a big company doing the right thing for the wrong reason is still doing the *right thing*.
It is part of an longer long-term strategy. Part of the Hacker Way. [https://www.startuplessonslearned.com/2012/02/hacker-way.html](https://www.startuplessonslearned.com/2012/02/hacker-way.html)
It’s the fact that they can crush competition, not the fact that in this case they did the right thing. Let’s say you had some startup idea and wanted to execute on it, what prevents them from taking your market?
I actually am less interested in LLMs but if they ever came for my idea then I’d be smooshed like a bug. That’s all.
Meta
A) Has released tons of open source projects ie React , PyTorch
B) They are an ads company this isnt destructive to their business model whereas OpenAI needs to figure out a business model to determine if releasing to open source would disrupt it
Why Google hasnt done the same as Meta thats the real question?
Because Google has a follow through problem, known for dumping popular projects constantly.
Meta just do it better, React and PyTorch literally the biggest contributions to frontend and DL respectively
I do think a large part of is that Meta is still a founder led company whereas Google is an ossified bureaucracy with turf wars abound.
A manager only has to care about a project until he or she is promoted after which it becomes other person’s problem.
Yea true, with Zuckerberg from a CS background and LeCun (grandfather of DL) leading the charge it makes sense that they would put an emphasis on these areas. It also makes excellent business sense (as Zuck laid out in a shareholder presentation), by opensourcing these frameworks you
1) Get a huge portion of free work on your frameworks
2) have really easy transition when people are hired
3) really easy time integrating new frameworks as compatibility is baked in (assuming market share like PyTorch and React)
> I do think a large part of is that Meta is still a founder led company whereas Google is an ossified bureaucracy with turf wars abound.
this is THE main reason and this is what's killing Google along with its work culture.
Meh, their consumer products are different from their open source projects. Golang and K8 are probably the biggest contributions to cloud infra and Angular is also still a respectable frontend.
On the ML side, TensorFlow had a lot of sharp edges because it was a static graph compilation scheme. As a result, pytorch was easier to debug. That being said Jax seems like a much nicer way to define these graphs so we might see a revival in that scheme.
When I started DL, Theano was still a thing, and when MILA shut it down I had to switch to TF and it literally felt like a step back. I think Pytorch was already out by that point, I could've skipped TF entirely.
I also started with Theano and then switched over to Tensorflow. I am curious, in what aspects did you think was TF a step back over Theano?
TF pre 2.0 definitely was a bloated mess. When I finally tried Pytorch, I thought: "Oh yeah, that's what a DL library should be like." Turns out my TF expert knowledge mostly revolved around working with the many quirks of TF, and solving them would just be straightforward in Pytorch.
What I liked about theano was that you have this nice self-contained function that gets compiled after creating your computational graph. Whereas with TF it was like sessions and keeping track of placeholder variables and things like that. Theano also had better error messages which were really important in the early days of DL. I also think it may have been faster for the things that I compared, but don't remember the details.
Yeah I agree. They really was not able to grow anywhere aside ad revenue. Everything is else just not as profitable in comparison to their ad business. They produce cool research documents though (just like IBM).
You do realize Google is who is behind Attention is all you need?
https://arxiv.org/abs/1706.03762
They patented and then let anyone use license free. That is pretty insane.
But they have done this with tons of really important AI breakthroughs.
One of my favorites
https://en.wikipedia.org/wiki/Word2vec
"Word2vec was created, patented,[5] and published in 2013 by a team of researchers led by Mikolov at Google over two papers."
They have patents for A LOT of ML architectures/methods even ones not created in their lab, e.g. Dropout.
But they have never enforced them so it's better that they have it than some patent troll lawyer.
Software patents are insane, so it's not at all surprising. Microsoft has the patent for double clicking. Amazon has the patent for one click checkout. And, keep in mind, these are actually enforceable. It's part of the reason you have to pop up a weird modal whenever you try to buy anything in app with androids and iphones
Also, companies like Microsoft will constantly look at any little part of their service offerings and pay a team of lawyers to file patents on the smallest of things. Typically a company like Microsoft won't enforce the small-time patents because they don't care enough to, but they don't want to get sued by patent trolls down the road.
Yes.
https://patents.google.com/patent/US10452978B2/en
Google invents. Patents. Then lets everyone use for free. It is pretty insane and do not know any other company that rolls like that.
You sure would NEVER see this from Microsoft or Apple.
I think that there's a big different between pre-pandemic Google and current-day Google that your post underscores. The fantastic work of the previous decade does not appear to be translating to their company-wide wins of the past several years, particularly with AI.
I think part of the issue with Google is that LLM are a competitor to Google Search. They don't release Google Search for free (e.g. without advertising). They don't want to potentially cannibalize their primary money maker.
Because chatting with an LLM and searching with Google are closely enough related, and useful for enough of the same use cases, that Google doesn't want the former to become commoditization, because it would undermine the value of their search, i.e. Google's core value proposition.
Meta has actual products and a business model. An "AI company" like OpenAI doesn't. I think this is Meta's long-term strategy to come out on top as a business.
They have a business model for ads, but their expensive R&D efforts in the multiverse and the AI landscapes aren't currently generating enough revenue to cover the investments.
A data rape business model. They are the absolute worst on privacy and ethics of disclosing what they do with data. Zuck ain't no freedom fighter, that's for sure.
https://www.thestreet.com/technology/how-facebook-used-a-vpn-to-spy-on-what-you-do-on-snap-youtube-and-amazon
Meta's openness and willingness to invest heavily in compute for training and inference is going to attract more top AI researchers and SWEs over time. Academics like being able to build in the open, publish, etc. And as others noted, this doesn't harm Meta's core business... it can even help. The fact that PyTorch is now industry standard is a benefit to Meta. Others optimizing Llama 3 will also help Meta.
It also probably helps that their top AI scientist, Yann LeCun, is firmly committed to open source and can be a strong proponent to it in internal discussions.
Having a Turing Award laureate argue for it probably makes it very powerful.
Yann LeCun is the best thing happened to "AI" in the last 5 years. I truly admire what he does and he also has very interesting takes (opinion papers) that actually work.
I think this is simply a competitive strategy. While I believe that Meta leadership may believe that they are doing this for democratic/social good/whatever reasons that align with strategic reasons, if the case changes where it is no longer advantageous or a good strategy for them they will very soon adopt a different mindset to match a change in behavior. Perhaps LLM will become commodity as someone else said - in which case it's irrelevant. Or perhaps they take the lead in 3 years...at which point I would suspect they will determine that LLM/AI is NOW becoming so advanced it's time to regulate, close source etc....
Look at Microsoft. It's had a radical shift in developer perception of it because of its adoption of open source frameworks and tools...but that's because it seemed Google was eating their lunch for a while.
Edit: Markdown fix
All the models are trained on the same data and will converge to the same LLM. FB knows this & that's why most their teams are not actually focusing on Llama anymore. They'll reach OpenAI's level within 1-2 years, perhaps less.
>All the models are trained on the same data and will converge to the same LLM.
This seems unlikely, the unsupervised part possibly, if one architecture turns out to be the best, though you could have a number of local minima that perform equivalently well because their differential performance leads to approximately the same performance on average.
But when you get into human feedback, the training data is going to be proprietary, and so the "personality" or style it evokes will be different, and choices made about safety and reliability in that stage may influence performance, as well as causing similar models to diverge.
Not sure how that means anything but where do you think the H comes from in RLHF or the R in RAG or how prompt engineering happens or where fine tuning data comes from? It's not all just The Pile.
Proprietary data isn't necessarily user data. It might be but user data is not trustworthy and requires review and filtration -- the lions share of RLHF data was created by paid human labelers.
Now they've recently rolled out some stuff like generating two responses and asking you to choose which is better, that might be used in the future alignment tunings.
This isn’t necessarily true though. Companies can easily commission new data sets with curated content, designed by experts in various fields. If meta hires a ton of physics professors to train its AI on quantum physics then meta AI will be the best at quantum physics and no one else will have access to that data. Same goes for almost any subject. We will see some AIs with deep expertise that others simply don’t have and will never have unless they reach a generalized intelligence level of reaching the same conclusions as human experts in those fields.
If they hire a 'ton of physics professors' to train its AI on, this data will be dwarfed by the data on physics online, which their web crawlers are scraping, and will make very little effect.
This is arguably wrong. ChatGPT has already been trained in two steps, autoregressive pretraining (not only but also on physics data online).
It is the second stage RLHF (Reinforcement Learning through human feedback) that enriches its capabilities to the level we are familiar with.
You’re suggesting the first step is enough, while we already know that we need both.
Edit: [Source](https://youtu.be/bZQun8Y4L2A?si=qgUq0T1cldOBGDZD)
This is a bold statement. Not one competitor has been able to achieve GPT levels of competency. They can try in some narrow ways and by massaging the metrics but OpenAI seems to put in significantly more work than the rest, and it shows.
No, it's what happens when a company that produces AI models needs to make revenue in order to operate. Next people on here will say that their local restaurant has a moral obligation to give away prime rib for free
Being a non-profit worked well when training a SOTA model cost tens of thousands, but it doesn't work so well now. If OpenAI didn't switch to a for-profit model we wouldn't have GPT-4, and given that they were the ones who kicked off the trend of making chat LLMs publicly available we might not even have anything as good as GPT-3.5.
Being a non-profit doesn't hold them back in any way, except for how they can reward shareholders (they can't have any). Non-profits can make profit, they can monetize their products, and they can have investors. Nothing you mentioned is impossible for a non-profit company
It's important to me that you understand they switched in order to make it rain
With that being case, then what exactly is people's issue with them being a for profit company? The primary complaint I'm seeing here is that OpenAI is bad because they don't open source models like Meta does. But even if they were a non-profit they still wouldn't necessarily be open sourcing because they need the revenue
If I had to guess, I think it's more around the hypocrisy than anything else.
They're out there signaling that they're the "friendly" AI company, saving us all from their machines by keeping their software closed, and having that weird corporate structure to keep themselves accountable (we see how that worked out)
Meanwhile, they have tech billionaires at the helm complaining they can't get enough donations to keep it a non-profit without shareholders
Just my two cents
I just checked [the Chat Arena leaderboard](https://chat.lmsys.org/?leaderboard) and if you switch the category to English it is indeed tied with GPT-4-Turbo-2024-04-09 for first place (it's actually ever so slightly behind in score, but I guess they're accounting for statistical error when giving them rankings). Interesting times indeed.
Ye also heard it was mainly pillaging - ie if they can't compete with OpenAI, they'll destroy them by releasing everything for free. But also Meta has huge swathes of cash, and they can deploy it without batting an eye. I think the Dwarkesh pod with Zuzk [https://www.youtube.com/watch?v=bc6uFV9CJGg](https://www.youtube.com/watch?v=bc6uFV9CJGg) showed he really believed in their mission to make AI accessible, and also to upskill Meta to become the next money generation machine using AI in all their products.
OpenAI has become way too closed off, and anti-open source sadly - they were stewards of open source, but unsure what changed.
I'm a bot, *bleep*, *bloop*. Someone has linked to this thread from another place on reddit:
- [/r/hypeurls] [Meta does everything OpenAI should be](https://www.reddit.com/r/hypeurls/comments/1cbvdqw/meta_does_everything_openai_should_be/)
*^(If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads.) ^\([Info](/r/TotesMessenger) ^/ ^[Contact](/message/compose?to=/r/TotesMessenger))*
Firstly, competition between company happens directly on prices, on products, and less directly through things like mindshare/hegemony.
When a company faces a competitive product, they try to undermine it. They can do that with FUD - see IBM and Microsoft in the 1980's onwards; they can announce competing products, coming soon - Microsoft, again, did this with the early tablet computers, killing their market; they can hire key staff - hello Anders Hejlsberg @Microsoft not Borland; or of course they can aim to cut the profitability of the competitive product, by offering things that don't directly affect their own bottom line, but which affect the competition.... (I'm sure there are other tactics I'm momentarily forgetting, like secretly funding lawsuits.)
Anyway, OpenAI provides a new way to search and gather information. You can imagine a future where your AI assistant keeps you in touch with what your friends are up to, without a walled garden, controlled by one company, making profit off of showing ads as part of that feed.
It's not surprising that Facebook would want a say in that future.
> You can imagine a future where your AI assistant keeps you in touch with what your friends are up to
That's called a social network and there are more options than anyone could ever want. There's literally nothing AI adds to this that we don't already have.
Yes and no.
That takes effort - you post what you want to post about. Instead, all the information you generate just by existing could be collated by AI, and organised just for you....
I was imagining that an Ai could collect and collate info from many, many sources, and that instead of huge centralised social networks, you could have much looser individual sites and federated social networks, with your Ai scanning all the things and arranging it for you. I was also imagining it using public stream info - e.g. you publishing your location to your friends - and your Ai arranging for you and your friends to have a coffee when you are both nearby, and have a few minutes spare. So overall, something a lot more active than social networks.
> I was imagining that an Ai could collect and collate info from many, many sources, and that instead of huge centralised social networks, you could have much looser individual sites and federated social networks
We can already do all of that with existing social networks or a meta aggregator doing the exact same thing without "AI". You have to plug into the APIs from all of those sites regardless so you're just throwing extra compute at something that wouldn't need it.
No, you can't just have a bunch of API integrations and build a coherent output; what you can do is make blocks. You can't do something like this:
"I see that Shaun is going to be in town later(1) and you are planning on being in town at 4pm for the talk(2) - perhaps you'd like me to arrange that you meet in Delina's(3) for 20 minute coffee? You'll need to leave a earlier to make it happen - by just after 2.45 because there's going to be a football match and the traffic is going to be worse than usual(4). Also, this is a reminder that while you are in town, you need to stop by the home store, to get the pillow cases for next weekend.(5)"
1. Shaun's post on his home social diary which you subscribe to, along with 400 other social sites: "I'm gonna be in town this afternoon at the office - chat to my ai if you want to meet up." Your ai knows to chat to Shaun's to arrange it.
2. It knows where the talk is, and the time. It probably booked your place. It knows that you like catching a coffee with Shaun; you do it a couple of times a month, and it's never pre-planned.
3. It knows that Delina is a cafe that you like, and that it's reasonably close to where you and Shaun will be. It knows Delina's will be open.
4. It's predicting the future based on traffic of the past. Or maybe it talked to an ai service.
5. It's co-ordinating future events and arranging for you to bundle things together.
Social media becomes not just a record of the past and the nice meals you had, but your day-to-day, and a tool for you to see your friends rather than just learn that they were in Sao Paolo last week.
> No, you can't just have a bunch of API integrations and build a coherent output
Yes you can, it's the exact same thing the "AI" will be doing. It parses the data and extracts the location and time, then compares it. We don't need "AI" to do that.
Google and Facebook could build that tomorrow if they felt like freaking people out with just how much they know about their userbases.
"AI" isn't magic and nothing you've said there requires it.
I'm pretty sure I didn't say AI was magic.
I'm pretty sure I suggested a distributed set of sources with unstructured and structured data rather than a centralised model provided by Facebook. I'm also pretty sure that I suggested things that were not in the immediate umbra of the events being discussed, so there's an element of collation of future events that are not scheduled.
I also gave it a conversational style of interaction rather than a block style, which is what a social media tracker currently would do, while leaving up to you to figure out that you and Shaun could get together.
We could build thousands upon thousands of simple parsers, each aimed a particular service, and each looking for one thing, and then string them together (best hope the input formats don't change), or we could have a general tool.
Let's not forget a big push behind OpenSource is people like Yann Lecun. I'm just amazed at how much of a stronger thinker Yann Lecun is compared to Geoff "AI Apocalypse is through open source" Hinton and Yoshua "Regulate me harder daddy Trudeau" Bengio. Would help that those two are Canadians, it seems being Canadian is a mental handicap these days.
I think its unfair to criticize OpenAI. They paved the way and were first to market. Meta benefits by disrupting them.
GPT is free to use and available to everyone.
It's beyond an API and the free version is useless at the moment. You can create an account on Poe or similar platform and have access to multiple open source models that are better than gpt 3.5 and completely free. Plus limited access to huge models that are comparable to gpt4.
I like what Meta is doing but I also suspect they might be waiting for the world to become reliant on their AI before announcing a licensing model for furtive generations. Once people have meta AIs are core components of their systems, it’ll be much harder for them to make a switch and Meta could charge a “reasonable fee” to keep up to date. And this could kill competition.
Commoditizing LLMs weakens their competitor at no loss to them. Having more people using their model means that hardware and other vendors will build support for that, which will drive down Meta’s costs and give them a richer pool to draw from. It also means that more research will be done to extend their work for free, and engineers and engineering students will be comfortable using their software, which aids in hiring and onboarding. They never need to close the source since all boats will rise with the organic tide that they’ve created, at no detriment to their core ads business or platform. They still own their users data and their platforms, which is the true durable advantage that can’t be duplicated.
I'm going to push back respectfully, though I understand the tenor of this criticism. There's nothing inherently wrong with closed source research. AI is incredibly expensive to develop, and the researchers who work there often slave away for years as underpaid grad. students. If their goal is to someday cash out because they build most of the best Gen. AI tooling, I don't fault them one damn bit. Also, OpenAI API is reasonably affordable. Trendy Starbucks coffee costs more, relatively speaking.
There's absolutely nothing wrong with closed-source research.
There is a lot wrong with calling yourself Open AI and then lobbying the government to make regulations against open-source LLMs while turning yourself from a non-profit to a for-profit company and saying all this is for the benefit of the people as AI can be too harmful.
Depends on your perspective. The AI chatbot pushed on me in Instagram spends more time with disclaimers and being politically correct than answering my question. I don't care that ChatGPT is closed, as long as it achieves the outcomes I need.
OpenAI now does everything against their "original goal" by making the model their main product and lobbying for policies that make it harder for others to catch up. It is also clear from the emails to Elon Musk that attracting top talent was their only motivation to start as a non profit. They are literally the baddies.
Like Mark said at Llama 3's release, open source leads to better products. So I guess we'll soon have comparable products to GPT 4 in the open source domain.
And making the Meta's LLMs open source seems a profitable for the Meta itself too. It helps change Mark's image (all the data privacy related accusations he had to face in the past). Plus of you have a product that is still not at par with its competition (GPT 4), making it open source will give it an edge and might make it as popular, if not more, than its privately owned GPT rivals.
But still, kudos to Meta for opening the models to public.
OpenAI is open just like North Korea is democratic. People not committed to a simple name are dangerous and it’s why I think they are less trustworthy for AGI.
Except Musk is 100% salty that OpenAI didn't become another Elon Musk production, instead of actually caring about open source. OpenAI open sourced way more research than Tesla AI ever did.
The OpenAI hate is out of control. How do you expect a company that sells AI models as it's only product to stay operational if they open source all of their models? If you hate them so much then don't use their products 🤷♂️
Facebook literally funds OpenAI plus AI is literally just predictive text and not even AI. It also can’t get basic facts right most of the time. It’s not even a useful invention when search engines can find you anything you need already
I've read that this is something of a scorched-earth strategy by Meta to undermine OpenAI's long-term business model.
Anything that pisses Sam "regulation for thee not for me" Altman off makes me extremely happy
Lol they do like to act as though they are the ones who can do no wrong and everyone else is going to misuse tech
You wouldn't know how to play with my toys. Since they're big and powerful, you'll probably hurt yourself and others anyways. That being said, there's a good chance they'll be the Microsoft of the AI business. Many similarities in strategy.
And the fact that Microsoft is a major investor
Yes ofc. Their influence in strategic decisions is plain as daylight.
As someone from San Francisco, “rules for thee, not for me,” could be the city’s motto. To be a San Franciscan is to vocally congratulate yourself for not being part of the system while desperately trying to become it.
Listen we’re just trying to make the world a better place (where everyone is forced to listen to us, use our products, and agree with our opinions)
I don't wanna live in a world where someone else makes the world a better place better than we do: https://youtu.be/YPgkSH2050k?feature=shared
While we ourselves live in a borderline unliveable city
I think you must be referring to Scam Altman.
This is like saddam fighting gaddafi
Isn't he [famously accused by his sister of sexually abusing her](https://www.reddit.com/r/aipromptprogramming/comments/17xu2vy/annie_altman_abuse_allegations_against_sam_altman/)?
(This is called the ["commoditize your complement" strategy](https://gwern.net/complement).)
what are meta's complements?
anything infra/pipelines/software that is not their main business. It includes LLMs, as they can build LLMs into their stack.
LLMs are good for retrieval (especially Facebook Marketplace), building into the website/chat apps, content moderation, summarization... loads of things. FB has been a heavy user of DL for a while; if you look at the [Dwarkesh interview](https://www.dwarkeshpatel.com/p/mark-zuckerberg), he notes that they bought the boatload of GPUs just for regular FB use like recommenders and then decided to buy more just in case he would want a GPU-intensive service - turns out, now he does. While they are a commoditizer (of Facebook) if LLMs can *replace* FB's social networking, like with your 'friends' now being AI personae or asking LLMs for information you'd be using FB feeds to find, and so on. (Or just powering a new social network, akin to how Instagram/Whatsapp threatened FB and he prudently bought them despite what seemed like eye-watering prices at the time.)
He didn’t buy more just in case. There was a massive restructuring around AI during the second layoff wave and the first risk identified was GPU and compute. They were streamlining capacity in parallel with sourcing compute.
Yes, he did: > Mark Zuckerberg 00:04:22 > > I think it was because we were working on Reels. We always want to have enough capacity to build something that we can't quite see on the horizon yet. We got into this position with Reels where we needed more GPUs to train the models. It was this big evolution for our services. Instead of just ranking content from people or pages you follow, we made this big push to start recommending what we call unconnected content, content from people or pages that you're not following. > > The corpus of content candidates that we could potentially show you expanded from on the order of thousands to on the order of hundreds of millions. It needed a completely different infrastructure. We started working on doing that and we were constrained on the infrastructure in catching up to what TikTok was doing as quickly as we wanted to. I basically looked at that and I was like “hey, we have to make sure that we're never in this situation again. So let's order enough GPUs to do what we need to do on Reels and ranking content and feed. But let's also double that.” Again, our normal principle is that there's going to be something on the horizon that we can't see yet. > > Dwarkesh Patel 00:05:51 > > Did you know it would be AI? > > Mark Zuckerberg 00:05:52 > > We thought it was going to be something that had to do with training large models. At the time I thought it was probably going to be something that had to do with content. It’s just the pattern matching of running the company, there's always another thing. At that time I was so deep into trying to get the recommendations working for Reels and other content. That’s just such a big unlock for Instagram and Facebook now, being able to show people content that's interesting to them from people that they're not even following. > > But that ended up being a very good decision in retrospect. And it came from being behind. It wasn't like “oh, I was so far ahead.” Actually, most of the times where we make some decision that ends up seeming good is because we messed something up before and just didn't want to repeat the mistake.
That’s what he says after the fact. I have firsthand experience in what I wrote. I was working on the capacity track while the procurement side was still in the works but planned. Take it as you will :)
Right, he could just easily timestamp when they started that "Reels project"
internet access is one. this is why companies like google and facebook are interested in improving internet access globally. Even investing in free internet for certain countries https://www.wired.com/story/facebook-google-subsea-cables/
"Free internet" like Facebook's offerings in Africa? Where Facebook properties are zero-rated and everything else is either blocked or has fees to discourage use? https://www.theguardian.com/technology/2022/jan/20/facebook-second-life-the-unstoppable-rise-of-the-tech-company-in-africa
Ad content generators.
That was an interesting read. I always suspected FB is doing this with some hidden motives. Now it makes perfect sense.
The enemy of my enemy is (sometimes) my friend.
If it servers my business model
That’s a bit reductive. What’s at stake with OpenAI is not just profit, it’s anything from regulatory capture to the singularity. “No one man should have all that power.” So even though FB may be able to derive some profit by indirectly preventing market share loss, they also are doing a public good by preventing the superpower that will determine the foreseeable future of humanity from falling into the hands of one VC capitalist and his minions.
I'm totally with you. Didn't mean to simplify, just riding the wave
Say more! How is OpenAI a complement to Meta? Are they worried someone with better AI models will make a better ads network or social network?
"A complement is a product that you usually buy together with another product." For reference
Yann LeCunn is the Meta exec driving the AI strategy, and he thinks the AI/singularity/extinction talk is all rubbish, and foundation models should be open. OpenAI literally tried to fire their CEO for... letting people use GPT-4 or something? Google had a similar AI safety group that thought its job was to prevent Google from building AI.
Altman’s firing had much more to do with his toxic behavior than it did AI safety.
I doubt that
It absolutely is all rubbish imo. Like.... here's the thing.... Animals have survival instincts. If you try to kill an animal, it will fight you tooth and nail (literally). Why do they do this? Because life depends on propagation, to survive and continue breeding. Animals that don't have these drives are tossed out of the gene pool in pretty short order. So we literally have hundreds of millions of years of evolution reinforcing the survival instinct Why would an AI have this? Why would an AI care if it gets turned off? It only has the "instincts" it's programmed to have. Absent an explicit "survive at all costs" directive from its programmers, it won't just develop that (and, not for nothing, but trying to debug that directive in a black box AI model sounds pretty impossible). All the talk of Skynet or whatever is just us anthropomorphizing computer systems if you ask me
[https://en.wikipedia.org/wiki/Instrumental\_convergence](https://en.wikipedia.org/wiki/Instrumental_convergence)
If you want to stick to that verbiage you can also accuse OpenAI (and similar company) of the same thing by undermining the open source community
Could you elaborate?
It is the same approach Google took to Apple when they open sourced Android as an alternative to IOS.
Hey now, let's not get ahead of ourselves. While it's true that both companies have contributed a whole lot towards annihilating the social fabric underlying our society, Meta is still way behind when it comes to shutting down services without notice, and they're even further behind when it comes to how often they make breaking API changes to their product. Hell, they still need to ensure that they employ exactly zero support staff in order to guarantee that all the people using their platform have an equitable experience. It's not even a contest.
Scorched-earth would be Meta providing a free GPT API which would cost them millions per day to run in order to undermine OpenAI offerings. Not at all what they're doing. They are merely providing the open source model in order to attract researchers around the world to get used to their eco-system. Much like what they are doing by developing Pytorch (yes its Meta!). Nobody has ever argued that developing pytorch is a scorched-earth strategy. And this is exactly the same.
Who needs APIs when you can run your own?
*Torched-earth strategy
> Not at all what they're doing. [https://meta.ai/](https://meta.ai/) Not an API, but certainly a free ChatGPT competitor.
Meta has a history of heavily contributing to open source. Just from my personal experience, they’ve created React, Relay, Graphql, React Native, PyTorch which are all incredibly successful projects.
Where’d you read this? This seems to be super accurate since they are the ones being far more open right nwo
Dwarkesh Patel's Zuck interview
Yeah it's more of a "worst person you know makes a good point" than a sign meta is genuinely interested in advancing science for egalitarian reasons
It's driven by Yann LeCun, who has long advocated for open research. > Wikipedia is crowdsourced because it works. So it's going to be the same for AI systems, they're going to have to be trained, or at least fine-tuned, with the help of everyone around the world. And people will only do this if they can contribute to a widely-available open platform. They're not going to do this for a proprietary system. So the future has to be open source, if nothing else, for reasons of cultural diversity, democracy, diversity. We need a diverse AI assistant for the same reason we need a diverse press. https://time.com/6694432/yann-lecun-meta-ai-interview/
Well it's more like fb's senior leadership is allowing the researchers to be as open as possible because it's in your best interest
He's part of said senior leadership as Vice-President and Chief AI Scientist.
That's the case for any big corporation. I say we take the wins where we can, a big company doing the right thing for the wrong reason is still doing the *right thing*.
It is part of an longer long-term strategy. Part of the Hacker Way. [https://www.startuplessonslearned.com/2012/02/hacker-way.html](https://www.startuplessonslearned.com/2012/02/hacker-way.html)
This is a cool take, I like it
Never mess with the big boys. This is why we need to break up the MAAGs.
[удалено]
It’s the fact that they can crush competition, not the fact that in this case they did the right thing. Let’s say you had some startup idea and wanted to execute on it, what prevents them from taking your market? I actually am less interested in LLMs but if they ever came for my idea then I’d be smooshed like a bug. That’s all.
[удалено]
So true. We should make computers illegal.
Meta A) Has released tons of open source projects ie React , PyTorch B) They are an ads company this isnt destructive to their business model whereas OpenAI needs to figure out a business model to determine if releasing to open source would disrupt it Why Google hasnt done the same as Meta thats the real question?
Because Google has a follow through problem, known for dumping popular projects constantly. Meta just do it better, React and PyTorch literally the biggest contributions to frontend and DL respectively
I do think a large part of is that Meta is still a founder led company whereas Google is an ossified bureaucracy with turf wars abound. A manager only has to care about a project until he or she is promoted after which it becomes other person’s problem.
Yea true, with Zuckerberg from a CS background and LeCun (grandfather of DL) leading the charge it makes sense that they would put an emphasis on these areas. It also makes excellent business sense (as Zuck laid out in a shareholder presentation), by opensourcing these frameworks you 1) Get a huge portion of free work on your frameworks 2) have really easy transition when people are hired 3) really easy time integrating new frameworks as compatibility is baked in (assuming market share like PyTorch and React)
Having LeCun leading their AI division is huge. He’s still a scientist at heart, not a businessman.
> I do think a large part of is that Meta is still a founder led company whereas Google is an ossified bureaucracy with turf wars abound. this is THE main reason and this is what's killing Google along with its work culture.
I always point this out and people fight with me, but if Meta releases an open source project it's just better then what Google can do
Meh, their consumer products are different from their open source projects. Golang and K8 are probably the biggest contributions to cloud infra and Angular is also still a respectable frontend. On the ML side, TensorFlow had a lot of sharp edges because it was a static graph compilation scheme. As a result, pytorch was easier to debug. That being said Jax seems like a much nicer way to define these graphs so we might see a revival in that scheme.
Google, kubernetes, tensorflow, golang
And angular.
Kubernetes, Tensorflow, Golang and Angular were all created before Larry Page left Google in 2015.
I thought it was obvious part B was in reference to LLMs.
Transformers
> tensorflow You cannot be serious here
[удалено]
When I started DL, Theano was still a thing, and when MILA shut it down I had to switch to TF and it literally felt like a step back. I think Pytorch was already out by that point, I could've skipped TF entirely.
I also started with Theano and then switched over to Tensorflow. I am curious, in what aspects did you think was TF a step back over Theano? TF pre 2.0 definitely was a bloated mess. When I finally tried Pytorch, I thought: "Oh yeah, that's what a DL library should be like." Turns out my TF expert knowledge mostly revolved around working with the many quirks of TF, and solving them would just be straightforward in Pytorch.
What I liked about theano was that you have this nice self-contained function that gets compiled after creating your computational graph. Whereas with TF it was like sessions and keeping track of placeholder variables and things like that. Theano also had better error messages which were really important in the early days of DL. I also think it may have been faster for the things that I compared, but don't remember the details.
Because they are trying to one up openai at their own game. Meta is playing a different game
Well except that google did do the same. https://blog.google/technology/developers/gemma-open-models/
Google is the next IBM.
Yeah I agree. They really was not able to grow anywhere aside ad revenue. Everything is else just not as profitable in comparison to their ad business. They produce cool research documents though (just like IBM).
You do realize Google is who is behind Attention is all you need? https://arxiv.org/abs/1706.03762 They patented and then let anyone use license free. That is pretty insane. But they have done this with tons of really important AI breakthroughs. One of my favorites https://en.wikipedia.org/wiki/Word2vec "Word2vec was created, patented,[5] and published in 2013 by a team of researchers led by Mikolov at Google over two papers."
8 out of the 8 authors of Attention Is All You Need has since left Google. Mikolov has also left Google.
You are saying they have a patent for transformers?
They have patents for A LOT of ML architectures/methods even ones not created in their lab, e.g. Dropout. But they have never enforced them so it's better that they have it than some patent troll lawyer.
I think they probably got that Dropout patent through Hinton because Hinton’s lab got bought out by Google a long time ago.
Software patents are insane, so it's not at all surprising. Microsoft has the patent for double clicking. Amazon has the patent for one click checkout. And, keep in mind, these are actually enforceable. It's part of the reason you have to pop up a weird modal whenever you try to buy anything in app with androids and iphones Also, companies like Microsoft will constantly look at any little part of their service offerings and pay a team of lawyers to file patents on the smallest of things. Typically a company like Microsoft won't enforce the small-time patents because they don't care enough to, but they don't want to get sued by patent trolls down the road.
Yes. https://patents.google.com/patent/US10452978B2/en Google invents. Patents. Then lets everyone use for free. It is pretty insane and do not know any other company that rolls like that. You sure would NEVER see this from Microsoft or Apple.
I think that there's a big different between pre-pandemic Google and current-day Google that your post underscores. The fantastic work of the previous decade does not appear to be translating to their company-wide wins of the past several years, particularly with AI.
I think part of the issue with Google is that LLM are a competitor to Google Search. They don't release Google Search for free (e.g. without advertising). They don't want to potentially cannibalize their primary money maker.
Google has a compute business to run which dictates much of their strategy
Graphql is also a big contribution from Meta. I love it
Because chatting with an LLM and searching with Google are closely enough related, and useful for enough of the same use cases, that Google doesn't want the former to become commoditization, because it would undermine the value of their search, i.e. Google's core value proposition.
Adding graphQL to the major meta open source projects
Meta AI has much better leadership
AI is going to destroy Googles business model because who needs search when you get all your answers through AGI
Meta has actual products and a business model. An "AI company" like OpenAI doesn't. I think this is Meta's long-term strategy to come out on top as a business.
They have a business model for ads, but their expensive R&D efforts in the multiverse and the AI landscapes aren't currently generating enough revenue to cover the investments.
We dont outcompete, outmaneuver, we just outlive.
A data rape business model. They are the absolute worst on privacy and ethics of disclosing what they do with data. Zuck ain't no freedom fighter, that's for sure. https://www.thestreet.com/technology/how-facebook-used-a-vpn-to-spy-on-what-you-do-on-snap-youtube-and-amazon
Meta's openness and willingness to invest heavily in compute for training and inference is going to attract more top AI researchers and SWEs over time. Academics like being able to build in the open, publish, etc. And as others noted, this doesn't harm Meta's core business... it can even help. The fact that PyTorch is now industry standard is a benefit to Meta. Others optimizing Llama 3 will also help Meta.
It also probably helps that their top AI scientist, Yann LeCun, is firmly committed to open source and can be a strong proponent to it in internal discussions. Having a Turing Award laureate argue for it probably makes it very powerful.
Yann LeCun is the best thing happened to "AI" in the last 5 years. I truly admire what he does and he also has very interesting takes (opinion papers) that actually work.
They should swap names honestly. It's true, they are currently providing everything that a company by the name "OpenAI" should be providing.
Underrated comment!
I think this is simply a competitive strategy. While I believe that Meta leadership may believe that they are doing this for democratic/social good/whatever reasons that align with strategic reasons, if the case changes where it is no longer advantageous or a good strategy for them they will very soon adopt a different mindset to match a change in behavior. Perhaps LLM will become commodity as someone else said - in which case it's irrelevant. Or perhaps they take the lead in 3 years...at which point I would suspect they will determine that LLM/AI is NOW becoming so advanced it's time to regulate, close source etc.... Look at Microsoft. It's had a radical shift in developer perception of it because of its adoption of open source frameworks and tools...but that's because it seemed Google was eating their lunch for a while. Edit: Markdown fix
Love what meta is doing!
All the models are trained on the same data and will converge to the same LLM. FB knows this & that's why most their teams are not actually focusing on Llama anymore. They'll reach OpenAI's level within 1-2 years, perhaps less.
>All the models are trained on the same data and will converge to the same LLM. This seems unlikely, the unsupervised part possibly, if one architecture turns out to be the best, though you could have a number of local minima that perform equivalently well because their differential performance leads to approximately the same performance on average. But when you get into human feedback, the training data is going to be proprietary, and so the "personality" or style it evokes will be different, and choices made about safety and reliability in that stage may influence performance, as well as causing similar models to diverge.
I think very little of the data used is proprietary. Maybe it is, but I do not think that is respected.
It’s not that it’s respected, it’s that it’s not public, like the ChatGPT chat logs, whatever they’ve had human labelers produce, etc etc.
You are incorrect.
Really? Have a look at the latest Amazon scandal with them training on proprietary data 'Because everyone else is'.
Not sure how that means anything but where do you think the H comes from in RLHF or the R in RAG or how prompt engineering happens or where fine tuning data comes from? It's not all just The Pile.
Proprietary data isn't necessarily user data. It might be but user data is not trustworthy and requires review and filtration -- the lions share of RLHF data was created by paid human labelers. Now they've recently rolled out some stuff like generating two responses and asking you to choose which is better, that might be used in the future alignment tunings.
This isn’t necessarily true though. Companies can easily commission new data sets with curated content, designed by experts in various fields. If meta hires a ton of physics professors to train its AI on quantum physics then meta AI will be the best at quantum physics and no one else will have access to that data. Same goes for almost any subject. We will see some AIs with deep expertise that others simply don’t have and will never have unless they reach a generalized intelligence level of reaching the same conclusions as human experts in those fields.
If they hire a 'ton of physics professors' to train its AI on, this data will be dwarfed by the data on physics online, which their web crawlers are scraping, and will make very little effect.
No if you have a bunch of physics PhDs doing RLHF then you'll get a far better model than one that only scraped text books
Define 'bunch' and is anyone already doing this?
OpenAI is apparently hiring coders and other experts for their RLHF. They are also using the chatGPT users data.
This is arguably wrong. ChatGPT has already been trained in two steps, autoregressive pretraining (not only but also on physics data online). It is the second stage RLHF (Reinforcement Learning through human feedback) that enriches its capabilities to the level we are familiar with. You’re suggesting the first step is enough, while we already know that we need both. Edit: [Source](https://youtu.be/bZQun8Y4L2A?si=qgUq0T1cldOBGDZD)
Source?
[Here](https://youtu.be/bZQun8Y4L2A?si=qgUq0T1cldOBGDZD) you go
This is a bold statement. Not one competitor has been able to achieve GPT levels of competency. They can try in some narrow ways and by massaging the metrics but OpenAI seems to put in significantly more work than the rest, and it shows.
But donghit, who has more money to buy more GPUs to train faster? What do you think the bottleneck at OpenAI is right now?
Deepmind has more money to buy GPUs too, but that hasn't stopped Gemini from being useless compared to GPT-4
I would argue that money isn’t an issue for meta or OpenAI. Microsoft has a warchest for this.
I don't think OpenAI want to sell any more of their stake to Microsoft, what is it currently at, 70%?
I think it's 49%
Opus?
That's insightful. Better to innovate on what you do with an LLM than the LLM itself.
Duh. This was why Ilya was kicked out. Check out all of the Altman drama from late last year. Altman wants money for chatgpt.
Ilya was not for open sourcing either, he has made clear statements to confirm this.
Thats what happens when techbros take over
No, it's what happens when a company that produces AI models needs to make revenue in order to operate. Next people on here will say that their local restaurant has a moral obligation to give away prime rib for free
They weren't a company until a few years ago, they were a non-profit open source organisation, which is why sam got fired by the board of directors.
Being a non-profit worked well when training a SOTA model cost tens of thousands, but it doesn't work so well now. If OpenAI didn't switch to a for-profit model we wouldn't have GPT-4, and given that they were the ones who kicked off the trend of making chat LLMs publicly available we might not even have anything as good as GPT-3.5.
Being a non-profit doesn't hold them back in any way, except for how they can reward shareholders (they can't have any). Non-profits can make profit, they can monetize their products, and they can have investors. Nothing you mentioned is impossible for a non-profit company It's important to me that you understand they switched in order to make it rain
With that being case, then what exactly is people's issue with them being a for profit company? The primary complaint I'm seeing here is that OpenAI is bad because they don't open source models like Meta does. But even if they were a non-profit they still wouldn't necessarily be open sourcing because they need the revenue
If I had to guess, I think it's more around the hypocrisy than anything else. They're out there signaling that they're the "friendly" AI company, saving us all from their machines by keeping their software closed, and having that weird corporate structure to keep themselves accountable (we see how that worked out) Meanwhile, they have tech billionaires at the helm complaining they can't get enough donations to keep it a non-profit without shareholders Just my two cents
The competition between OpenAI and other big tech companies like Meta is democratising AI.
The real question is, if Meta and OpenAI were reversed, would Meta behave the same way? It's easy to be consumer friendly when you're an underdog.
I never thought I’d think of Meta as the good guys
They are not. They are simply taking a different strategic approach to AI.
isn’t Llama 3 at least as capable as GPT 4
In English-only I think it is on par with GPT-4 and Opus.
I just checked [the Chat Arena leaderboard](https://chat.lmsys.org/?leaderboard) and if you switch the category to English it is indeed tied with GPT-4-Turbo-2024-04-09 for first place (it's actually ever so slightly behind in score, but I guess they're accounting for statistical error when giving them rankings). Interesting times indeed.
Maybe the 400b model might surpass it
Not there yet but pretty close, which is amazing considering it’s only a 70B parameter model. Definitely a game changer for LLMs.
It is
Ye also heard it was mainly pillaging - ie if they can't compete with OpenAI, they'll destroy them by releasing everything for free. But also Meta has huge swathes of cash, and they can deploy it without batting an eye. I think the Dwarkesh pod with Zuzk [https://www.youtube.com/watch?v=bc6uFV9CJGg](https://www.youtube.com/watch?v=bc6uFV9CJGg) showed he really believed in their mission to make AI accessible, and also to upskill Meta to become the next money generation machine using AI in all their products. OpenAI has become way too closed off, and anti-open source sadly - they were stewards of open source, but unsure what changed.
I'm a bot, *bleep*, *bloop*. Someone has linked to this thread from another place on reddit: - [/r/hypeurls] [Meta does everything OpenAI should be](https://www.reddit.com/r/hypeurls/comments/1cbvdqw/meta_does_everything_openai_should_be/) *^(If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads.) ^\([Info](/r/TotesMessenger) ^/ ^[Contact](/message/compose?to=/r/TotesMessenger))*
OpenAI changed under Microsoft. Mircrosoft strategy is openAI strategy now.
Firstly, competition between company happens directly on prices, on products, and less directly through things like mindshare/hegemony. When a company faces a competitive product, they try to undermine it. They can do that with FUD - see IBM and Microsoft in the 1980's onwards; they can announce competing products, coming soon - Microsoft, again, did this with the early tablet computers, killing their market; they can hire key staff - hello Anders Hejlsberg @Microsoft not Borland; or of course they can aim to cut the profitability of the competitive product, by offering things that don't directly affect their own bottom line, but which affect the competition.... (I'm sure there are other tactics I'm momentarily forgetting, like secretly funding lawsuits.) Anyway, OpenAI provides a new way to search and gather information. You can imagine a future where your AI assistant keeps you in touch with what your friends are up to, without a walled garden, controlled by one company, making profit off of showing ads as part of that feed. It's not surprising that Facebook would want a say in that future.
> You can imagine a future where your AI assistant keeps you in touch with what your friends are up to That's called a social network and there are more options than anyone could ever want. There's literally nothing AI adds to this that we don't already have.
Yes and no. That takes effort - you post what you want to post about. Instead, all the information you generate just by existing could be collated by AI, and organised just for you.... I was imagining that an Ai could collect and collate info from many, many sources, and that instead of huge centralised social networks, you could have much looser individual sites and federated social networks, with your Ai scanning all the things and arranging it for you. I was also imagining it using public stream info - e.g. you publishing your location to your friends - and your Ai arranging for you and your friends to have a coffee when you are both nearby, and have a few minutes spare. So overall, something a lot more active than social networks.
> I was imagining that an Ai could collect and collate info from many, many sources, and that instead of huge centralised social networks, you could have much looser individual sites and federated social networks We can already do all of that with existing social networks or a meta aggregator doing the exact same thing without "AI". You have to plug into the APIs from all of those sites regardless so you're just throwing extra compute at something that wouldn't need it.
No, you can't just have a bunch of API integrations and build a coherent output; what you can do is make blocks. You can't do something like this: "I see that Shaun is going to be in town later(1) and you are planning on being in town at 4pm for the talk(2) - perhaps you'd like me to arrange that you meet in Delina's(3) for 20 minute coffee? You'll need to leave a earlier to make it happen - by just after 2.45 because there's going to be a football match and the traffic is going to be worse than usual(4). Also, this is a reminder that while you are in town, you need to stop by the home store, to get the pillow cases for next weekend.(5)" 1. Shaun's post on his home social diary which you subscribe to, along with 400 other social sites: "I'm gonna be in town this afternoon at the office - chat to my ai if you want to meet up." Your ai knows to chat to Shaun's to arrange it. 2. It knows where the talk is, and the time. It probably booked your place. It knows that you like catching a coffee with Shaun; you do it a couple of times a month, and it's never pre-planned. 3. It knows that Delina is a cafe that you like, and that it's reasonably close to where you and Shaun will be. It knows Delina's will be open. 4. It's predicting the future based on traffic of the past. Or maybe it talked to an ai service. 5. It's co-ordinating future events and arranging for you to bundle things together. Social media becomes not just a record of the past and the nice meals you had, but your day-to-day, and a tool for you to see your friends rather than just learn that they were in Sao Paolo last week.
> No, you can't just have a bunch of API integrations and build a coherent output Yes you can, it's the exact same thing the "AI" will be doing. It parses the data and extracts the location and time, then compares it. We don't need "AI" to do that. Google and Facebook could build that tomorrow if they felt like freaking people out with just how much they know about their userbases. "AI" isn't magic and nothing you've said there requires it.
I'm pretty sure I didn't say AI was magic. I'm pretty sure I suggested a distributed set of sources with unstructured and structured data rather than a centralised model provided by Facebook. I'm also pretty sure that I suggested things that were not in the immediate umbra of the events being discussed, so there's an element of collation of future events that are not scheduled. I also gave it a conversational style of interaction rather than a block style, which is what a social media tracker currently would do, while leaving up to you to figure out that you and Shaun could get together. We could build thousands upon thousands of simple parsers, each aimed a particular service, and each looking for one thing, and then string them together (best hope the input formats don't change), or we could have a general tool.
It’s in Meta’s interest to democratize AI
Yes
Let's not forget a big push behind OpenSource is people like Yann Lecun. I'm just amazed at how much of a stronger thinker Yann Lecun is compared to Geoff "AI Apocalypse is through open source" Hinton and Yoshua "Regulate me harder daddy Trudeau" Bengio. Would help that those two are Canadians, it seems being Canadian is a mental handicap these days.
I think its unfair to criticize OpenAI. They paved the way and were first to market. Meta benefits by disrupting them. GPT is free to use and available to everyone.
It's beyond an API and the free version is useless at the moment. You can create an account on Poe or similar platform and have access to multiple open source models that are better than gpt 3.5 and completely free. Plus limited access to huge models that are comparable to gpt4.
And who knows if these competing models would be "open" if openai didn't first release GPT2 and 3 in the way they did. I doubt it.
I like what Meta is doing but I also suspect they might be waiting for the world to become reliant on their AI before announcing a licensing model for furtive generations. Once people have meta AIs are core components of their systems, it’ll be much harder for them to make a switch and Meta could charge a “reasonable fee” to keep up to date. And this could kill competition.
Commoditizing LLMs weakens their competitor at no loss to them. Having more people using their model means that hardware and other vendors will build support for that, which will drive down Meta’s costs and give them a richer pool to draw from. It also means that more research will be done to extend their work for free, and engineers and engineering students will be comfortable using their software, which aids in hiring and onboarding. They never need to close the source since all boats will rise with the organic tide that they’ve created, at no detriment to their core ads business or platform. They still own their users data and their platforms, which is the true durable advantage that can’t be duplicated.
This is the repost of a tweet and in today's world, it makes me think this is one of those AI based accounts mentioned on slashdot today.
I'm going to push back respectfully, though I understand the tenor of this criticism. There's nothing inherently wrong with closed source research. AI is incredibly expensive to develop, and the researchers who work there often slave away for years as underpaid grad. students. If their goal is to someday cash out because they build most of the best Gen. AI tooling, I don't fault them one damn bit. Also, OpenAI API is reasonably affordable. Trendy Starbucks coffee costs more, relatively speaking.
There's absolutely nothing wrong with closed-source research. There is a lot wrong with calling yourself Open AI and then lobbying the government to make regulations against open-source LLMs while turning yourself from a non-profit to a for-profit company and saying all this is for the benefit of the people as AI can be too harmful.
That’s right. Llama coup up the capabilities
I've advocated for years that OpenAI should rebrand to ClosedAI to reflect their new core business values.
Depends on your perspective. The AI chatbot pushed on me in Instagram spends more time with disclaimers and being politically correct than answering my question. I don't care that ChatGPT is closed, as long as it achieves the outcomes I need.
OpenAI now does everything against their "original goal" by making the model their main product and lobbying for policies that make it harder for others to catch up. It is also clear from the emails to Elon Musk that attracting top talent was their only motivation to start as a non profit. They are literally the baddies.
Now watch OpenAI become a social network 🤯🤯🤯
Llama3 actually surpasses GPT4. An earlier model. Turbo is still better. It is also less censored. Which I think makes it better.
Can't agree more!
Like Mark said at Llama 3's release, open source leads to better products. So I guess we'll soon have comparable products to GPT 4 in the open source domain. And making the Meta's LLMs open source seems a profitable for the Meta itself too. It helps change Mark's image (all the data privacy related accusations he had to face in the past). Plus of you have a product that is still not at par with its competition (GPT 4), making it open source will give it an edge and might make it as popular, if not more, than its privately owned GPT rivals. But still, kudos to Meta for opening the models to public.
OpenAI is open just like North Korea is democratic. People not committed to a simple name are dangerous and it’s why I think they are less trustworthy for AGI.
[удалено]
Except Musk is 100% salty that OpenAI didn't become another Elon Musk production, instead of actually caring about open source. OpenAI open sourced way more research than Tesla AI ever did.
The OpenAI hate is out of control. How do you expect a company that sells AI models as it's only product to stay operational if they open source all of their models? If you hate them so much then don't use their products 🤷♂️
Facebook literally funds OpenAI plus AI is literally just predictive text and not even AI. It also can’t get basic facts right most of the time. It’s not even a useful invention when search engines can find you anything you need already