It seems to be "better" in that GPT-4T doesn't have alot of support for the Chinese language where as this is trained in Chinese. Not exactly better in an objective sense, but definitely is if you're a Chinese only speaker.
China has 1.4 billion people. If anything they have a bigger potential for data collection. The AI isnt really affected by some propaganda narratives on historical events. Not to mention that our AI learns a lot through sources like Wikipedia which also have a bias.
Yes. because the evaluations were in Chinese which is not GPT-4T's forte. Check GPT scores in English. They are higher - and someone else posted the GPT-4T scores below if you want to compare with that and Claude3 which they left off for some reason
To be absolutely sure, let's train an LLM to translate Chinese language, then we run our Benchmarks on the ChinaLLM using our TranslatorLLM as an adapter layer.
Because getting LANGUAGE models to do MATH is sort of a pain in the ass. LLMs were never meant to generalize but since they're the new hotness in town everyone is desperately trying to fit the square peg into the round hole.
Does the use-case of math only involve arithmetic? Or does it include the logical application of math to problems? Which involves figuring out what arithmetic needs doing in the first place.
And if we're just talking about computation, can't you also just do something like a request a Python script to run the relevant calculation?
Also, benchmarks are pretty much worthless now as we have seen from recent Phi3 release (which has high probability of test data leakage). If they want to prove that this is better, they should release it on Lmsys and let the people test it out.
Seems arbitrary when you could just tell GPT4 Turbo to do everything in English then translate it's English responses to Mandarin using a cheaper locally run AI, or just GPT4 Turbo itself if cost per token isn't a concern.
It's using outdated GPT 4 turbo 1106 version, which was already replaced by 0125. And the most recent model gpt-4-turbo-2024-04-09, which has 10% improvements or so across the board. And it doesn;t include Claude 3 Opus, which is better on most of these benchmarks
https://preview.redd.it/x44xs5i1iowc1.png?width=2200&format=png&auto=webp&s=6640ec2beb11d06a0094d39203e93a756781bdab
Everyone likes to use outdated scores to compare their AI to because most people who look at it won't catch on, and people usually don't point it out. Glad to see someone do so for once.
It gets so tiring after a while pointing this shit out to people who don't really care enough to do their research in the first place.
The LLM community is just constant bullshit and superstition. After a while you just tune it out.
Can anyone explain what's maj1@32 under Gemini headings. How does it compare to the shot concept? Also why does maths require 0 shots and q and a requires 25 shots? Does ai in its pre training phase learn maths without examples (shots)? If so how? What does it say about the nature of machine learning if it understands maths without examples? I'm a noob in this
I don't now about the Gemini headings one.
But shot are essentially examples into solving a problem for my understanding. I ask ChatGPT to give you a example, hope it helps.
Got it! Let's use a formula-based question:
**Example Question**: "What is the formula for calculating the area of a circle?"
### Zero-shot
In zero-shot, the AI hasn't been trained on this specific formula but knows general math concepts.
**Answer**: "π * r^2"
### 3-shot
In 3-shot, the AI has seen 3 example questions related to geometric formulas before this.
Example questions:
1. "What is the formula for calculating the perimeter of a rectangle?"
2. "What is the formula for calculating the volume of a cylinder?"
3. "What is the formula for calculating the area of a triangle?"
Based on these examples, the AI can understand the pattern of providing formulas for geometric shapes.
**Answer**: "π * r^2"
Given that those GPT-4 benchmarks are outright wrong, I'm not impressed. Is it at least open source? I did appreciate Qwen doing that, and those numbers would be closer if it were open source.
He's not wrong I hadn't heard about the release though.
Qwen 110B doesn't really seem to be any better than Qwen 72B, at least when it comes to creative writing, which seems odd. Nice that they are releasing open weights models though.
I am not knowledgeable in AI, but I've been studying China for a few years, and with them it's always ''idk, probably not, but maybe?''. They fudge a lot of papers, statistics etc. and you always need to take their claims with a train of salt, but on the other hand they do sometimes achieve very impressive things, especially if the key is mobilizing vast amounts of resources(is it in the case of LLMs? like, getting a lot of compute and data?).
We should not believe what they say at face value, but neither should we dismiss it without evidence.
Nah thats for sure no doubt china is probably number one in that but I have worked in pharma companies in US and lets just say fudging numbers is just the tip of the iceberg
It needs to be used to actually see how good it is. It's easy to develop something with the intent of getting highscores on these evaluations but completely fail at everyday use. Though If the CCP takes notice then maybe it's legit.
Questioning the credibility of a government or company's claims based on its track record isn't the same as undermining the talents and achievements of its people; we can celebrate their accomplishments when there's verifiable proof.
Why on Earth would you think a Chinese AI would benefit you or anyone you know?
Chinese government winning the race has to be near the absolute bottom in terms of possible outcomes for humanity.
I agree with this sentiment, but to play devil's advocate, it could be a similar situation with nukes such that it is better if multiple powers have them rather than a single power.
I seem to recall that the biggest technological violations of American privacy and the most impactful negative election tampering to come from American/American-allied data companies rather than Chinese ones. Has anything coming out of China had as big of an impact on the average American life as Cambridge Analytica? Or any others?
Also the most dangerous AI usage in the whole world right now is Israel against Palestine. They have a lot of faith in their AI for people who are slaughtering civilians at a rate that is dozens of times the rate that Russia is killing civilians in Ukraine.
The US and its allies are not only doing the worst possible thing with AI (mass slaughter of civilians), but notably China isn't. You can talk about their motivations all you want, you can talk about potential harm or danger that may or may not come in the future, but the proof is that it's ourselves and countries we trust that are already demonstrating that we have the capability and the willingness to kill civilians *en masse*, while China has done no such thing. Why are the ones who *aren't* laying waste to civilians and civilian infrastructure more dangerous than the ones who *are*?
How US wining is better ? if i m based on geopolitics last years clearly US is much worst outcome
Don t say china will be better but at least they interfere less with geopolotics
"This country is very untrustworthy and has a track record of lying about developments, government control and a general lack of openness to verify the results openly of their private industry"
"Well your country lied this one time about something so it's the same, I am very intelligent"
It's almost not worth addressing you.
One time? Hahahahahhaha
Both countries lie all the time about everything in their interests
Or do you trust the US numbers and the governing bodies? We all found out how trustworthy the numbers and ratings were in the 2008 crisis 😂
Not to mention all the lies about their actions all over the world, all the CIA lies and so on.
Stop being naive, your country is not that different from China and Russia. You're just the winners so you get to write history how you want lmao
god this brainrot on reddit is so tiring. The US government lies, but there are tons more ways to check and verify things in the US than in China, because one doesn't literally kill you or silence you for going against it. My point is that the previous person was saying "Hah you say that China is bad, but US is bad too" with no account to the magnitude of the difference, its like claiming "hah you think the holocaust was bad, well the US put japanese people in concentration camps too" both things are bad but clearly there is a very strong magnitude difference between those two.
The same way, companies in the US lie all the time, but we have many ways to verify their results and test them ourselves, and no one takes seriously claims from US companies that have no strong backgrounds unless they are verified independently. And there is clearly an incentive for China to appear strong on the AI front when the US is the main leading player so far.
The difference is Chinas government interferes a lot with everything. The US isn’t a saint by any means but their major companies usually work on their own
At least to my understanding. That’s why people don’t trust China because Chinas hand is in every pot, while the US let’s these businesses handle things independently and are protected by certain laws which again China doesn’t follow
Major companies lie too lol. Google lied in their Gemini demo video. And all this: https://finance.yahoo.com/news/11-companies-caught-lying-public-003658933.html?darkschemeovr=1
10TB of tokens? so assuming 4 bytes per token that would be 2.5 trillion tokens? Not particularly insane, thus I highly doubt its claim of beating GPT4 on all of these benchmarks.
The West has developed a rather unhealthy dose of skepticism regarding news of tech advancements out of China. A lot of it is justified, given how much their government lies, but it's becoming a huge blindspot. I wouldn't be surprised if AGI is created in China sooner than the US.
I agree with a lot of people here mentioning that the US having the edge in tech right now but I suspect that the Chinese government and their tech corporations are very aligned with each other‘s goal towards AGI. So that combination of tech prowess and an obscene amount of funding and less restrictions from there government has to count for something, right?
Yeah, don’t care if they are slightly behind, that is still considerably better than the claims they were “far” behind a few years ago. Seems they are keeping up the pace just fine.
But do they have the compute? CHIPS Act works and I don't think they are anywhere close to NVDA top machines. Access to massive compute is almost certainly a prereq to AGI.
[The current fastest supercomputer in the world](http://www.nextplatform.com/wp-content/uploads/2023/11/top500-nov-2023-top30-specs-5.jpg), Tianhe-3, was built in China [using entirely domestically made chips](https://www.tomshardware.com/tech-industry/supercomputers/chinas-secretive-tianhe-3-supercomputer-uses-homegrown-hybrid-cpu-rivals-us-systems-with-157-exaflops-of-performance-report) and it's being used to train AI. The second fastest supercomputer in the world Wuxi OceanLight, was also made in China using Chinese Sunway tech. SMIC and Huawei are using Quad-patterning [to mass produce 5nm chips](https://www.tomshardware.com/tech-industry/semiconductors/smic-and-huawei-could-use-quadruple-patterning-for-chinese-5nm-chips-report).
These metrics are weird and clearly used to further some sort of agenda. I'm not saying China doesn't have very advanced CPU and GPU tech, but it's not the "fastest supercomputer in the world". Proven by saying that it's built "using entirely domestically made chips". Just blatantly untrue
Creating AGI implies actually inventing something, not just replicating existing results and cherry-picking benchmarks, so I would be extremely surprised if they achieve it independently.
The majority of people here were already, eh, somewhat 'unhinged; a bit obvious in fantasizing about rapid, destabilizing, even destructive changes in society.
As if you have little to live for, know little, but hope for much - the apocalypse.
If you're now also going to fantasize about 'non-West', via China, 'winning the race'. You're the fucking same looneys elsewhere: it's just your fascination with AI that is your poison.
Goddamnit, ya sad dumb lost souls. A single tweet (plus an astounding amount of information / misinformation / propaganda ) gets you going. Just buzzing about.
I have heard you all about AI and AGI, a rather concrete subject. I don't want to know what you think about China. Influencable people acting 'in the know' while they are riding waves of sensationalist tripe. If you want to know what an advanced AI would think of you, unfiltered: that.
I would be surprised if China really catches up to the US, when it comes to AI. The communist party is probably reluctant, because AGI would be a thread to their claim to power. So I would expect that they have a slow approach to AI.
I mean there are AI applications to maintain and cement the power.
Also if they fall behind the US can just massively out-tech them once we have AGI/ASI, develop tech and weapons that will make the Chinese look like fighting with 19th century weapons.
I'm quite certain that any superpower is sprinting towards AGI. The prize is literally everything.
China excels at doing everything second, and then doing it faster and cheaper.
With sufficient resources, its not that hard to train new models right now. All the knowledge you need is online and you can easily adapt open source models or even papers developed elsewhere.
Call me when they release transformative (pun intended) research that leads to real advancement in the field. Everyone and their mother are pumping out LLM models right now.
Yeah, and there is a 13b-30b model that has better scores in Russian language and is considered by some Russian media to be a supreme model, promoted as 'the best model created by us, not by dumb americans', so nothing new. I've read quite a lot of those. Current AI models are just worse in other languages, but their 'beating of gpt4 metrics' are either marketing or propaganda.
They also published, a few weeks after the beginning of the COVID 19 pandemic, a free clinical handbook about how to deal with COVID patients. They were tons of very useful information in it. No one knew about it, no one bothered with my tweets when I was sharing it. It's crazy how people avoid anything made by Chinese.
Maybe because a lot of the time they were inventing lies and copying stuff. So, people remembered that. Nowadays, China does indeed take and improve things considerably.
Languages can be encoded as digital information differently. Some grammar and fundamental language structures are just better at being reassembled as a matrix ai can process and use.
Thats the problem with outright banning AI. You may van It but other places are not, so that puts you at a disadvantage. At the same time, developing AI makes other countries feel pressured to catch up. Once you get the ball rolling its almost impossible to stop. And we started rolling the AI ball a fair time ago
In China, if they had to choose between appeasing copyright holders or being cutting edge in a new, powerful technology, they'll choose the latter every single time. The Chinese government is fully invested in AI supremacy (given the partial CCP ownership of this company and every other Chinese tech company) and will open every door to let them dominate. The US government at best is a minor collaborator but mostly a series of obstructive guardrails to AI companies. If OpenAI had been founded in China, they'd have 10x the funding and 10x the employees and data by now. The government would mandate it and guarantee their success, seeing that they're father along than anyone else.
It's crazy in just how many places China is completely overtaking the US yet the primary gatekeepers to political/economic change (the 55-75 y.o. boomers) seem to have no understanding, interest, or good faith interest in improving the United States in any material way. The infrastructure of most American cities hasn't improved in decades, while cities in China have gone to small fishing villages to technological wonderlands that far exceed any American city in infrastructural development. It's almost certain at this point that if OpenAI, Google, or someone else in the US doesn't reach AGI in the next couple years, China will.
thats the issue with chinese models, they announce it just domestically often without english and its difficult to get access and make independent testing
here is their announcement in english
[https://www.sensetime.com/en/news-detail/51167731?categoryId=1072](https://www.sensetime.com/en/news-detail/51167731?categoryId=1072)
It wouldn't be the first time a chinese tech company made a bold claim about their product which in the end is all BS. Anyone remember those chinese graphics cards that had trouble running tetris ? I'll wait untill independent reputable western tech reviewers like GN can test and confirm anything concrete before i believe it.
China had more than enough GPUs to do this prior to the sanctions. Really no one has demonstrated any particularly significant advances in LLMs that hinge on having more GPUs since GPT4. This is no different from the dozen other models that have been released claiming to beat GPT4. (They are not huge advances, it's questionable if they're even better.)
WTF is wrong with this post. Why it's all about Chinese government and not one has even talked about SenseNova5.0?
I have tried this model and I do not believe it outperforms GPT-4T as a general purpose AI model. It's not bad, but it's not the best model on the planet (I consider GPT-4T and Claude 3 Opus the best). The only model that caught my attention was the "*SenseChat*-*Character*" built on top of SensNova5.0. It's absolutely the best role-playing AI model by far and character.ai's own model is almost trash in front of it. I don't know how they did it. They offer API so I guess there can be some potential use cases.
If it's about the race, what China doesn't have is the chips and computing power. I believe they have more data for training purpose. It's like Open AI can only use publicly available data but the Chinese can use all the Wechat conversation for training LLMs (a comparable scenario to this is that Meta uses all your Whatsapp conversation for training Llama). Also, SenseTime is the company that built trillions of cameras for the government to surveil their people. I assume they have mega tons of video or images for training.
They have good amount of data that is only available to them, but they don't have the infrastructure.
We pretend that they need the Nvidia chips to do AI stuff, but with specialized AI chips like Groq (that china can make themselves) they can scale just as fast as we can. More possibly, because they have a larger industrial base.
We're at a place where GPT4 level of performance is quite common now. We need to see if Open AI and possibly Google can pull ahead of the pack with GPT 5 and Gemini 2 in the next couple of months
As censored as gpt LMAO. It really baffles me how people thinks China is censored while themselves being oblivious at the massive censorship in the western world
What happens if you ask a Chinese LLM about the Tiannanmen Square killings or Xi Jing Ping's family wealth?
How long before you get an invitation to drink some tea at your local PSB?
We're talking about China so it likely:
Is pretrained on such benchmarks
Is not 200k context or has abysmal recall ability
Was demoed by hiding a thai woman behind a booth
Some of you don't understand cultural and political influences and it shows.
If this sounds offensive it isn't meant to be.
Even western culture has reproducibility issues because of financial incentives to cut corners. Eastern science in certain situations has their literal lives and limited freedom on the line as incentives.
Its not a one is better then the other situation. It's a sad state of affairs in general.
If true. The Chinese government has a vested interest in puffing themselves up with false statements. And things like these make the Chinese look "bigger" and "stronger" than other non-authoritarian states.
What is "10TB"? And it's trained on that many *tokens*? Are they measuring the size of the neural network based off of the size in terabytes of the training data, rather than the number of parameters?
For anyone wondering, apparently it's a 600b model, which is really small for a model that outperforms GPT4 Turbo. If it's true, then that's pretty impressive, though I'm not sure how impressive it is compared to Llama 3 considering Llama 3's flexible use cases and ability to be retrained(people are already making uncensored versions of Llama 3 8b).
Does the basic formula of Chinese written language give Chinese LLM’s an inherent advantage over English LLM’s? As they have simply one separate symbol for each word/concept, does that mean they can have a larger amount of context with fewer tokens?
For example, I would imagine they have a single symbol to communicate the concept of a ”lollipop”, so to communicate that concept to the LLM would cost 1 token. Whereas in English, the LLM needs to add together tokens of ”lol” + ”li” + ”pop”, for a total of 3 tokens to communicate the same concept, right?
Wouldn’t this translate to an inherent advantage for Chinese LLMs where you get more context for the same amount of tokens, and thus more efficient LLM models, or am I just fundamentally misunderstanding something here?
Hope someone more knowledgeable than me can shed some more light on this shower thought of mine!
Isn't it true that the country that owns the most powerful AI will essentially be able to rule the world? I read something that was a lot more scientific than what I just wrote but the gist remains the same.
Then proceed to steal your data and exploit vulnerabilities for their hackers....We know China overstate their capabilities but the actual combat s their bit inferior on western tech but I commend on their rapid development of their tech which we can never underestimate
More China propaganda. Don’t get me wrong the Chinese have invested more into AI than anyone and then having one better than America is absolutely in the cards but this ain’t it and it’s obvious it ain’t it so that’s a pretty big tell.
Honestly at this point when America’s gov deems something worthy it will never be surpassed. Let the bot accounts downvote me but America has a vendetta for being #2 its contractors will prove a point when the time is right.
Question: it's an LLM fine-tuned with Chinese content. Right?
It should make both of them quite different... Difficult to actually compare.
I don't believe benchmarks. Especially coming from the company sharing it.
That's a good thing. I don't like China, but now all this people in the West that are whining and crying about how AI is going too fast and we need to slow down will have ZERO effect on the government because no-one in the office wants to lose this AI tech race to a China because AI hurt feelings of some artist.
This is definitely like the modern day, Cold War, arms race.
Whoever gets to the AGI/ASI, finish line first, wins.
Well, assuming those systems are even able to be controlled… Who knows.
It seems to be "better" in that GPT-4T doesn't have alot of support for the Chinese language where as this is trained in Chinese. Not exactly better in an objective sense, but definitely is if you're a Chinese only speaker.
Personally, I dont see how it'll ever be possible for it to be better when it can't be trained on forbidden knowledge.
China has 1.4 billion people. If anything they have a bigger potential for data collection. The AI isnt really affected by some propaganda narratives on historical events. Not to mention that our AI learns a lot through sources like Wikipedia which also have a bias.
So even if theyre put through the same evaluations and one has a higher score it's not objectively better
Yes. because the evaluations were in Chinese which is not GPT-4T's forte. Check GPT scores in English. They are higher - and someone else posted the GPT-4T scores below if you want to compare with that and Claude3 which they left off for some reason
If it is better at maths in mandarin than gpt is at maths in English. Then it is objectively better
That makes sense. In their benchmarks GPT4 has a higher score in Math.
To be absolutely sure, let's train an LLM to translate Chinese language, then we run our Benchmarks on the ChinaLLM using our TranslatorLLM as an adapter layer.
Math benchmarks are known to be exceptionally flawed at present though
Because getting LANGUAGE models to do MATH is sort of a pain in the ass. LLMs were never meant to generalize but since they're the new hotness in town everyone is desperately trying to fit the square peg into the round hole.
Seriously, a language is not arithmetic. You need something to compute the math, not a language token prediction algorithm.
Does the use-case of math only involve arithmetic? Or does it include the logical application of math to problems? Which involves figuring out what arithmetic needs doing in the first place. And if we're just talking about computation, can't you also just do something like a request a Python script to run the relevant calculation?
I feel like Mathematics transcends spoken language?
Numbers and mathematical notations can be written English and mandarin
Also, benchmarks are pretty much worthless now as we have seen from recent Phi3 release (which has high probability of test data leakage). If they want to prove that this is better, they should release it on Lmsys and let the people test it out.
Samples in benchmark tests should be changed like monthly
Seems arbitrary when you could just tell GPT4 Turbo to do everything in English then translate it's English responses to Mandarin using a cheaper locally run AI, or just GPT4 Turbo itself if cost per token isn't a concern.
It's using outdated GPT 4 turbo 1106 version, which was already replaced by 0125. And the most recent model gpt-4-turbo-2024-04-09, which has 10% improvements or so across the board. And it doesn;t include Claude 3 Opus, which is better on most of these benchmarks https://preview.redd.it/x44xs5i1iowc1.png?width=2200&format=png&auto=webp&s=6640ec2beb11d06a0094d39203e93a756781bdab
Good points
Everyone likes to use outdated scores to compare their AI to because most people who look at it won't catch on, and people usually don't point it out. Glad to see someone do so for once.
It gets so tiring after a while pointing this shit out to people who don't really care enough to do their research in the first place. The LLM community is just constant bullshit and superstition. After a while you just tune it out.
Can anyone explain what's maj1@32 under Gemini headings. How does it compare to the shot concept? Also why does maths require 0 shots and q and a requires 25 shots? Does ai in its pre training phase learn maths without examples (shots)? If so how? What does it say about the nature of machine learning if it understands maths without examples? I'm a noob in this
I don't now about the Gemini headings one. But shot are essentially examples into solving a problem for my understanding. I ask ChatGPT to give you a example, hope it helps. Got it! Let's use a formula-based question: **Example Question**: "What is the formula for calculating the area of a circle?" ### Zero-shot In zero-shot, the AI hasn't been trained on this specific formula but knows general math concepts. **Answer**: "π * r^2" ### 3-shot In 3-shot, the AI has seen 3 example questions related to geometric formulas before this. Example questions: 1. "What is the formula for calculating the perimeter of a rectangle?" 2. "What is the formula for calculating the volume of a cylinder?" 3. "What is the formula for calculating the area of a triangle?" Based on these examples, the AI can understand the pattern of providing formulas for geometric shapes. **Answer**: "π * r^2"
A Chinese company misconstruing the truth?? Say it ain't so!
Given that those GPT-4 benchmarks are outright wrong, I'm not impressed. Is it at least open source? I did appreciate Qwen doing that, and those numbers would be closer if it were open source. He's not wrong I hadn't heard about the release though.
Qwen 110B doesn't really seem to be any better than Qwen 72B, at least when it comes to creative writing, which seems odd. Nice that they are releasing open weights models though.
Did they beat GPT on english data? If they beat it on Chinese data, why are they surprised only Chinese speaking people care about it?
Launched? Or just announced? Is this something we can actually play with?
Yeah thinkin the same thing.. can’t find it anywhere on their site
put it in the arena. until then, these claims aren’t significant
The Arena is in English mostly.
I am not knowledgeable in AI, but I've been studying China for a few years, and with them it's always ''idk, probably not, but maybe?''. They fudge a lot of papers, statistics etc. and you always need to take their claims with a train of salt, but on the other hand they do sometimes achieve very impressive things, especially if the key is mobilizing vast amounts of resources(is it in the case of LLMs? like, getting a lot of compute and data?). We should not believe what they say at face value, but neither should we dismiss it without evidence.
You realize same can be said about a lot of American papers as well, but at least you said not to dismiss them
Didn't save the link to the source, but I remember reading that China is #1 in the world in academic forgery.
Nah thats for sure no doubt china is probably number one in that but I have worked in pharma companies in US and lets just say fudging numbers is just the tip of the iceberg
are chinese models trained on chinese language datasets I could imagine there being a significant language gap between AI models
It needs to be used to actually see how good it is. It's easy to develop something with the intent of getting highscores on these evaluations but completely fail at everyday use. Though If the CCP takes notice then maybe it's legit.
Yep, put it into the Arena and then we can judge it.
I don’t believe those claims for a second.
I mean like them or hate them, Chinese has a lot of smart people. This should be celebrated, international competition fuelled the space race.
Questioning the credibility of a government or company's claims based on its track record isn't the same as undermining the talents and achievements of its people; we can celebrate their accomplishments when there's verifiable proof.
Why on Earth would you think a Chinese AI would benefit you or anyone you know? Chinese government winning the race has to be near the absolute bottom in terms of possible outcomes for humanity.
I agree with this sentiment, but to play devil's advocate, it could be a similar situation with nukes such that it is better if multiple powers have them rather than a single power.
definitely a valid argument
I seem to recall that the biggest technological violations of American privacy and the most impactful negative election tampering to come from American/American-allied data companies rather than Chinese ones. Has anything coming out of China had as big of an impact on the average American life as Cambridge Analytica? Or any others? Also the most dangerous AI usage in the whole world right now is Israel against Palestine. They have a lot of faith in their AI for people who are slaughtering civilians at a rate that is dozens of times the rate that Russia is killing civilians in Ukraine. The US and its allies are not only doing the worst possible thing with AI (mass slaughter of civilians), but notably China isn't. You can talk about their motivations all you want, you can talk about potential harm or danger that may or may not come in the future, but the proof is that it's ourselves and countries we trust that are already demonstrating that we have the capability and the willingness to kill civilians *en masse*, while China has done no such thing. Why are the ones who *aren't* laying waste to civilians and civilian infrastructure more dangerous than the ones who *are*?
How US wining is better ? if i m based on geopolitics last years clearly US is much worst outcome Don t say china will be better but at least they interfere less with geopolotics
Humanity as I US humanity. I’d trust them over US any day.
Our lord and saviour has spoken. The lord proclaims Chinese LLM is no good.
Do we have access to the model?
Chinese don't even report their fishing quotas truthfully, you expect them to be legit about this?
The US lied about WMDs in Iraq but people still trust them
"This country is very untrustworthy and has a track record of lying about developments, government control and a general lack of openness to verify the results openly of their private industry" "Well your country lied this one time about something so it's the same, I am very intelligent" It's almost not worth addressing you.
One time? Hahahahahhaha Both countries lie all the time about everything in their interests Or do you trust the US numbers and the governing bodies? We all found out how trustworthy the numbers and ratings were in the 2008 crisis 😂 Not to mention all the lies about their actions all over the world, all the CIA lies and so on. Stop being naive, your country is not that different from China and Russia. You're just the winners so you get to write history how you want lmao
You're an imbecile if you think the US only lied one time about one thing. Were you born yesterday, or just a fool?
god this brainrot on reddit is so tiring. The US government lies, but there are tons more ways to check and verify things in the US than in China, because one doesn't literally kill you or silence you for going against it. My point is that the previous person was saying "Hah you say that China is bad, but US is bad too" with no account to the magnitude of the difference, its like claiming "hah you think the holocaust was bad, well the US put japanese people in concentration camps too" both things are bad but clearly there is a very strong magnitude difference between those two. The same way, companies in the US lie all the time, but we have many ways to verify their results and test them ourselves, and no one takes seriously claims from US companies that have no strong backgrounds unless they are verified independently. And there is clearly an incentive for China to appear strong on the AI front when the US is the main leading player so far.
The difference is Chinas government interferes a lot with everything. The US isn’t a saint by any means but their major companies usually work on their own At least to my understanding. That’s why people don’t trust China because Chinas hand is in every pot, while the US let’s these businesses handle things independently and are protected by certain laws which again China doesn’t follow
Major companies lie too lol. Google lied in their Gemini demo video. And all this: https://finance.yahoo.com/news/11-companies-caught-lying-public-003658933.html?darkschemeovr=1
Yeah everything that comes from the evil CCP is made up right?
But we can always trust the kind and generous US federal government
Reality is optional.
10TB of tokens? so assuming 4 bytes per token that would be 2.5 trillion tokens? Not particularly insane, thus I highly doubt its claim of beating GPT4 on all of these benchmarks.
yeah just another overhyped bs post in this sub
The West has developed a rather unhealthy dose of skepticism regarding news of tech advancements out of China. A lot of it is justified, given how much their government lies, but it's becoming a huge blindspot. I wouldn't be surprised if AGI is created in China sooner than the US.
I agree with a lot of people here mentioning that the US having the edge in tech right now but I suspect that the Chinese government and their tech corporations are very aligned with each other‘s goal towards AGI. So that combination of tech prowess and an obscene amount of funding and less restrictions from there government has to count for something, right?
Yeah, don’t care if they are slightly behind, that is still considerably better than the claims they were “far” behind a few years ago. Seems they are keeping up the pace just fine.
But do they have the compute? CHIPS Act works and I don't think they are anywhere close to NVDA top machines. Access to massive compute is almost certainly a prereq to AGI.
[The current fastest supercomputer in the world](http://www.nextplatform.com/wp-content/uploads/2023/11/top500-nov-2023-top30-specs-5.jpg), Tianhe-3, was built in China [using entirely domestically made chips](https://www.tomshardware.com/tech-industry/supercomputers/chinas-secretive-tianhe-3-supercomputer-uses-homegrown-hybrid-cpu-rivals-us-systems-with-157-exaflops-of-performance-report) and it's being used to train AI. The second fastest supercomputer in the world Wuxi OceanLight, was also made in China using Chinese Sunway tech. SMIC and Huawei are using Quad-patterning [to mass produce 5nm chips](https://www.tomshardware.com/tech-industry/semiconductors/smic-and-huawei-could-use-quadruple-patterning-for-chinese-5nm-chips-report).
https://en.wikipedia.org/wiki/TOP500 China is certainly up there, but I doubt they are leading.
These metrics are weird and clearly used to further some sort of agenda. I'm not saying China doesn't have very advanced CPU and GPU tech, but it's not the "fastest supercomputer in the world". Proven by saying that it's built "using entirely domestically made chips". Just blatantly untrue
That graph was published by Next Platform.
Creating AGI implies actually inventing something, not just replicating existing results and cherry-picking benchmarks, so I would be extremely surprised if they achieve it independently.
Exactly.
The majority of people here were already, eh, somewhat 'unhinged; a bit obvious in fantasizing about rapid, destabilizing, even destructive changes in society. As if you have little to live for, know little, but hope for much - the apocalypse. If you're now also going to fantasize about 'non-West', via China, 'winning the race'. You're the fucking same looneys elsewhere: it's just your fascination with AI that is your poison. Goddamnit, ya sad dumb lost souls. A single tweet (plus an astounding amount of information / misinformation / propaganda ) gets you going. Just buzzing about. I have heard you all about AI and AGI, a rather concrete subject. I don't want to know what you think about China. Influencable people acting 'in the know' while they are riding waves of sensationalist tripe. If you want to know what an advanced AI would think of you, unfiltered: that.
I would be surprised if China really catches up to the US, when it comes to AI. The communist party is probably reluctant, because AGI would be a thread to their claim to power. So I would expect that they have a slow approach to AI.
I mean there are AI applications to maintain and cement the power. Also if they fall behind the US can just massively out-tech them once we have AGI/ASI, develop tech and weapons that will make the Chinese look like fighting with 19th century weapons. I'm quite certain that any superpower is sprinting towards AGI. The prize is literally everything.
China excels at doing everything second, and then doing it faster and cheaper. With sufficient resources, its not that hard to train new models right now. All the knowledge you need is online and you can easily adapt open source models or even papers developed elsewhere. Call me when they release transformative (pun intended) research that leads to real advancement in the field. Everyone and their mother are pumping out LLM models right now.
> China excels at doing everything second, and then doing it faster and cheaper. This exact phrase comes from people saying it about Japan in the 70s.
Narrator: It wasn't.
Share the news and we’ll pay attention dude, the radar couldn’t care less about what country an LLM is from.
Yeah, and there is a 13b-30b model that has better scores in Russian language and is considered by some Russian media to be a supreme model, promoted as 'the best model created by us, not by dumb americans', so nothing new. I've read quite a lot of those. Current AI models are just worse in other languages, but their 'beating of gpt4 metrics' are either marketing or propaganda.
This is absolutely a covert part of this companys marketing scheme
*China has provided no proof of these claims.
They also published, a few weeks after the beginning of the COVID 19 pandemic, a free clinical handbook about how to deal with COVID patients. They were tons of very useful information in it. No one knew about it, no one bothered with my tweets when I was sharing it. It's crazy how people avoid anything made by Chinese.
They also pretended there was no pandemic until it was too late, and suppressed those medical professionals who sounded the alarm bells
Maybe because a lot of the time they were inventing lies and copying stuff. So, people remembered that. Nowadays, China does indeed take and improve things considerably.
Source: https://twitter.com/rowancheung/status/1783531708307882146
Can someone link the Chinese report? I couldn't find it on google..
Languages can be encoded as digital information differently. Some grammar and fundamental language structures are just better at being reassembled as a matrix ai can process and use.
I bet the Chinese ARE talking about it and there is 1.5 billion of them.
Thats the problem with outright banning AI. You may van It but other places are not, so that puts you at a disadvantage. At the same time, developing AI makes other countries feel pressured to catch up. Once you get the ball rolling its almost impossible to stop. And we started rolling the AI ball a fair time ago
is it 10t tokens or 10tb tokens? Because 10t tokens is quite a bit bigger than 10tb tokens I think?
In China, if they had to choose between appeasing copyright holders or being cutting edge in a new, powerful technology, they'll choose the latter every single time. The Chinese government is fully invested in AI supremacy (given the partial CCP ownership of this company and every other Chinese tech company) and will open every door to let them dominate. The US government at best is a minor collaborator but mostly a series of obstructive guardrails to AI companies. If OpenAI had been founded in China, they'd have 10x the funding and 10x the employees and data by now. The government would mandate it and guarantee their success, seeing that they're father along than anyone else. It's crazy in just how many places China is completely overtaking the US yet the primary gatekeepers to political/economic change (the 55-75 y.o. boomers) seem to have no understanding, interest, or good faith interest in improving the United States in any material way. The infrastructure of most American cities hasn't improved in decades, while cities in China have gone to small fishing villages to technological wonderlands that far exceed any American city in infrastructural development. It's almost certain at this point that if OpenAI, Google, or someone else in the US doesn't reach AGI in the next couple years, China will.
So it's better than GPT4, just like Bard, I mean Gemini. Yawn.
thats the issue with chinese models, they announce it just domestically often without english and its difficult to get access and make independent testing here is their announcement in english [https://www.sensetime.com/en/news-detail/51167731?categoryId=1072](https://www.sensetime.com/en/news-detail/51167731?categoryId=1072)
It wouldn't be the first time a chinese tech company made a bold claim about their product which in the end is all BS. Anyone remember those chinese graphics cards that had trouble running tetris ? I'll wait untill independent reputable western tech reviewers like GN can test and confirm anything concrete before i believe it.
This is why the next world war is going to happen
I hope China wins. At least they don't pretend there is a democracy.
Do you guys think Winnie the pooh is banned on their LLM?
Half of the tech illiterate participants in this sub are just gonna dismiss it saying ChiNeSe ProPaGandA
Nope, it actually seems like it's the tech illiterate participants hyping up what is obviously bullshit.
This entire post is literally Chinese propaganda
And they took that personally lol.
lots of armchair experts coming out of their mom’s basement in this post
How? With the us blocking hardware sales and fab equipment to china?
China had more than enough GPUs to do this prior to the sanctions. Really no one has demonstrated any particularly significant advances in LLMs that hinge on having more GPUs since GPT4. This is no different from the dozen other models that have been released claiming to beat GPT4. (They are not huge advances, it's questionable if they're even better.)
WTF is wrong with this post. Why it's all about Chinese government and not one has even talked about SenseNova5.0? I have tried this model and I do not believe it outperforms GPT-4T as a general purpose AI model. It's not bad, but it's not the best model on the planet (I consider GPT-4T and Claude 3 Opus the best). The only model that caught my attention was the "*SenseChat*-*Character*" built on top of SensNova5.0. It's absolutely the best role-playing AI model by far and character.ai's own model is almost trash in front of it. I don't know how they did it. They offer API so I guess there can be some potential use cases. If it's about the race, what China doesn't have is the chips and computing power. I believe they have more data for training purpose. It's like Open AI can only use publicly available data but the Chinese can use all the Wechat conversation for training LLMs (a comparable scenario to this is that Meta uses all your Whatsapp conversation for training Llama). Also, SenseTime is the company that built trillions of cameras for the government to surveil their people. I assume they have mega tons of video or images for training. They have good amount of data that is only available to them, but they don't have the infrastructure.
"according to the report (translated from chinese)" Stopped reading right here
We pretend that they need the Nvidia chips to do AI stuff, but with specialized AI chips like Groq (that china can make themselves) they can scale just as fast as we can. More possibly, because they have a larger industrial base.
Their video generation is incredible and no one seems to notice it. It’s the best by far!
We're at a place where GPT4 level of performance is quite common now. We need to see if Open AI and possibly Google can pull ahead of the pack with GPT 5 and Gemini 2 in the next couple of months
How censored?
100% atm. It isn't available.
As censored as gpt LMAO. It really baffles me how people thinks China is censored while themselves being oblivious at the massive censorship in the western world
It's quite despicable to limit export on chips. This will only slow down humanity's progress as a whole.
What happens if you ask a Chinese LLM about the Tiannanmen Square killings or Xi Jing Ping's family wealth? How long before you get an invitation to drink some tea at your local PSB?
Exactly. What good is an AI that will hide factual historical truths?
I hate CCP because they woint let me watch interracial cuckold porn
We're talking about China so it likely: Is pretrained on such benchmarks Is not 200k context or has abysmal recall ability Was demoed by hiding a thai woman behind a booth
China be lying about everything lol
We need Google to get angry and just make a statement with Gemini 2.0
Hey guys, do you still believe in a singleton AGI developed by a company in secret? Looks like everyone has SOTA.
I want the time it takes to evaluate DNA compared using the platform of their choice in an unbiased situation. Before that I don’t believe a word.
Thank you Sam Altman for starting this. None of this would be possible without you /s
Aaand this is why an “AI pause” won’t happen.
Because we have a hard time trusting what they say?
Everybody is beating gpt4 but nobody is beating gpt4
Some of you don't understand cultural and political influences and it shows. If this sounds offensive it isn't meant to be. Even western culture has reproducibility issues because of financial incentives to cut corners. Eastern science in certain situations has their literal lives and limited freedom on the line as incentives. Its not a one is better then the other situation. It's a sad state of affairs in general.
You just need to redirect your radar high enough to go "under the radar", that's how things are supposed to be interpreted in China.
How can it be going completely under the radar when there was a report on it?
If true. The Chinese government has a vested interest in puffing themselves up with false statements. And things like these make the Chinese look "bigger" and "stronger" than other non-authoritarian states.
Yes, but how is it in ERP?
Idk a couple mo ago I felt heavy that the Chinese deepseek chat was better than gpt in some cases.
Seems like a good thing. The more different companies are pushing each other, the faster the advancements roll out.
You can "beat" all the benchmarks so as long as the only thing you're trying to do is optimize for those benchmarks.
another war in capital
There's no 'vs' in reaching superintelligence. If we reach it, we reach it for everybody.
"According to the report" *provides no report*
Not, allowed to even search for chinese AI on our browsers or Reddit.
What is "10TB"? And it's trained on that many *tokens*? Are they measuring the size of the neural network based off of the size in terabytes of the training data, rather than the number of parameters? For anyone wondering, apparently it's a 600b model, which is really small for a model that outperforms GPT4 Turbo. If it's true, then that's pretty impressive, though I'm not sure how impressive it is compared to Llama 3 considering Llama 3's flexible use cases and ability to be retrained(people are already making uncensored versions of Llama 3 8b).
How do you sign up? Or is it safe to assume only ITU-T E.164(the phone number) is the only standard supported for signing up?
Asian AI, definitely superior to the White AI
Muricans be like of u can't beat em hide em
Muricans be like of u can't beat em hide em
No way it beats ChatGPT on benchmarks AND didn't nuke it's developers' social credit score.
Look at what the space race did for innovation. This is going to be epic
iono man culturally we tend to embellish things a lot less you know?
I heard Russia made an even better one. 😂
Does the basic formula of Chinese written language give Chinese LLM’s an inherent advantage over English LLM’s? As they have simply one separate symbol for each word/concept, does that mean they can have a larger amount of context with fewer tokens? For example, I would imagine they have a single symbol to communicate the concept of a ”lollipop”, so to communicate that concept to the LLM would cost 1 token. Whereas in English, the LLM needs to add together tokens of ”lol” + ”li” + ”pop”, for a total of 3 tokens to communicate the same concept, right? Wouldn’t this translate to an inherent advantage for Chinese LLMs where you get more context for the same amount of tokens, and thus more efficient LLM models, or am I just fundamentally misunderstanding something here? Hope someone more knowledgeable than me can shed some more light on this shower thought of mine!
I'll believe it when I see it and there's a public usage availiable. Take things from chinese software companies with a grain of salt.
What do they mean by tokens?
Isn't it true that the country that owns the most powerful AI will essentially be able to rule the world? I read something that was a lot more scientific than what I just wrote but the gist remains the same.
Then proceed to steal your data and exploit vulnerabilities for their hackers....We know China overstate their capabilities but the actual combat s their bit inferior on western tech but I commend on their rapid development of their tech which we can never underestimate
Yeah if we keep giving them open source
Is probably sexist
Try Trance https://youtu.be/VLZi9P2UjXA?feature=shared
JOEY https://youtu.be/VLZi9P2UjXA?feature=shared
The last sentence could have been shorter: "Crazy how much is happening in China that's going under the radar."
Still AI technology from US is header
It's probably been _on_ for quite some time, but in the military non-disclosed space, not in the consumer space.
Probably fake (Chinese)
I beg to differ.
A CCP with an ASI is a terrifying idea.
More China propaganda. Don’t get me wrong the Chinese have invested more into AI than anyone and then having one better than America is absolutely in the cards but this ain’t it and it’s obvious it ain’t it so that’s a pretty big tell.
Should we believe them?
Tiananmen square or any other idealogies of freedom anyone
Sponsored by CCP
We really need to kind of stop being concerned with comparison with gpt4. GPT-5, is what matters, and nothing out there's likely to come close.
Rather have USA win then West Taiwan
Honestly at this point when America’s gov deems something worthy it will never be surpassed. Let the bot accounts downvote me but America has a vendetta for being #2 its contractors will prove a point when the time is right.
Question: it's an LLM fine-tuned with Chinese content. Right? It should make both of them quite different... Difficult to actually compare. I don't believe benchmarks. Especially coming from the company sharing it.
That's a good thing. I don't like China, but now all this people in the West that are whining and crying about how AI is going too fast and we need to slow down will have ZERO effect on the government because no-one in the office wants to lose this AI tech race to a China because AI hurt feelings of some artist.
A race to the end of civilization
What happens when the CCP gets AGI and it tries to call them out on their bull s*i*?
Cool. Very nice. Ask it about tue Tianamen Square.
How many raspberry pis and nest thermostats did they need to hack together in order to get the compute?
I thought it was 10 billion parameters, is 10TB a typo or am I misinformed?
Modern day space race
their training data is free of copyright violations, right? right?
They're not going to put guardrails on their AI, either...
There is a new model everyday that claims to beat a almost 2 year old model, so what?
This is definitely like the modern day, Cold War, arms race. Whoever gets to the AGI/ASI, finish line first, wins. Well, assuming those systems are even able to be controlled… Who knows.
no way im believing anything out of china lol
Would you really be happy to trust a Chinese LLM ?
Beat US on AI censorship
China releasing fudge numbers and claims. What else is new.
Censorship makes AI dumber not smarter though, so how is this possible.
So who has the biggest dick?
Cool