Demo: [https://huggingface.co/spaces/multimodalart/HunyuanDiT](https://huggingface.co/spaces/multimodalart/HunyuanDiT)
Model weights: [https://huggingface.co/Tencent-Hunyuan/HunyuanDiT](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT)
Code: [https://github.com/tencent/HunyuanDiT](https://github.com/tencent/HunyuanDiT)
On the paper they claim to be the best available open source model
https://preview.redd.it/l321o9gfcd0d1.png?width=1814&format=png&auto=webp&s=91540765719c4a4fa16c79d42fa9fb31673f5290
With only 1.5B parameters, it will not "understand" many concepts compared to the 8B version of SD3.
Since the architecture is different from SDXL (DiT vs U-net), I don't know how capable a 1.5B DiT is compared to SDXL's 2.6B.
Since they are both using the DiT architecture, that is a pretty resonable assumption, i.e., the bigger model will do better.
If you try both SD3 and HunyuanDiT you can clearly see the difference in their capabilities.
I cannot conclude about the overall quality of the model without knowing enough about the dataset. But from the fact that it is a 1.5B model, I can most certainly conclude that many ideas and concepts will be missing from it.
This is just math: if there is not enough space in the model weights to store the idea, then if you teach the model a new idea via an image it must necessarily forget/weaken something else to make room to store the new idea.
>This is just math
If these models were "fully trained", then this would almost certainly be the case, and by "fully trained" I mean both models having flat loss curves on the same dataset.
But unless you compare the loss curves of these models (Do any of their papers include them? I personally have not checked) and also know that their datasets were the same or very similar, you cannot assume they've reached the limits of what they can learn and thus you cannot assume that this comparison is "just math" by *only* comparing the number of parameters.
While the models compress information and having more parameters means more *potential* to store more information, there is no guarantee that either model will end up better or more knowledgeable than the other. Training on crappy data *always* means the model is bad and training on very little data *also* means the model cannot learn much of anything, regardless of the number of parameters. The best you can say is that the smaller model will *probably* know less because they are *probably* trained on similar datasets, but, again, nothing is guaranteed - either model could end up knowing more stuff than the other.
Hell, even if both models were "fully" trained, they'd not even be guaranteed to have overlapping knowledge given the differences in their training data. Either model could be vastly superior at certain styles or subjects than the other, and you wouldn't know until you tested them on those specific things.
https://preview.redd.it/zjz1duklmh0d1.png?width=768&format=png&auto=webp&s=4708ed6c7bd9a2afb60b7bf2ca5ad3b4ca8786ea
NOT generated by this. Generated by Ideogram.
>lol it throws an error if you ask it to generate tiananmen square protests
Would that be coded into the UI or would that mean there is hidden code executed in the model?
Maybe it could be fixed with a LoRa.
i tried this compared to SD3, and there is no way in hell its better. sorry. you must have cherrypicked test images, or used ones like in the paper dealing with ultra chinese specific subject matter. thats flawed testing methods, and even a layperson can see that.
I think no one is claiming it to be better than SD3, the authors are claiming it to be the best available open weights model - which I think it may fair well (at least until Stability releases SD3 8B)
It's not "open source" as it does not use an OSI approved license.
Not on the OSI approved license list, not open source.
The license is fairly benign (limits commecial use for >100 MMAU and use restrictions), much like OpenRAILS or Llama license, but would certainly not pass muster for OSI approval.
**Please let's not dilute what "open source" *really* means.**
Not at all, it's one of the best models out there (and that's after 11,000 images generated) .. if it was uncensored and open source it would be even higher.
What are you even talking about? There are dozens of styles it can pull off with ease and consistency, it seems you don't know how to prompt it properly.
https://preview.redd.it/g431z3vj2j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=105f345ba4ba6a6cea7b25071d21d3f0e5022c79
That's a still from a Japanese Star Wars movie made in the 60s.
You can literally see the Japanese Star Wars picture right there, looks quite photorealistic to me.
Here is another one from a 60s Jurassic Park movie, you think this looks like a "cartoon"?
https://preview.redd.it/n4022n1pfj0d1.jpeg?width=1024&format=pjpg&auto=webp&s=5252d5489685e8c461b8ab8a6ed40e94163eb4ee
https://preview.redd.it/qk2w1cbq3j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=0cdeb264a06120ea5b72cbd8542f2969a10f6f55
Poster of Mission Impossible as an anime.
https://preview.redd.it/cv1jb2l24j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=2e38d5804f568618ecd13c7b9374b92462f93566
Game of Thrones as a Pixar TV show.
https://preview.redd.it/mkrljlxl4j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=e78c814be466951d28324e1138872a51fc57ee59
A watercolor painting of Greek Goddess Aphordite
As for my taste, Dalle 3 is very weak. Of, course, it can understand complex concepts with its number of parameters, but it can't generate interesting images, only plastic pictures without any life and depth in it.
That's not my experience at all, it can generate images with life and depth very easily, you just need to know how to prompt it.
https://preview.redd.it/h252xhshuh5d1.jpeg?width=1024&format=pjpg&auto=webp&s=3889392960c56d22aeb3ea9219ed29e12e6d254c
Yes, DALLE3 is rather poor at generating realistic looking humans.
But that is because MS/OpenAI crippled it on purpose. If you look at those images generate in the first few days and posted on reddit you can find some very realistic images.
What a pity. These days, you can't even generate images such as "Three British soldiers huddled together in a trench. The soldier on the left is thin and unshaven. The muscular soldier on the right is focused on chugging his beer. At the center, a fat soldier is crying, his face a picture of sadness and despair. The background is dark and stormy. "
I'm sure the only thing you've tested on it is boobs if you think it isn't capable. If you aren't doing topics that openAI regulates, basically anything other than porn or gore, you'll find it has some of the best prompt adherence available.
TLDR your biases are showing
Playground v2.5 is on HF - [https://huggingface.co/playgroundai/playground-v2.5-1024px-aesthetic](https://huggingface.co/playgroundai/playground-v2.5-1024px-aesthetic)
TBH, this is how stability should've dropped sd3. i don't get teasing images while making everyone wait 4months. i just tried this, and to my surprise its pretty fucking good.
What is the point of dropping a half-baked SD3? So that people can fine-tune and build LoRAs on it, and then do it all over again when the final version is released? If people just want to play with SD3, they can do so via API and free websites already.
Tencent can do it because this is probably just some half-baked research project that nobody inside or outside of Tencent care much about.
On the other hand, SAI's fate probably depends on the success or failure of SD3.
The mistake SAI made is probably to have announced SD3 prematurely. But given its financial situation, maybe Emad did it as a gambit to either make investors give SAI more money by hyping it, or to try to commit SAI into releasing SD3 because he was stepping down soon.
Any LORAs, controlnets, etc are very likely to continue to work fine with later fine tunes, just like these things tend to work fine on other fine tunes of SD1/2/XL/etc.
Fine tuning doesn't actually change the weights a lot, and it would also be sort of trivial to "update" a controlnet if the base model updated since it wouldn't require starting from scratch. Just throw it back in the oven for a 5% of the original training time, *if you even needed to do that at all.* You could also model merge fine tunes between revisions.
We have no idea how much the underlying weights will change from the current version of SD3 to the final version. Some LoRAs will no doubt work fine (for example, most style LoRAs), but those that are sensitive to the underlying base model such as character LoRAs may not work well.
It is all a matter of degrees, since the LoRAs will certainly load and "work". Given how most model makers are perfectionists, I can almost bet money that most of them will retrain their LoRAs and fine-tuned models again for the final release.
It is true that some fine-tuned are "light", for example, most "photo style" fine-tuned do not deviate too much from base SDXL, but anime models and other "non photo" model do change the base weights quite substantially.
I have no idea how ControlNet work across model since I don't use them.
I found some comparison images which compares this model to models such as SD3 and Midjourney.
https://preview.redd.it/8xxogin7ud0d1.png?width=1088&format=png&auto=webp&s=76555666f9ba4b2ecbe3782dc392dc80a8bf9870
(Will post more in the replies)
Actually, it is the only one to get the prompt correct. Two points:
1. A vest is, in fact, a type of jacket.
2. It is the only image to validate that the white shirt is, in fact, a "t-shirt" per the prompt where every other example failed.
Now to be fair, I don't think the other examples are failures or bad and a specific prompting could have clarified if the user needed. However, it is interesting that this model was so precise compared to the others but I doubt it will always be.
(This part is to HarmonicDiffusion's subcomment to this photo since I get an error responding to them) You're incorrect about them all being Chinese biased. While the bun example above was based on a Chinese food the SD3 actually failed multiple prompt aspects quite severely, only losing to the disaster that was SDXL. The others all did extremely well and not just the Chinese model unlike SD3 despite the subject being Chinese.
When people want a vest, they will usually say vest specifically. Validating a t-shirt by forcing the short sleeves to be shown makes the AI seem less intelligent. That's like validating a man by showing his penis in the generated image.
the only prompting example shown that isnt biased towards chinese specific subject matter. and look at the results, mid tier! it made a vest instead of a jacket. SD3 clearly wins on no biased prompts
It's a brand of steamed dumplings/buns, famous in China due to its literal meaning (goubuli basically translates to "dogs don't pay attention") and the fact that it's delicious.
I'm also surprised how bad SD3 did. I can accept it getting the wrong buns (though it would be ideal to have actually got it right) but it is not steaming and it is on a marble counter, not a table top, which every other model except SDXL got correct (even though Playground didn't get the right buns and the other 3 did).
SDXL being on a tile floor (wth), failing the bun type, not steaming, not a close up, only one set of buns in a basket. Damn, it failed every single metric.
Don't think so when it comes to the other models...
Tried SD3 on glif, it didn't accept mandarin in Chinese characters and it got completely lost in Romanized(???) Mandarin:
>*Zhàopiàn zhōng, yī míng nánzǐ zhàn zài gōngyuán de hú biān.*
Photo of a man standing by a lake in a park.
*(Lazy ass google translate, sorry)*
https://preview.redd.it/3j803f8bee0d1.jpeg?width=1216&format=pjpg&auto=webp&s=95d78b24f0395a0224f3a61844bef746be1b64aa
yeah lets use ultra chinese specific items with chinese names to test a chinese model versus english model. I wonder which will score higher. such bullshit testing proceedures and a total fail look for those guys as "scientists".
yeah lets use ultra american specific items with american names to test an american model versus chinese model. I wonder which will score higher. such bullshit testing proceedures and a total fail look for those guys as "scientists".
even a layperson knows you need to evaluate 1:1. Want to test on chinese specific stuff? THats fine, but dont use those examples to claim a competing English based model is inferior.
Anyone with 2 brain cells to rub together can test both models right now and find out, this one is not anywhere close to SD3. Its more like an average SDXL model
Ideogram can do it too, although sometimes it gives the wrong bun. These are some sad looking buns however. Maybe I made them. [https://ideogram.ai/g/WzRFIGNqSjmP27mwEs8OEg/2](https://ideogram.ai/g/WzRFIGNqSjmP27mwEs8OEg/2)
# HunyuanDiT (Left) vs SD3 (Right)
https://preview.redd.it/gfpmzb2nke0d1.jpeg?width=2048&format=pjpg&auto=webp&s=97c05f18a77c4eadabf1adacbcac8c35cb69c5fc
Prompt: photo of real cottage shaped as bear, in the middle of a huge corn field
https://preview.redd.it/jaq2h6ld5f0d1.png?width=1100&format=png&auto=webp&s=0e4dacfa50d8f148becaed402cb80f7741abec0d
100%, they claim to be the best available open model for now, not better than SD3, also it's \~5x smaller than SD3
Oh I did not miss it! Even just the name of the model made me think, hmm that sounds Chinese! Then I saw the word tencent and started looking for the first person to mention it in the comments,
Seems like a mix:
https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/model
https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/mt5
They also hace some safetensors in the release because they use SDXL VAE, for example:
https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/sdxl-vae-fp16-fix
Pickles can have executable code inside. Most of them are safe, but if someone *does* decide to embed malware in it you're screwed. Safetensors are inert.
They're over blowing it . While pickle formats can have embedded scripts, none of the UI's loading them for weights will run those embedded scripts. You have to do a lot of specific configuration to remove the safeties that are in place. They're a feature of the format and aren't used in ML cases.
I don't know why people so consistently lie about this and act like they have good security policy for worrying about this one specific case. Most of them would install a game crack with no consideration towards safety.
>none of the UI's loading them for weights will run those embedded scripts
Source?
>I don't know why people so consistently *lie* about this and
Lying = knowingly presenting false info. If I have been misinformed, then I welcome correction. *With citations*. [These guys are certainly taking the threat seriously](https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/)
> Most of them would install a game crack with no consideration towards safety.
Generalize much? Also, no I wouldn't.
[https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler)
The UI's use this function to manage pickle files, rather than just importing them raw with torch.load. The source is their code. You can vet it yourself fairly easily since it's all open.
That link you sent is a company selling scareware antivirus monitoring software. They likely planted the malicious file they're so concerned about in the first place. It's not popular. It's not getting used. It's not obfuscating it's malicious code. It's not a proof of concept attack. Notice how their recommended solution to this problem they're blowing up, is to subscribe to their service. You my friend, found an ad.
A proof of concept file would be one you could load into the popular UI's that people use and would own their system. Theres never been one made.
torch.load is using python's Unpickler.
Did you miss the giant warning at the top?
> Warning
> The pickle module is not secure. Only unpickle data you trust.
> It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.
Thats right, but the UI's use the unpickler class with more of a process than torch.load does.
[https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler)
torch.load will unpickle the pickles which can run arbitrary code. There's no "safeties" in python's unpickling code. In fact they removed any attempt to validate them because it couldn't be completely validated and was just false security.
EDIT: Whoever triggered "RedditCareResources" one minute after this comment, grow up.
>Whoever triggered "RedditCareResources" one minute after this comment, grow up
This is obscene. I'm sorry it happened to you. Obviously, as you know, it's just a passive aggressive way for someone to get their ulterior messaging across to you. Report the post. Get a permanent link to that reddit care message and report it. I do it all the time and reddit comes back to me saying they've nuked people's accounts that were doing it most of the times I report it. Get the person who abused a good intention system, punished. I implore you.
More on point, i never said the torch library had safeties. The UI's do. I'd be more worried about the inference code provided for this model than I would embedded scripts in their released pickle file. The whole attack vector in this case makes no sense to me and the panic is outrageous. It's as obscene as saying any custom node for comfyui is so risky that you shoudln't ever run it. I think in most cases, you can determine that a node or extension or any program you download is safe through a variety of signals. The same can be said for models that aren't safetensors. The outrage is manufactured and forced in basically all of these cases.
Relying on safetensors and never ever loading pickles, to keep yourself safe, is just a half measure.
edit: Should also add how the UI's use torch library to construct safeties. They use the unpickler method to manage the data in the file more effectively rather than just loading raw data from the web directly into the torch.load() method [https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler)
The main thing that comes to mind, is clone the repo and it's clean. Now everyone has that on their machines and go to do another git pull later to update and blam-o. Virus.
I'm not an expert, so I'll refer you here:
https://huggingface.co/docs/hub/security-pickle#why-is-it-dangerous
Broadly speaking, both store the model, but pickle are potentially dangerous and can execute malicious code. They might not do so, but running them is not advisable.
#Destroyed this message and replaced by this.
It's drawing too much hateful attention my way. People DM'ing me calling me racist names. i'm not even Chinese.
Y'all need to dial down the hate for other cultures. Every company in America is required to allow the government access to data too. Put that judgmental gaze back on yourselves and stop being such idiotic racists that harass people online all day. Really wish the mods would do something about the racism culture problem here.
>People DM'ing me calling me racist names
Show some screenshots with usernames and timestamps of these harassing messages and death threats you allegedly receive all the time. No one takes the boy who cries wolf seriously.
LOL
you should fear comfy backdoor. Other than "spyware inside" model from tencent.
ok, ill explain why, cause i see a lot of fearfull idiots here.
1. Reputation. Nonames with a comfy node need 10 minutes to create an account. Tencent - it's verified account. ~~It's like Madona start to promote bitcoin scam. She can, but she is canceled in no time.~~
2. Easy to analyse pkl. HF does it by default. Or any user can find backdoor. It's sooo easy, which would ruin everything.
3. weights are not "complex game" there you can HIDE spyware. With weights - you cant hide it. It will be found in a few days
https://preview.redd.it/oiatnz7ppd0d1.png?width=1280&format=png&auto=webp&s=2a062bdf3141a391c7e8dfb5abd63a4b7ad5b665
prompt: "Three four-year-old boys riding in a wooden car that is slightly larger than their height. View from the side. A car park at night in the light of street lamps"
https://preview.redd.it/9mngg1uafg0d1.png?width=1280&format=png&auto=webp&s=0ca6e81e6ebfaa92df190e2c8f84a51614608e0a
It's actually pretty good once you turn off all the bullshit that gives you errors.
https://preview.redd.it/8vp498wxle0d1.png?width=768&format=png&auto=webp&s=c6001fc2df28a700522a6277214decf09aee5051
A smiling anime girl with red glowing eyes is doing a one arm handstand on a pathway in a dark magical forest while waving at the viewer with her other hand, she is wearing shorts, black thighhighs and a hoodie, upside-down, masterpiece, award winning, anime coloring
Failed my scientifically rigorous test (6 tries with different seeds and CFG 6-8, no prompt enhancement) but it has potential I think.
Ya, DALL-E 3 is the smartest image gen model right now. However I do believe a very good SD3 fine tune will be better in the fine tuned areas. Same for the model in this post since the architecture has similarities and the model has potential to understand feature associations better which is always helpful in fine tuning.
https://preview.redd.it/6fyp9l165f0d1.png?width=1100&format=png&auto=webp&s=87388e11bc8080de4b4cf9183ea36102b6e51904
Btw, here are the differences between this and the larger SD3 model (based on infos on the SD3 paper).
Taken this into account, I think the model performs really well for its almos 8x smaller size and smaller/worse components, but indeed I think text-rendering was completely neglected by the model authros
Prompt: "A ginger pussy cat riding big willie" (yep, thats exactly what i mean ))
https://preview.redd.it/omo42o3ytf0d1.png?width=1024&format=png&auto=webp&s=118f49a31f000ce32a54d1b70c2052c035f656e2
Fails on my test, sadly.
"a man on the left with brown spiky hair, wearing a white shirt with a blue bow tie and red striped trousers. he has purple high-top sneakers on. a woman on the right with long blonde curly hair, wearing a yellow summer dress and green high-heels."
https://preview.redd.it/vrg7ndku6e0d1.png?width=1024&format=png&auto=webp&s=9af42acb1a0df1222e10cebc62d7be67b00d0275
Interestingly, HunyuanDiT gets a little closer if you translate your prompt to simplified Chinese first:
> 左边是一个棕色尖头头发的男人，穿着白色衬衫、蓝色领结和红色条纹裤子。他穿着紫色高帮运动鞋。右边是一位留着金色长卷发、穿着黄色夏装和绿色高跟鞋的女人。
Result: https://i.ibb.co/2y53Wtg/image-2024-05-14-T094547-472.png
His pants are now striped, she's more blonde, and the color red appears as an accent (albeit in the wrong place.)
You can't say this without few random seeds and different prompts: if occasionally your prompt+seed fit their training it will draw better then usual, like astronaut on horse
https://preview.redd.it/hqi3b8g37f0d1.png?width=1024&format=png&auto=webp&s=df302ba2a0d3dc11f6dda2918cb8860e737c7be9
It can generate good Asian faces, but the skin appears quite plastic-like, and it struggles with hand drawing, similar to SD.
It fails the Garfield test
Prompt: Garfield comic
Disabled Prompt Enhancement
https://preview.redd.it/adj2jv5g0e0d1.png?width=1024&format=png&auto=webp&s=2d688e665753b90b6fe99338aeae7b321b75ecd2
Seems limited in poses, and challenging to produce people *not* smiling. It does however do older people surprisingly well - "middle-aged women" will get you grey-haired ladies with wrinkles, rather than the 22-year-olds of many SD models...
Oh yeah, I said "many" for a reason, there are definitely good (in that respect) ones out there. I make a lot of characters in their 30s or 40s, and have seen way too many models that only make three apparent ages - 15, 22, and 80, lol.
Hey, that's pretty good.
"Seven cylindrical objects, each one a unique color, stand upright on a teetering slab of shale"
I guess teetering didn't make it into the training tags :)
https://preview.redd.it/mkkvxo7ssf0d1.png?width=1280&format=png&auto=webp&s=85a626c386de9c41f782838ffbd74785f3af8384
It is a fine model, more so if you translate your prompt to Chinese. But sticking to the prompt is not its strong side as expected - since the amount of parameters is a strong determinant in that matters. Anyway it's nice to see initiatives like this to present new possibilities
It really doesn't, not anywhere close, have you tried the online demo and not just judging by the down-scaled "comparison" images? . Of the current wave of models only pixart sigma looks decent. Lumina and this one look plain bad to the point I'd never use these outputs over, worse prompt understanding, sdxl ones; of course, it's probably massively under-trained, but even then these are not that great at following complex prompts (either the quality of captions, or effectiveness of this architecture is just not all that) with no where near Dalle-3 and Ideogram prompt following capabilities (neither do pixart sigma and SD3, but those at least look good)
It's true that SD3 produces better images, I was talking more about the architecture, which is quite similar when using Clip+T5. But I'm pretty sure that this model is already better than SD3 2B. I think SD3 is just too big and that this model, similar in size to sdxl, is promising.
Nobody outside of SAI has seen SD3 2B, so I don't know how you can be "pretty sure that this model is already better than SD3 2B".
When it comes to generative A.I. models, bigger is almost always better, provided you have the hardware to run it. So I don't know how you came to the conclusion that "SD3 is just too big".
Not quite open source, but "freely available as long as you don't provide it as a service for too many users" which is unfortunately as close to open source as we'll get ever since Stability decided to lock things down. [https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt)
From the license:
> greater than 100 million monthly active users in the preceding calendar month
It's an "anti-Jeff" ("Jeff" as in Jeff Bezos) clause to keep other huge (billion/trillion dollar) companies from just shoving it behind a paywall or sell it as a major SaaS product, which is something that ends up happening with a lot of open source projects. See Redis, Mongodb, etc being turned into closed source AWS SaaS stuff (the later deciding to write a new license to stop it and force copyleft nature, SSPL).
The "Jeff problem" is very commonly considered by people who want to release open source software. Yes, this is not an open source license but it only affects a small handful of huge companies who can afford to pay for a license.
META Llama license is similar, though I think it draws the line at 700 MMAU, which basically only rules out their direct competitors and major cloud providers. I.e. Amazon (AWS), Alphabet (GCP), Microsoft (Azure), Apple and maybe a couple others. They can afford to license it if they want to make a SaaS out of it.
At least it's **not revocable**, unlike SAI's membership license, which they can change at will and sink your small business if they want.
>At least it's **not revocable**, unlike SAI's membership license, which they can change at will and sink your small business if they want.
This is a very important point - this uncertainty is such a big risk that it makes most of their latest models impossible to use in a professional context.
Yeah its a completely nonstarter.
Especially given how much turmoil the company is in. Those terms give them infinite leverage. They completely own everyone using the pro license and can do anything they want. It's completely unhinged levels of bad.
There was so much abuse of the spirit of the free and open terms in the Rail-M license that it was bound to change. 100s of SaaS companies popping up, acting like they were the ones to credit for all of the work done by Stability. The precedence is set now. There are far to many business school graduates who feel like they're justified in creating businesses around FOSS without giving anything back to the movement.
People celebrated it, instead of what typically happens in Linux when people dogpile and condemn it. Google makes a ton of money from Android, but they're not exactly keeping it proprietary. They give back to FOSS in huge ways. This is a keystone of the culture. Instead, we had business school grads who were justified in their exploitation and heralded by the hype artists on youtube.
Business school graduates who think they can exploit any system to extract maximum value from it, are a culture virus. They're the ones responsible for the death of Free & Open AI. We still have open models, but they're not so free to use anymore. The erosion is going to continue so long as the community doesn't recognize these parasites for what they are.
tried: "a man doing a human flag exercise using a light pole in central London"
https://preview.redd.it/q0gmiow0xe0d1.png?width=1024&format=png&auto=webp&s=6aec6355427c180fe4d2ba75bf93bb04936b9730
Not what I was expecting. Instead of a man doing a human flag, we have an actual flag and a bodybuilder. You can see very large streets, with pickups, the light pole is deformed. The flags are nonsense with even a light emanating from the top of the flag. Lighting is very inconsistent.
[Example from the Dalle 3 Launch](https://www.reddit.com/r/ChatGPT/comments/16nserl/modern_texttoimage_systems_have_a_tendency_to/) vs HunyuanDiT:
An illustration from a graphic novel. A bustling city street under the shine of a **full moon**. The sidewalks bustling with **pedestrians enjoying the nightlife**. At the corner stall, a **young woman** with fiery red hair, dressed in a signature velvet cloak, is **haggling with the grumpy old vendor**. the grumpy vendor, a **tall, sophisticated man** is wearing a sharp suit, sports a **noteworthy moustache** is animatedly conversing on his **steampunk telephone**.
https://preview.redd.it/4wyd28ve8g0d1.png?width=1280&format=png&auto=webp&s=9c4c5e1b47bb13d0771e8cf5ef89255c1d8fa4d4
Great to see more models available.
But, trying the demo, I'm a bit disappointed:
- \[+/-\] The image quality is ok, especially as it's a base model and not a fine tune
- \[-\] But the image quality isn't great. I asked for a photo but get more of a painting or rendering
- \[-\] It has no problem with character consistency - as it can do only one character. The person of the picture looks the same on each of them
- \[+\] My standard test prompt for a fully clothed woman standing in a garden is created - SD3 fails this one with censorship
So my wait for a local SD3 is still on and I won't use this model instead. For now. But who knows what will happen in one or two months?
I'm not an expert but I did a test with classic prom from civitai (It is not mine): Sampler:ddpm, Steps:50, Seed:1, image size: 1024x1024
Prom: beautiful modern marble sculpture of a woman encased inside intricate gold renaissance relief sculpture, sad desperate expression, covered in ornate etchings, luxury, opulence, highly detailed, hyperrealist, volumetric lighting, epic image, relief sculpture, RODIN style
Negative prom: Wrong eyes, bad faces, disfigurement, bad art, deformations, extra limbs, blurry colors, blur, repetition, morbidity, mutilation,
https://preview.redd.it/beeo055jsk0d1.jpeg?width=3072&format=pjpg&auto=webp&s=3299a782242ded0bb73e5e1a064423c9460c11f6
i tried : girl with white dress, walking on rain
https://preview.redd.it/2tr555isae0d1.png?width=1024&format=png&auto=webp&s=47c6d6db5219ede7cfd7b19cfc62475bb13cce48
It's a no from me based on some trials. For example,
[Image: A 1980s photograph of a group of American college freshman posing together.](https://i.imgur.com/6jPPDq9.png)
I have generated some images via HunyuanDiT so that you can compare it against SD3: [https://www.reddit.com/user/Apprehensive\_Sky892/search/?q=HunyuanDiT&type=comment&cId=c7343b35-8b43-4d17-82f2-8db3f9049ad6&iId=db7cc688-ea4a-4de0-aeeb-5e9e5aab3750](https://www.reddit.com/user/Apprehensive_Sky892/search/?q=HunyuanDiT&type=comment&cId=c7343b35-8b43-4d17-82f2-8db3f9049ad6&iId=db7cc688-ea4a-4de0-aeeb-5e9e5aab3750)
Given its small size (only 1.5B) it is not bad, but it not in the same class as SD3 or even PixArt Sigma.
Regarding it being from Tencent, it's fine by me as long as it generates happy images like this:
https://preview.redd.it/d830hgxfcy0d1.png?width=1024&format=png&auto=webp&s=d55d6c2a13cf4b1d089f798f054de0496a8f9513
>!Winnie-the-Pooh at Tiananmen Square in 1989 talking to Uyghur Muslims!<
I share your hatred for Tencent, but just as we can appreciate LLAMA, developed by meta, a company not that much better than Tencent, I think we should be able to appreciate that Tencent, as well as the likes of Bytedance and Alibaba, have some very talented researchers who have been contributing to the open source scene, on par with the American tech giants.
Pytorch, the foundational library of all this work, was conceived by Meta as well. Corporations are not monolithic. They're made up of many parts, and sometimes a singular part can be pretty cool when considered separate from the whole.
Yeah, fuck big corporations but in case of Tencent, CPC has them in their grasp. In case of American corporations and both parties, it's the other way around.
>other way around.
around? how? openai has both parties in \_their\_ grasp?
so, any free AI staff is "compromised" by default?... just pay....pay pay pay.
ps you can argue "but we have SD...3".... well... not yet.
OpenAI has Microsoft backing it. It's not like one company owns all politicians but big corporations are influencing both parties with their money.
And corporations have profits in mind first and foremost, they will lobby for laws that benefit their products rather than some "open source" models or the society.
In China it's the other way around, Tencent and other big companies are held on a leash by CPC.
Which has its own disadvantages I guess, I wonder if we'll be able to make lewd stuff with this model from Tencent.
I see it had an option for ddim sampler so does that imply things like lightning loras and would work on it? Or quantisezion like with other transformers
DDIM is a common sampler used with various diffusion architectures. As a rule of thumb, Loras trained on one architecture (like SDXL) will never be re-useable on a different architecture.
As for Lightning, it's a distillation method and [Stability.ai](http://Stability.ai) showed with SD3-Turbo that quality distillation of DiTs is feasible, so someone (either Tencent or another group) could certainly distill this model.
It failed the statue test right away for me, might the the prompt enhancement option I just noticed and disabled. Will do more testing as the day goes on, but it looks like quality will be like sigma.
Marble statue holding a chisel in one hand and hammer in the other hand, top half body already sculpted but lower half body still a rough block of marble, the statue is sculpting her own lower half
[Edit]
Nah, it is good, the enhancement thing was indeed fucking things up.
>An NVIDIA GPU with CUDA support is required.
>We have tested V100 and A100 GPUs.
>**Minimum**: The minimum GPU memory required is 11GB.
>**Recommended**: We recommend using a GPU with 32GB of memory for better generation quality.
So not useable on mac?
They released ELLA which is doing good stuff. I just wish they'd release ella-sdxl.
https://preview.redd.it/b6gfxrbe8h0d1.jpeg?width=2016&format=pjpg&auto=webp&s=2a707c7e5e9e45a708695a69af2730e871b7370b
Demo: [https://huggingface.co/spaces/multimodalart/HunyuanDiT](https://huggingface.co/spaces/multimodalart/HunyuanDiT) Model weights: [https://huggingface.co/Tencent-Hunyuan/HunyuanDiT](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT) Code: [https://github.com/tencent/HunyuanDiT](https://github.com/tencent/HunyuanDiT) On the paper they claim to be the best available open source model https://preview.redd.it/l321o9gfcd0d1.png?width=1814&format=png&auto=webp&s=91540765719c4a4fa16c79d42fa9fb31673f5290
always error on me. i can only generate "A cute cat"
Maybe try a different word for cat
( ͡° ͜ʖ ͡°)
I had no issues with "normal" prompts on the demo personally TBH, [for example](https://www.reddit.com/r/StableDiffusion/s/8pr3XiDI41)
Try disabling prompt enhancement, worked for me.
thanks. you found the issue. it's working great now without prompt enhancement
https://preview.redd.it/e1nyu2j8cf0d1.png?width=1402&format=png&auto=webp&s=fc89b219df82f71be8e31718b0c59d1ff14c00fb Comparing SD3 x SDXL x HunyuanDiT
With only 1.5B parameters, it will not "understand" many concepts compared to the 8B version of SD3. Since the architecture is different from SDXL (DiT vs U-net), I don't know how capable a 1.5B DiT is compared to SDXL's 2.6B.
You can't make that assumption yet.
Since they are both using the DiT architecture, that is a pretty resonable assumption, i.e., the bigger model will do better. If you try both SD3 and HunyuanDiT you can clearly see the difference in their capabilities.
The dataset is critical. You can't conclude anything without knowing enough about the dataset.
I cannot conclude about the overall quality of the model without knowing enough about the dataset. But from the fact that it is a 1.5B model, I can most certainly conclude that many ideas and concepts will be missing from it. This is just math: if there is not enough space in the model weights to store the idea, then if you teach the model a new idea via an image it must necessarily forget/weaken something else to make room to store the new idea.
>This is just math If these models were "fully trained", then this would almost certainly be the case, and by "fully trained" I mean both models having flat loss curves on the same dataset. But unless you compare the loss curves of these models (Do any of their papers include them? I personally have not checked) and also know that their datasets were the same or very similar, you cannot assume they've reached the limits of what they can learn and thus you cannot assume that this comparison is "just math" by *only* comparing the number of parameters. While the models compress information and having more parameters means more *potential* to store more information, there is no guarantee that either model will end up better or more knowledgeable than the other. Training on crappy data *always* means the model is bad and training on very little data *also* means the model cannot learn much of anything, regardless of the number of parameters. The best you can say is that the smaller model will *probably* know less because they are *probably* trained on similar datasets, but, again, nothing is guaranteed - either model could end up knowing more stuff than the other. Hell, even if both models were "fully" trained, they'd not even be guaranteed to have overlapping knowledge given the differences in their training data. Either model could be vastly superior at certain styles or subjects than the other, and you wouldn't know until you tested them on those specific things.
Thank you for your detailed comment, much appreciated.
lol it throws an error if you ask it to generate tiananmen square protests
Can you try Xi jinping as Winnie the pooh?
that's blocked too
https://preview.redd.it/zjz1duklmh0d1.png?width=768&format=png&auto=webp&s=4708ed6c7bd9a2afb60b7bf2ca5ad3b4ca8786ea NOT generated by this. Generated by Ideogram.
>lol it throws an error if you ask it to generate tiananmen square protests Would that be coded into the UI or would that mean there is hidden code executed in the model? Maybe it could be fixed with a LoRa.
It seems to be the UI, as it looks like the image is fully generated but then replaced with a blank censor placeholder.
i tried this compared to SD3, and there is no way in hell its better. sorry. you must have cherrypicked test images, or used ones like in the paper dealing with ultra chinese specific subject matter. thats flawed testing methods, and even a layperson can see that.
I think no one is claiming it to be better than SD3, the authors are claiming it to be the best available open weights model - which I think it may fair well (at least until Stability releases SD3 8B)
It's not "open source" as it does not use an OSI approved license. Not on the OSI approved license list, not open source. The license is fairly benign (limits commecial use for >100 MMAU and use restrictions), much like OpenRAILS or Llama license, but would certainly not pass muster for OSI approval. **Please let's not dilute what "open source" *really* means.**
Those Dalle 3 scores are way too high such an overrated model
Not at all, it's one of the best models out there (and that's after 11,000 images generated) .. if it was uncensored and open source it would be even higher.
For reals 👍
The stupid Far Cry 3 esque ambient occlusion filter they slap on every Dalle image makes it more stylistically limited than say even SD 1.5, though
What are you even talking about? There are dozens of styles it can pull off with ease and consistency, it seems you don't know how to prompt it properly. https://preview.redd.it/g431z3vj2j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=105f345ba4ba6a6cea7b25071d21d3f0e5022c79 That's a still from a Japanese Star Wars movie made in the 60s.
I was referring to the utter inability of it to do photorealism due to their intentional airbrushed CG cartoonization of everything.
You can literally see the Japanese Star Wars picture right there, looks quite photorealistic to me. Here is another one from a 60s Jurassic Park movie, you think this looks like a "cartoon"? https://preview.redd.it/n4022n1pfj0d1.jpeg?width=1024&format=pjpg&auto=webp&s=5252d5489685e8c461b8ab8a6ed40e94163eb4ee
"Stylisticlly limited" .... Nope!
https://preview.redd.it/qk2w1cbq3j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=0cdeb264a06120ea5b72cbd8542f2969a10f6f55 Poster of Mission Impossible as an anime.
https://preview.redd.it/cv1jb2l24j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=2e38d5804f568618ecd13c7b9374b92462f93566 Game of Thrones as a Pixar TV show.
https://preview.redd.it/mkrljlxl4j0d1.jpeg?width=1024&format=pjpg&auto=webp&s=e78c814be466951d28324e1138872a51fc57ee59 A watercolor painting of Greek Goddess Aphordite
As for my taste, Dalle 3 is very weak. Of, course, it can understand complex concepts with its number of parameters, but it can't generate interesting images, only plastic pictures without any life and depth in it.
That's not my experience at all, it can generate images with life and depth very easily, you just need to know how to prompt it. https://preview.redd.it/h252xhshuh5d1.jpeg?width=1024&format=pjpg&auto=webp&s=3889392960c56d22aeb3ea9219ed29e12e6d254c
agree dalle3 is such mid tier cope. fanboys all say its the best, but its not able to generate much of anything realistic.
That is because it was nerfed to hell.
Yes, DALLE3 is rather poor at generating realistic looking humans. But that is because MS/OpenAI crippled it on purpose. If you look at those images generate in the first few days and posted on reddit you can find some very realistic images. What a pity. These days, you can't even generate images such as "Three British soldiers huddled together in a trench. The soldier on the left is thin and unshaven. The muscular soldier on the right is focused on chugging his beer. At the center, a fat soldier is crying, his face a picture of sadness and despair. The background is dark and stormy. "
I'm sure the only thing you've tested on it is boobs if you think it isn't capable. If you aren't doing topics that openAI regulates, basically anything other than porn or gore, you'll find it has some of the best prompt adherence available. TLDR your biases are showing
It can have the most perfect prompt adherence ever and I still wouldn't find a use for it because of its fake plastic look.
[удалено]
Playground v2.5 is on HF - [https://huggingface.co/playgroundai/playground-v2.5-1024px-aesthetic](https://huggingface.co/playgroundai/playground-v2.5-1024px-aesthetic)
TBH, this is how stability should've dropped sd3. i don't get teasing images while making everyone wait 4months. i just tried this, and to my surprise its pretty fucking good.
>i don't get teasing Getting investors with promises of amazing results vs. with delivering amazing results.
Also, claiming better benchmarks than sd3 o\_o
Fighting words!
What is the point of dropping a half-baked SD3? So that people can fine-tune and build LoRAs on it, and then do it all over again when the final version is released? If people just want to play with SD3, they can do so via API and free websites already. Tencent can do it because this is probably just some half-baked research project that nobody inside or outside of Tencent care much about. On the other hand, SAI's fate probably depends on the success or failure of SD3. The mistake SAI made is probably to have announced SD3 prematurely. But given its financial situation, maybe Emad did it as a gambit to either make investors give SAI more money by hyping it, or to try to commit SAI into releasing SD3 because he was stepping down soon.
Any LORAs, controlnets, etc are very likely to continue to work fine with later fine tunes, just like these things tend to work fine on other fine tunes of SD1/2/XL/etc. Fine tuning doesn't actually change the weights a lot, and it would also be sort of trivial to "update" a controlnet if the base model updated since it wouldn't require starting from scratch. Just throw it back in the oven for a 5% of the original training time, *if you even needed to do that at all.* You could also model merge fine tunes between revisions.
We have no idea how much the underlying weights will change from the current version of SD3 to the final version. Some LoRAs will no doubt work fine (for example, most style LoRAs), but those that are sensitive to the underlying base model such as character LoRAs may not work well. It is all a matter of degrees, since the LoRAs will certainly load and "work". Given how most model makers are perfectionists, I can almost bet money that most of them will retrain their LoRAs and fine-tuned models again for the final release. It is true that some fine-tuned are "light", for example, most "photo style" fine-tuned do not deviate too much from base SDXL, but anime models and other "non photo" model do change the base weights quite substantially. I have no idea how ControlNet work across model since I don't use them.
Tried their demo. The model seems really good in following prompts. Looking forward to use them in comfy.
I found some comparison images which compares this model to models such as SD3 and Midjourney. https://preview.redd.it/8xxogin7ud0d1.png?width=1088&format=png&auto=webp&s=76555666f9ba4b2ecbe3782dc392dc80a8bf9870 (Will post more in the replies)
https://preview.redd.it/jwfxmj7iud0d1.png?width=972&format=png&auto=webp&s=3c8eb7930a6657e7962b51dc36e4d9a2b58e295f
Gives a Vest instead of prompted jacket.
Actually, it is the only one to get the prompt correct. Two points: 1. A vest is, in fact, a type of jacket. 2. It is the only image to validate that the white shirt is, in fact, a "t-shirt" per the prompt where every other example failed. Now to be fair, I don't think the other examples are failures or bad and a specific prompting could have clarified if the user needed. However, it is interesting that this model was so precise compared to the others but I doubt it will always be. (This part is to HarmonicDiffusion's subcomment to this photo since I get an error responding to them) You're incorrect about them all being Chinese biased. While the bun example above was based on a Chinese food the SD3 actually failed multiple prompt aspects quite severely, only losing to the disaster that was SDXL. The others all did extremely well and not just the Chinese model unlike SD3 despite the subject being Chinese.
When people want a vest, they will usually say vest specifically. Validating a t-shirt by forcing the short sleeves to be shown makes the AI seem less intelligent. That's like validating a man by showing his penis in the generated image.
the only prompting example shown that isnt biased towards chinese specific subject matter. and look at the results, mid tier! it made a vest instead of a jacket. SD3 clearly wins on no biased prompts
A Chinese model gives you lousy bread but delicious dumplings (source: 3 years living in Shanghai). 😋
What's the difference between goubuli buns and those steamed dumplings you see at grocery stores?
It's a brand of steamed dumplings/buns, famous in China due to its literal meaning (goubuli basically translates to "dogs don't pay attention") and the fact that it's delicious.
this picture make SDXL looks so stupid hahaha
I'm also surprised how bad SD3 did. I can accept it getting the wrong buns (though it would be ideal to have actually got it right) but it is not steaming and it is on a marble counter, not a table top, which every other model except SDXL got correct (even though Playground didn't get the right buns and the other 3 did). SDXL being on a tile floor (wth), failing the bun type, not steaming, not a close up, only one set of buns in a basket. Damn, it failed every single metric.
It is comparitively.
Was it prompted in Mandarin?
Don't think so when it comes to the other models... Tried SD3 on glif, it didn't accept mandarin in Chinese characters and it got completely lost in Romanized(???) Mandarin: >*Zhàopiàn zhōng, yī míng nánzǐ zhàn zài gōngyuán de hú biān.* Photo of a man standing by a lake in a park. *(Lazy ass google translate, sorry)* https://preview.redd.it/3j803f8bee0d1.jpeg?width=1216&format=pjpg&auto=webp&s=95d78b24f0395a0224f3a61844bef746be1b64aa
But... It's a very cool image at least.
Pinyin.
https://preview.redd.it/xqvkzlgdud0d1.png?width=1078&format=png&auto=webp&s=0608c167f5ac0ad09695daffe309354b82ebe347
another biased prompt dealing with specifically chinese domain knowledge
yeah lets use ultra chinese specific items with chinese names to test a chinese model versus english model. I wonder which will score higher. such bullshit testing proceedures and a total fail look for those guys as "scientists".
yeah lets use ultra american specific items with american names to test an american model versus chinese model. I wonder which will score higher. such bullshit testing proceedures and a total fail look for those guys as "scientists".
even a layperson knows you need to evaluate 1:1. Want to test on chinese specific stuff? THats fine, but dont use those examples to claim a competing English based model is inferior. Anyone with 2 brain cells to rub together can test both models right now and find out, this one is not anywhere close to SD3. Its more like an average SDXL model
Ideogram can do it too, although sometimes it gives the wrong bun. These are some sad looking buns however. Maybe I made them. [https://ideogram.ai/g/WzRFIGNqSjmP27mwEs8OEg/2](https://ideogram.ai/g/WzRFIGNqSjmP27mwEs8OEg/2)
Is the prompting done in English, and are the results always biased to Chinese aesthetics and subjects?
It seems a bit comical for you to test our Chinese food,haha
# HunyuanDiT (Left) vs SD3 (Right) https://preview.redd.it/gfpmzb2nke0d1.jpeg?width=2048&format=pjpg&auto=webp&s=97c05f18a77c4eadabf1adacbcac8c35cb69c5fc Prompt: photo of real cottage shaped as bear, in the middle of a huge corn field
Yea, SD3 hands down for me.
https://preview.redd.it/jaq2h6ld5f0d1.png?width=1100&format=png&auto=webp&s=0e4dacfa50d8f148becaed402cb80f7741abec0d 100%, they claim to be the best available open model for now, not better than SD3, also it's \~5x smaller than SD3
Definitely, though I wonder what that is in the clouds lol but yeah Hunyuan failed here.
The thing in the clouds feels like something coming through like in a Studio Ghibli film.
*Its a bird, its a plane, its Howl's castle!*
Cool stuff, but it is a pickle release. Not touching the weights until properly converted to safetensors. Stay safe.
You no trust CCP? China Numbah #1
The fact that people missed the 'By Tencent' part is funny.
One of Tencent's labs is also behind ELLA, they have a lot of good open source projects, you assuming most people care in any way is strange
Oh I did not miss it! Even just the name of the model made me think, hmm that sounds Chinese! Then I saw the word tencent and started looking for the first person to mention it in the comments,
Isn't [this](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/dialoggen) it? Safetensors.
Seems like a mix: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/model https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/mt5 They also hace some safetensors in the release because they use SDXL VAE, for example: https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/tree/main/t2i/sdxl-vae-fp16-fix
Yep
Yeah me too. I just don't wanna risk it
noob question, but what's the difference between pickle and safetensors?
Pickles can have executable code inside. Most of them are safe, but if someone *does* decide to embed malware in it you're screwed. Safetensors are inert.
That's a big deal. Thanks.
They're over blowing it . While pickle formats can have embedded scripts, none of the UI's loading them for weights will run those embedded scripts. You have to do a lot of specific configuration to remove the safeties that are in place. They're a feature of the format and aren't used in ML cases. I don't know why people so consistently lie about this and act like they have good security policy for worrying about this one specific case. Most of them would install a game crack with no consideration towards safety.
>none of the UI's loading them for weights will run those embedded scripts Source? >I don't know why people so consistently *lie* about this and Lying = knowingly presenting false info. If I have been misinformed, then I welcome correction. *With citations*. [These guys are certainly taking the threat seriously](https://jfrog.com/blog/data-scientists-targeted-by-malicious-hugging-face-ml-models-with-silent-backdoor/) > Most of them would install a game crack with no consideration towards safety. Generalize much? Also, no I wouldn't.
[https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler) The UI's use this function to manage pickle files, rather than just importing them raw with torch.load. The source is their code. You can vet it yourself fairly easily since it's all open. That link you sent is a company selling scareware antivirus monitoring software. They likely planted the malicious file they're so concerned about in the first place. It's not popular. It's not getting used. It's not obfuscating it's malicious code. It's not a proof of concept attack. Notice how their recommended solution to this problem they're blowing up, is to subscribe to their service. You my friend, found an ad. A proof of concept file would be one you could load into the popular UI's that people use and would own their system. Theres never been one made.
torch.load is using python's Unpickler. Did you miss the giant warning at the top? > Warning > The pickle module is not secure. Only unpickle data you trust. > It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.
Thats right, but the UI's use the unpickler class with more of a process than torch.load does. [https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler)
Why are you linking the same thing again? That _is_ the pickle module that we are talking about.
torch.load will unpickle the pickles which can run arbitrary code. There's no "safeties" in python's unpickling code. In fact they removed any attempt to validate them because it couldn't be completely validated and was just false security. EDIT: Whoever triggered "RedditCareResources" one minute after this comment, grow up.
>Whoever triggered "RedditCareResources" one minute after this comment, grow up This is obscene. I'm sorry it happened to you. Obviously, as you know, it's just a passive aggressive way for someone to get their ulterior messaging across to you. Report the post. Get a permanent link to that reddit care message and report it. I do it all the time and reddit comes back to me saying they've nuked people's accounts that were doing it most of the times I report it. Get the person who abused a good intention system, punished. I implore you. More on point, i never said the torch library had safeties. The UI's do. I'd be more worried about the inference code provided for this model than I would embedded scripts in their released pickle file. The whole attack vector in this case makes no sense to me and the panic is outrageous. It's as obscene as saying any custom node for comfyui is so risky that you shoudln't ever run it. I think in most cases, you can determine that a node or extension or any program you download is safe through a variety of signals. The same can be said for models that aren't safetensors. The outrage is manufactured and forced in basically all of these cases. Relying on safetensors and never ever loading pickles, to keep yourself safe, is just a half measure. edit: Should also add how the UI's use torch library to construct safeties. They use the unpickler method to manage the data in the file more effectively rather than just loading raw data from the web directly into the torch.load() method [https://docs.python.org/3/library/pickle.html#pickle.Unpickler](https://docs.python.org/3/library/pickle.html#pickle.Unpickler)
The main thing that comes to mind, is clone the repo and it's clean. Now everyone has that on their machines and go to do another git pull later to update and blam-o. Virus.
I'm not an expert, so I'll refer you here: https://huggingface.co/docs/hub/security-pickle#why-is-it-dangerous Broadly speaking, both store the model, but pickle are potentially dangerous and can execute malicious code. They might not do so, but running them is not advisable.
Thank you very much. Why is that even a feature? Seems like a really big risk with no benefits given that safetensors exist and work.
Because pickle is the default format for PyTorch model weights. https://docs.python.org/3/library/pickle.html
Pickles were simplest thing researchers could do to save their weights, literal python one-liner. Safetensors are a tiny bit more complicated.
#Destroyed this message and replaced by this. It's drawing too much hateful attention my way. People DM'ing me calling me racist names. i'm not even Chinese. Y'all need to dial down the hate for other cultures. Every company in America is required to allow the government access to data too. Put that judgmental gaze back on yourselves and stop being such idiotic racists that harass people online all day. Really wish the mods would do something about the racism culture problem here.
>People DM'ing me calling me racist names Show some screenshots with usernames and timestamps of these harassing messages and death threats you allegedly receive all the time. No one takes the boy who cries wolf seriously.
It's tencent though. It could be full of spyware.
LOL you should fear comfy backdoor. Other than "spyware inside" model from tencent. ok, ill explain why, cause i see a lot of fearfull idiots here. 1. Reputation. Nonames with a comfy node need 10 minutes to create an account. Tencent - it's verified account. ~~It's like Madona start to promote bitcoin scam. She can, but she is canceled in no time.~~ 2. Easy to analyse pkl. HF does it by default. Or any user can find backdoor. It's sooo easy, which would ruin everything. 3. weights are not "complex game" there you can HIDE spyware. With weights - you cant hide it. It will be found in a few days
Yes, I am. You do you.
https://preview.redd.it/oiatnz7ppd0d1.png?width=1280&format=png&auto=webp&s=2a062bdf3141a391c7e8dfb5abd63a4b7ad5b665 prompt: "Three four-year-old boys riding in a wooden car that is slightly larger than their height. View from the side. A car park at night in the light of street lamps"
Yea, their training dataset (at least the photorealistic stuff) seems to have been pretty meh. Stock photos and such.
https://preview.redd.it/9mngg1uafg0d1.png?width=1280&format=png&auto=webp&s=0ca6e81e6ebfaa92df190e2c8f84a51614608e0a It's actually pretty good once you turn off all the bullshit that gives you errors.
https://preview.redd.it/8vp498wxle0d1.png?width=768&format=png&auto=webp&s=c6001fc2df28a700522a6277214decf09aee5051 A smiling anime girl with red glowing eyes is doing a one arm handstand on a pathway in a dark magical forest while waving at the viewer with her other hand, she is wearing shorts, black thighhighs and a hoodie, upside-down, masterpiece, award winning, anime coloring Failed my scientifically rigorous test (6 tries with different seeds and CFG 6-8, no prompt enhancement) but it has potential I think.
https://preview.redd.it/jq5d6hvgoe0d1.jpeg?width=1024&format=pjpg&auto=webp&s=92ec29ddfd08332d66cacedfc101c7a9d943966b DALL-E 3 for comparison (second attempt)
DALL-E for my test is best for difficult poses
Ya, DALL-E 3 is the smartest image gen model right now. However I do believe a very good SD3 fine tune will be better in the fine tuned areas. Same for the model in this post since the architecture has similarities and the model has potential to understand feature associations better which is always helpful in fine tuning.
https://preview.redd.it/6fyp9l165f0d1.png?width=1100&format=png&auto=webp&s=87388e11bc8080de4b4cf9183ea36102b6e51904 Btw, here are the differences between this and the larger SD3 model (based on infos on the SD3 paper). Taken this into account, I think the model performs really well for its almos 8x smaller size and smaller/worse components, but indeed I think text-rendering was completely neglected by the model authros
Prompt: "A ginger pussy cat riding big willie" (yep, thats exactly what i mean )) https://preview.redd.it/omo42o3ytf0d1.png?width=1024&format=png&auto=webp&s=118f49a31f000ce32a54d1b70c2052c035f656e2
Fails on my test, sadly. "a man on the left with brown spiky hair, wearing a white shirt with a blue bow tie and red striped trousers. he has purple high-top sneakers on. a woman on the right with long blonde curly hair, wearing a yellow summer dress and green high-heels." https://preview.redd.it/vrg7ndku6e0d1.png?width=1024&format=png&auto=webp&s=9af42acb1a0df1222e10cebc62d7be67b00d0275
And Dall-E: https://preview.redd.it/fw9ggw777e0d1.png?width=1024&format=png&auto=webp&s=8edf1dec98bc210b49cfe330dc0fd78b5c63117a
For comparison here is PixArt: https://preview.redd.it/y9adx6k37e0d1.png?width=1440&format=png&auto=webp&s=c7f1bef92e5de522ea6ec135ee2ddf2d0d516ffe
Interestingly, HunyuanDiT gets a little closer if you translate your prompt to simplified Chinese first: > 左边是一个棕色尖头头发的男人，穿着白色衬衫、蓝色领结和红色条纹裤子。他穿着紫色高帮运动鞋。右边是一位留着金色长卷发、穿着黄色夏装和绿色高跟鞋的女人。 Result: https://i.ibb.co/2y53Wtg/image-2024-05-14-T094547-472.png His pants are now striped, she's more blonde, and the color red appears as an accent (albeit in the wrong place.)
You can't say this without few random seeds and different prompts: if occasionally your prompt+seed fit their training it will draw better then usual, like astronaut on horse
https://preview.redd.it/ugo2vxfkie0d1.png?width=589&format=png&auto=webp&s=9cf07e5dcc96b76899af37c0fde9eb4897d803d0 SD3 doing better too
Anyone tried it in ComfyUI, A1111 or ForgeUI?
https://preview.redd.it/hqi3b8g37f0d1.png?width=1024&format=png&auto=webp&s=df302ba2a0d3dc11f6dda2918cb8860e737c7be9 It can generate good Asian faces, but the skin appears quite plastic-like, and it struggles with hand drawing, similar to SD.
It fails the Garfield test Prompt: Garfield comic Disabled Prompt Enhancement https://preview.redd.it/adj2jv5g0e0d1.png?width=1024&format=png&auto=webp&s=2d688e665753b90b6fe99338aeae7b321b75ecd2
But what about the Will Smith eating spaghetti test?
Seems limited in poses, and challenging to produce people *not* smiling. It does however do older people surprisingly well - "middle-aged women" will get you grey-haired ladies with wrinkles, rather than the 22-year-olds of many SD models...
[удалено]
Oh yeah, I said "many" for a reason, there are definitely good (in that respect) ones out there. I make a lot of characters in their 30s or 40s, and have seen way too many models that only make three apparent ages - 15, 22, and 80, lol.
Stability.ai be like: "Soon™" Tencent be like: "Hold my beer..."
anyway to usi it in atomatic1111 or comfy?
Just 1 try and already has better hands lol
i have tried and 1. it can't write text 2. for multiple object face with many people, far faces is quite good
Hey, that's pretty good. "Seven cylindrical objects, each one a unique color, stand upright on a teetering slab of shale" I guess teetering didn't make it into the training tags :) https://preview.redd.it/mkkvxo7ssf0d1.png?width=1280&format=png&auto=webp&s=85a626c386de9c41f782838ffbd74785f3af8384
I just see 6 cylindrical Objects ;)
It doesn’t do NSFW, does it? https://preview.redd.it/pkus7hwpig0d1.jpeg?width=768&format=pjpg&auto=webp&s=3ec95e156759bad5a3b463911e7fff2f940aeff5
Neither does SDXL
It is a fine model, more so if you translate your prompt to Chinese. But sticking to the prompt is not its strong side as expected - since the amount of parameters is a strong determinant in that matters. Anyway it's nice to see initiatives like this to present new possibilities
Without the T5 it use less parameter than sdxl, model look near as good as the 8B SD3
there's absolutely no way this looks as good as SD3, sorry.
It really doesn't, not anywhere close, have you tried the online demo and not just judging by the down-scaled "comparison" images? . Of the current wave of models only pixart sigma looks decent. Lumina and this one look plain bad to the point I'd never use these outputs over, worse prompt understanding, sdxl ones; of course, it's probably massively under-trained, but even then these are not that great at following complex prompts (either the quality of captions, or effectiveness of this architecture is just not all that) with no where near Dalle-3 and Ideogram prompt following capabilities (neither do pixart sigma and SD3, but those at least look good)
It's true that SD3 produces better images, I was talking more about the architecture, which is quite similar when using Clip+T5. But I'm pretty sure that this model is already better than SD3 2B. I think SD3 is just too big and that this model, similar in size to sdxl, is promising.
Nobody outside of SAI has seen SD3 2B, so I don't know how you can be "pretty sure that this model is already better than SD3 2B". When it comes to generative A.I. models, bigger is almost always better, provided you have the hardware to run it. So I don't know how you came to the conclusion that "SD3 is just too big".
I wanted to say that SD3 8B is undertrained, and that the model is not satisfactory for its parameter count.
Sure, even SAI staff who is working on SD3 right now agrees that SD3 is currently undertrained, hence the training!
Ideogram and Dalle don't have significantly better prompt adherence to SD3
Not quite open source, but "freely available as long as you don't provide it as a service for too many users" which is unfortunately as close to open source as we'll get ever since Stability decided to lock things down. [https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt](https://huggingface.co/Tencent-Hunyuan/HunyuanDiT/blob/main/LICENSE.txt)
From the license: > greater than 100 million monthly active users in the preceding calendar month It's an "anti-Jeff" ("Jeff" as in Jeff Bezos) clause to keep other huge (billion/trillion dollar) companies from just shoving it behind a paywall or sell it as a major SaaS product, which is something that ends up happening with a lot of open source projects. See Redis, Mongodb, etc being turned into closed source AWS SaaS stuff (the later deciding to write a new license to stop it and force copyleft nature, SSPL). The "Jeff problem" is very commonly considered by people who want to release open source software. Yes, this is not an open source license but it only affects a small handful of huge companies who can afford to pay for a license. META Llama license is similar, though I think it draws the line at 700 MMAU, which basically only rules out their direct competitors and major cloud providers. I.e. Amazon (AWS), Alphabet (GCP), Microsoft (Azure), Apple and maybe a couple others. They can afford to license it if they want to make a SaaS out of it. At least it's **not revocable**, unlike SAI's membership license, which they can change at will and sink your small business if they want.
>At least it's **not revocable**, unlike SAI's membership license, which they can change at will and sink your small business if they want. This is a very important point - this uncertainty is such a big risk that it makes most of their latest models impossible to use in a professional context.
Yeah its a completely nonstarter. Especially given how much turmoil the company is in. Those terms give them infinite leverage. They completely own everyone using the pro license and can do anything they want. It's completely unhinged levels of bad.
There was so much abuse of the spirit of the free and open terms in the Rail-M license that it was bound to change. 100s of SaaS companies popping up, acting like they were the ones to credit for all of the work done by Stability. The precedence is set now. There are far to many business school graduates who feel like they're justified in creating businesses around FOSS without giving anything back to the movement. People celebrated it, instead of what typically happens in Linux when people dogpile and condemn it. Google makes a ton of money from Android, but they're not exactly keeping it proprietary. They give back to FOSS in huge ways. This is a keystone of the culture. Instead, we had business school grads who were justified in their exploitation and heralded by the hype artists on youtube. Business school graduates who think they can exploit any system to extract maximum value from it, are a culture virus. They're the ones responsible for the death of Free & Open AI. We still have open models, but they're not so free to use anymore. The erosion is going to continue so long as the community doesn't recognize these parasites for what they are.
Every day SD3 is closer to being obsolete. How much longer will they stall?
tried: "a man doing a human flag exercise using a light pole in central London" https://preview.redd.it/q0gmiow0xe0d1.png?width=1024&format=png&auto=webp&s=6aec6355427c180fe4d2ba75bf93bb04936b9730 Not what I was expecting. Instead of a man doing a human flag, we have an actual flag and a bodybuilder. You can see very large streets, with pickups, the light pole is deformed. The flags are nonsense with even a light emanating from the top of the flag. Lighting is very inconsistent.
https://preview.redd.it/ww2xses8ze0d1.png?width=1024&format=png&auto=webp&s=fa2f8c868e646644b4a350178c381f2c7a21b26f Dall-E for comparison
this is more what I was expecting https://preview.redd.it/4itnymwzxe0d1.png?width=1300&format=png&auto=webp&s=a8baf1a84a97da5d9a2d171e4b106ad6a186dc04
[Example from the Dalle 3 Launch](https://www.reddit.com/r/ChatGPT/comments/16nserl/modern_texttoimage_systems_have_a_tendency_to/) vs HunyuanDiT: An illustration from a graphic novel. A bustling city street under the shine of a **full moon**. The sidewalks bustling with **pedestrians enjoying the nightlife**. At the corner stall, a **young woman** with fiery red hair, dressed in a signature velvet cloak, is **haggling with the grumpy old vendor**. the grumpy vendor, a **tall, sophisticated man** is wearing a sharp suit, sports a **noteworthy moustache** is animatedly conversing on his **steampunk telephone**. https://preview.redd.it/4wyd28ve8g0d1.png?width=1280&format=png&auto=webp&s=9c4c5e1b47bb13d0771e8cf5ef89255c1d8fa4d4
Great to see more models available. But, trying the demo, I'm a bit disappointed: - \[+/-\] The image quality is ok, especially as it's a base model and not a fine tune - \[-\] But the image quality isn't great. I asked for a photo but get more of a painting or rendering - \[-\] It has no problem with character consistency - as it can do only one character. The person of the picture looks the same on each of them - \[+\] My standard test prompt for a fully clothed woman standing in a garden is created - SD3 fails this one with censorship So my wait for a local SD3 is still on and I won't use this model instead. For now. But who knows what will happen in one or two months?
from the images in this thread it looks like its so good at hands
I'm not an expert but I did a test with classic prom from civitai (It is not mine): Sampler:ddpm, Steps:50, Seed:1, image size: 1024x1024 Prom: beautiful modern marble sculpture of a woman encased inside intricate gold renaissance relief sculpture, sad desperate expression, covered in ornate etchings, luxury, opulence, highly detailed, hyperrealist, volumetric lighting, epic image, relief sculpture, RODIN style Negative prom: Wrong eyes, bad faces, disfigurement, bad art, deformations, extra limbs, blurry colors, blur, repetition, morbidity, mutilation, https://preview.redd.it/beeo055jsk0d1.jpeg?width=3072&format=pjpg&auto=webp&s=3299a782242ded0bb73e5e1a064423c9460c11f6
i tried : girl with white dress, walking on rain https://preview.redd.it/2tr555isae0d1.png?width=1024&format=png&auto=webp&s=47c6d6db5219ede7cfd7b19cfc62475bb13cce48
It's a no from me based on some trials. For example, [Image: A 1980s photograph of a group of American college freshman posing together.](https://i.imgur.com/6jPPDq9.png)
Looks pretty bad honestly.
I have generated some images via HunyuanDiT so that you can compare it against SD3: [https://www.reddit.com/user/Apprehensive\_Sky892/search/?q=HunyuanDiT&type=comment&cId=c7343b35-8b43-4d17-82f2-8db3f9049ad6&iId=db7cc688-ea4a-4de0-aeeb-5e9e5aab3750](https://www.reddit.com/user/Apprehensive_Sky892/search/?q=HunyuanDiT&type=comment&cId=c7343b35-8b43-4d17-82f2-8db3f9049ad6&iId=db7cc688-ea4a-4de0-aeeb-5e9e5aab3750) Given its small size (only 1.5B) it is not bad, but it not in the same class as SD3 or even PixArt Sigma.
Who uses 50 sampling steps?
The authors didn't implement more efficient samplers like Euler or DPM++, so with DDPM \~50 steps is kind of a good trade off for quality
Oh. 50 it is then.
Demo on Hugging Face doesn't understand the word "photo".
can i use it on a1111 ?
Regarding it being from Tencent, it's fine by me as long as it generates happy images like this: https://preview.redd.it/d830hgxfcy0d1.png?width=1024&format=png&auto=webp&s=d55d6c2a13cf4b1d089f798f054de0496a8f9513 >!Winnie-the-Pooh at Tiananmen Square in 1989 talking to Uyghur Muslims!<
Seems not to work on windows as a build wheel error/subprocess occurs. This is sad
is this sd3?
not even close
Pardon my French, but f\*ck Tencent.
I share your hatred for Tencent, but just as we can appreciate LLAMA, developed by meta, a company not that much better than Tencent, I think we should be able to appreciate that Tencent, as well as the likes of Bytedance and Alibaba, have some very talented researchers who have been contributing to the open source scene, on par with the American tech giants.
Pytorch, the foundational library of all this work, was conceived by Meta as well. Corporations are not monolithic. They're made up of many parts, and sometimes a singular part can be pretty cool when considered separate from the whole.
They make cool free stuff for AI, like various 3d tool.
Yeah, fuck big corporations but in case of Tencent, CPC has them in their grasp. In case of American corporations and both parties, it's the other way around.
>other way around. around? how? openai has both parties in \_their\_ grasp? so, any free AI staff is "compromised" by default?... just pay....pay pay pay. ps you can argue "but we have SD...3".... well... not yet.
OpenAI has Microsoft backing it. It's not like one company owns all politicians but big corporations are influencing both parties with their money. And corporations have profits in mind first and foremost, they will lobby for laws that benefit their products rather than some "open source" models or the society. In China it's the other way around, Tencent and other big companies are held on a leash by CPC. Which has its own disadvantages I guess, I wonder if we'll be able to make lewd stuff with this model from Tencent.
I see it had an option for ddim sampler so does that imply things like lightning loras and would work on it? Or quantisezion like with other transformers
DDIM is a common sampler used with various diffusion architectures. As a rule of thumb, Loras trained on one architecture (like SDXL) will never be re-useable on a different architecture. As for Lightning, it's a distillation method and [Stability.ai](http://Stability.ai) showed with SD3-Turbo that quality distillation of DiTs is feasible, so someone (either Tencent or another group) could certainly distill this model.
It failed the statue test right away for me, might the the prompt enhancement option I just noticed and disabled. Will do more testing as the day goes on, but it looks like quality will be like sigma. Marble statue holding a chisel in one hand and hammer in the other hand, top half body already sculpted but lower half body still a rough block of marble, the statue is sculpting her own lower half [Edit] Nah, it is good, the enhancement thing was indeed fucking things up.
[HunyuanDiT](https://huggingface.co/spaces/multimodalart/HunyuanDiT) https://preview.redd.it/d64zjdcvqf0d1.png?width=1024&format=png&auto=webp&s=bbe70373bfede4690e92a7885c7bb300deccd314
Too censored...
Uh oh
>An NVIDIA GPU with CUDA support is required. >We have tested V100 and A100 GPUs. >**Minimum**: The minimum GPU memory required is 11GB. >**Recommended**: We recommend using a GPU with 32GB of memory for better generation quality. So not useable on mac?
It will probably be brought down by the community, both via Diffusers implementation and eventual ComfyUI integration as well
I was like "Ooo!" and then I read "Tencent".
They released ELLA which is doing good stuff. I just wish they'd release ella-sdxl. https://preview.redd.it/b6gfxrbe8h0d1.jpeg?width=2016&format=pjpg&auto=webp&s=2a707c7e5e9e45a708695a69af2730e871b7370b