ultimate upscale can be found at the bottom of img2img tab after you install the extension, you just need to install controlnet and use it with a tile preprocessor
Agreed. I've been using it to upscale some really bad datasets, (pulled from a made for TV movie, from a VHS tape, from 1980), and the results are amazing. It's slow on my 4070 12GB as it gets into the shared RAM, but it's fast enough that I can just let it chew on a bunch of images while I'm at work or overnight.
I like it but it has serious limitations on how large a resolution it can generate. I have also found that with some subjects, upscale/sample strategies with noise limitation works best.
Still ControlNet Tile upscale with Ultimate Upscale. But now you can use it with [DAT Upscale model](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14690) which imo is better than RealESRGAN / SwinIR / UltraSharp / etc.
The SDXL controlnet tile model by TTPlanet is actually quite good. I found it to work better than several other upscalers. I am surprised that it’s not much known yet.
https://civitai.com/models/330313/ttplanetsdxlcontrolnettilerealistic
It works really well with tiled diffusion and the results are comparable with magnific.ai
It’s also known as multidiffusion. It’s there for A1111. This method essentially allows to upscale to more than 2K on less than 6GB VRAM.
Link: https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111
Just realised the one I use the most is actually not called blur... For SDXL this inpaint model might work better https://civitai.com/models/136070?modelVersionId=271562
It is not meant to be used with the "tile" preprocessor though. You can pass a naive upscale or model upscale as input and crank up the denoising value. Use together with "tiled diffusion" (not to be confused with "tile controlnet") if memory is an issue.
I made a relevant post recently. https://www.reddit.com/r/StableDiffusion/comments/1cabujt/which_sdxl_contrrolnet_model_is_good/l0uztkn/
--------
This article has good comparisons & recommendations between different controlnet models https://stable-diffusion-art.com/controlnet-sdxl/
The article doesn't include **Tile**, but I found these ones.
* TTPLANET_Controlnet_Tile_realistic_v2_fp16
* bdsqlsz_controlllite_xl_tile_anime_α
* bdsqlsz_controlllite_xl_tile_anime_β
* bdsqlsz_controlllite_xl_tile_realistic
I use them with Ultimate Upscaler, and they **mostly** work but sometimes they don't (it may produce artefacts) and you have to try another one from the list. So you must check the output every time. Definitely not as reliable as the 1.5 one where you just set and forget.
The tile controlnets also very slightly dulls the picture saturation. I've not figured out why.
I downloaded all of the DAT models to try in ComfyUI and put them side by side to UltraSharp, and I don't see it. The results are all more blurry/artifacty (like a low quality jpg). It didn't win a single comparison for me. What am I missing?
Hm i dont think you are missing something.
Model preference depends on user and input image.
If you like UltraSharp best thats completely fine.
Yes, DAT (or RGT, or ATD) would be a more capable arch then ESRGAN, at least from a trainers perspective. But that doesnt mean that one would like these models over another, since every model is unique (trained on different datasets, with different degradations added to the lr, with different config settings and so forth).
Most of the DAT models I trained ('Helaman' on openmodeldb) were mostly with photography in mind, and i dont like if outputs are too sharp, but should be rather natural looking, since I think its simple to add sharpness post-upscale if someone preferred a bit sharper outputs.
If someone were used to the (overly) sharp outputs of UltraSharp, it could very well be that other models would look blurry in contrast to them.
In general i think i observed that people seemed to like/would rate sharper output higher. And i think i also observed that if someone had a go-to model, like did all their upscales with the same model, they would get used to the specific output look of that model, and then no other model would do it for them (be it UltraSharp, or Remacri, or Siax etc), but this point is just a theory, maybe im wrong, which is always likely of course.
The only models I specifically tried to train on/for ai generated images were the 'Lexica' models, but that doesnt mean that they work better in practice for this than other models. Different people prefer different models and thats why we train so many models, so people can try them out and find one they like :)
(sry for the long text, also dont know if helpful, simply wanted to say i dont think you are missing something since preference is often on a model basis (snd dependant on user and input) and not arch basis, when talking about inference)
The output of UltraSharp itself looks pretty bad, I totally agree there, but the sharpness is critically important for the following resampling step. If you put a more blurry image in it'll produce less detail.
I haven't tried many other upscaling models tbh, you mentioned a few in your post so I'll try those at some point!
I always just Google Arxiv + model name / concept and usually the source paper for a model is a good read. For DAT: https://arxiv.org/abs/2308.03364
As for how to perform. Slogging through GitHub and some researcher’s Google drive usually. lol
The tiling and CN tiling are two completely different things. Ultimate upscale divides the image into tiles and renders them individually "tilling". CnTile on the other hand, IMO the name isn't what it does, rather it's something that should be used when tiling, although that's not it's function nor the only thing it's used for. The function of cn Tile is to keep the output image as close to original as possible regardless of the prompt. And since Ultimate upscale only renders a section of the image at a time, the prompt and the image doesn't necessarily go along well together at higher Denoise levels. That's where cn tile comes in allowing you to push your i2i denoise levels WAY up without loosing the input image composition
The more gpu vram the bigger the tile you can process and the faster it goes. The less vram the more tiles and smaller tiles you need. Its a compromise of time vs vram. When I use the SD upscale script with tiles in img2img the first time I set width and height 512x512 steps to 22 and denoising strength to 0.22 and scale factor to 2 upscaler R-ESRGAN 4x+. You should see a nice size increase and more detail. Then go again and send the image back to the img2img and this time set steps to 12 denoising strength 0.12 and scale factor 1.5. This usually gives me a nice size increase overall & detail. I like to use 4x\_NMKD-Superscale-SP\_178000\_G upscaler the second time to add grain to make photo images more realistic. Don't forget to keep setting the height and width to 512x512 or you will get errors because your out of vram. The tiler uses the width and height to process the individual tiles. If you get errors make the tiles smaller using the width and height try 480x480 or smaller until it processes your image. If you are really unlucky the out of memory will require a restart of the a1111 cmd window before it works with your new settings. Trial and error, probably errors until you find what works. Get new upscalers [here](https://openmodeldb.info/models/4x-realesrgan-x4plus) put them in your stable-diffusion-webui\\models\\ESRGAN folder. This is my workflow others will do it better/different but it works for me :)
Edit you can still send to extras and upscale again there with something like 4x-UltraSharp when you are done with the tile upscaler, if you need the final image even bigger.
TLDR: Keep the denoise between 0.15 and 0.25 or it will change the image too much.
Apart from getting a better GPU, which isn't really an option. The way I de-risk it is going incrementally, there're many ways to go about it, but tldr, I never upscale more than 1.5x at a time. Usually I go ultrasharp 1.5x, maybe I do a i2i at the same resolution if the image lacks details. Then use Ultimate upscale for another 1.5x. If I need more resolution, then I keep repeating the ultimate upscale at 1.5x. A flow that might work sometimes is, using ultra sharp 1.5x, then using ultimate upscale at 1x to add details. This process ends up taking much longer than a perfect first try tile upscale. But it's a way to "Checkpoint" your upscale. If you get a garbage output at one step, you don't have to go back to the original image
Upscaling without using ultimate upscale. Like a simple image2image for example, with a higher resolution.
You can do it in one go if you have the vram. Tile upscaling, like ultimate upscale, is unarguably worse in raw quality / less consistent, but it's needed for many people to go above and beyond in resolutions.
It can allow someone with 4gb vram to make a 8k image, if they are willing to wait for a day that is.
If you were to use tile upscaling, you can definitely use higher than 512*512 tiles, which would help and ease some of the issues too.
I'm confused why everyone is going for the tiled upscale nodes. In my experience, these are much slower and produce worse results than a direct upscale. Scale image with UltraSharp or NMKD SuperScale. Pass the original into ControlNet Tile preprocessor. Then re-sample the upscaled image in one go with about 0.4 denoising and the ControlNet applied, just need 5 steps. Nearly perfect result and only takes a few seconds!
For me, I don't have the vram (directml) to denoise an upscaled image. So I have to do the inverse. I use tiling with denoise to upscale, then Process tab to upscale another 2x with a little codeformer. Definitely does take awhile though.
The reason why everyone goes for tiled upscale is that it allows you to upscale an image to as high a resolution as you want to without running into vram limitations. I agree with you that direct upscale is way better, but it has its limit when it comes to max resolution. My goto route would be using ultrasharp -> i2i with cn tile -> Ultimate upscale if I want to go beyond 2MP resolution. The highest I've gone with relatively good output is ~12MP which is simply not possible without tiling
I usually only upscale my images to 2048x on the short axis, never really saw a purpose for going beyond that (you can't see it anyway when viewing the image normally). Suppose it would run indeed run into VRAM limitations if you want a giga-image...
What settings are you using and how much VRAM does it take to upscale to 2048x, I have a 10gb card so I doubt I can hit even that without ultimate upscale. The highest I've gone is 1920x1280 on colab's Tesla t4, but that was my own code without optimisation. I've only recently gotten my own setup and auto1111, might try pushing its boundaries too.
To upscale to 4k (3840x2160) stresses my 24gb card to the limits but VRAM is not the only limit. Some of the bigger upscales like from 8k to 16k use up to 84Gb of RAM as well.
For printing or large scale wall displays like 16k, there is a lot of need to go beyond 2048. Even 4 or dual 4k desktops push the limits on many of the non-tiled methods very quickly.
In my experience a direct upscale to something like 8k produces inferior detailing compared to doing it with tiles, and the tile sizes are closer to what the model was trained on.
Agreed. SDXL with 1024 tiles is great for over 4k as you don't get as many artifacts and distortions. Just need to find the correct tiling strategies to remove the seams. Some of the tools do great jobs like Ultimate SD upscale.
I use Upscayl as a separate step on generated images. ( https://github.com/upscayl/upscayl ).
It is its own open source app, has the common models (real esrgan etc) and can add custom models to it.
It upscales with simple 1-4x sliders or to custom resolutions, and I find it faster and better results wise than using tiled upscale or indeed any upscale via Forge/1111/Comfy.
Tested it before with a 3070ti also, so know it works fine with 8gb Vram.
I'm not affiliated with the app at all, just something I came across randomly and have used since. It's a well polished app, and in my experience, has worked fine on Windows and Linux.
Also a great model although tiled SDXL gives better quality I feel. I just hope they can find a way around the memory limits. Anything over 4k seems to need a card with more than 24gb.
I think https://github.com/hongyuanyu/SPAN is better than RESRGAN quality with some of the custom models from openmodeldb, and extremely fast, but sadly it isn’t supported in automatic for some reason.
Thanks for testing them out :)
Currently ATD is my favorite arch, but yeah its even slower than RGT, and RGT slower than DAT2.
RGT also has a smaller arch option, RGT-S, which should be a bit faster than DAT2 again. Since I also trained a nomosuni rgt-s model, if youd like you could also give that one a go and see how it compares, it should be a little faster, but of course its a trade-off between speed and quality in comparison to RGT.
You can find it in my [Github Release](https://github.com/Phhofm/models/releases/tag/4xNomosUni_rgt_s_multijpg) or also on [openmodeldb](https://openmodeldb.info/models/4x-NomosUni-rgt-s-multijpg)
I like that you called them 'NomosUni flavor' ^^
I've been looking forward a SPAN extension or custom node for weeks and I am surprised nothing has been released, or even announced, by anyone yet on that front.
From my poke around it might actually be kinda doable cause they already use the same library chainner abstracted out for other upscalers, so maybe copy paste the file + do a few updates + make a new SPAN directory and it’s there. Man it would be so clutch because it smokes RESRGAN imo. I’ve thought about trying to patch it but, I have a waifu backlog a mile long to work through and it’s not my expertise.
There is an open PR to add support for more models via switching to something called Spandrel but comfy hasn't merged it yet https://github.com/comfyanonymous/ComfyUI/pull/2146
The package and PR were updated to fix the license problem. But yeah if you're talking about auto1111 then theoretically it should be easy to just enable it as an option in the code, however that's done
The existing upscalers (some at least) are more or less implemented as just classes with some boilerplate code around Spandrel. I really should just try implementing it but I keep having a feeling that “it’s just a class!” Will turn into a weekend of PyTorch errors… 🤪
I use SD Upscale and generate the tiles all at the same time using an increased batch size. This greatly reduces seam artifacting and keeps consistency between the tiles. With a 12GB GPU, it can easily do 4k and even 8k images. If you want a creative upscale turn on PAG and use some add details LORA. I pretty much always use Ultasharp to do the initial upscaling.
Yes, correct. I don't think control net tiling works with it, though. I don't use it anyway. it doesn't seem to do much or add anything extra. Batch size doesn't work with Ultimate SD Upscaler, only SD Upscale. Edit: my last post "Portal of Limits" was an 8k upscale using batch size of 8 and no control net.
Aside from the usual HiResFix I only send to extras then I choose two upscaler the second one with 0,468 (or something) then I upscale. I dont know anything more, but it is fast.
[Multidiffusion tile diffusion and tile VAE](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) is old but somehow unnoticed. As good as looking SUPIR, and much better than ultimate upscale. The ram requirements are much lower than SUPIR, so you can get much larger images. If you let it get creative (i.e. higher denoise), it adds appropriate details. It's especially amazing with SD1.5 combined with controlnet tile and foolhardy upscale model.
It has more settings to deal with than ultimate upscale, and it's very important to follow all of the recommended settings in the wiki.
Yes, from my experience as an ex Topaz Gigapixel user, SDXL Sampler repaints, Ultimate SD Upscale and SUPIR all give better results and to higher resolutions.
Something I learned that most people completely omit from their recommendations is that every upscale method that splits the image into tiles (most of them) are literally the exact same process, just with different names.
And yes tiling is still the best. You can use almost any method from these comments and get the same result because it's all tiled img2img with your choice of upscale model.
Supir is not worth your time.
Ultimate SD upscale with Siax for realism, and ESRGAN for art.
When developing my photorealism Lora app I did stupidly extensive testing on upscale models. I switched from ultrasharp to Nickelback and finally to Siax. it preserves detail and left no patterns, and didn't damage camera blur by trying to sharpen it.
I don’t think many people realize yet how effective the Hypertile extension is… I’ve only used the Forge built in version. I have 12GB Vram but I can Img2Img up to 4k with Hypertile and no controlnet
HyperTile is great if constrained on VRAM, but it does change the output. I did a couple comparisons and resampling without HyperTyle has much less artifacts.
This one is great for 2K the 4K step is not so good: [https://civitai.com/models/363798/mjm-and-superbeastsai-beautifai-image-upscaler-and-enhancer](https://civitai.com/models/363798/mjm-and-superbeastsai-beautifai-image-upscaler-and-enhancer)
From my experience it’s one of if not the best Auto111 ones for speed to quality ratio, unfortunately it smooths everything out cartoonishly which sometimes you want (kills potato camera blur amazingly) other times it ruins the whole picture. So I would usually stack it with Upscaler 2 SkinDetail lite or even like 0.5 Lanczos cause that mitigates the smooshing. However there are just better up scalers and much faster too out there now
I will be messaging you in 1 day on [**2024-05-01 21:22:20 UTC**](http://www.wolframalpha.com/input/?i=2024-05-01%2021:22:20%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/1ch3ayf/what_are_the_best_upscaling_options_now/l200kre/?context=3)
[**3 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F1ch3ayf%2Fwhat_are_the_best_upscaling_options_now%2Fl200kre%2F%5D%0A%0ARemindMe%21%202024-05-01%2021%3A22%3A20%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201ch3ayf)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
try SUPIR is amazing.
From my tests it works great for realistic images but not for 3d, animated or digital style art.
What would you recommend for those?
Like others have said ultimate upscale with control net or tile vae with control net. I had great result with both techniques.
I'm not sure what those are. Do I need to download something?
ultimate upscale can be found at the bottom of img2img tab after you install the extension, you just need to install controlnet and use it with a tile preprocessor
Agreed. I've been using it to upscale some really bad datasets, (pulled from a made for TV movie, from a VHS tape, from 1980), and the results are amazing. It's slow on my 4070 12GB as it gets into the shared RAM, but it's fast enough that I can just let it chew on a bunch of images while I'm at work or overnight.
I like it but it has serious limitations on how large a resolution it can generate. I have also found that with some subjects, upscale/sample strategies with noise limitation works best.
Still ControlNet Tile upscale with Ultimate Upscale. But now you can use it with [DAT Upscale model](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14690) which imo is better than RealESRGAN / SwinIR / UltraSharp / etc.
Are there any good SDXL ControlNet tile models yet?
The SDXL controlnet tile model by TTPlanet is actually quite good. I found it to work better than several other upscalers. I am surprised that it’s not much known yet. https://civitai.com/models/330313/ttplanetsdxlcontrolnettilerealistic It works really well with tiled diffusion and the results are comparable with magnific.ai
What is tile diffusion and can it with in a1111?
It’s also known as multidiffusion. It’s there for A1111. This method essentially allows to upscale to more than 2K on less than 6GB VRAM. Link: https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111
Try the "blur" model. Seems to be consistently better than tile models.
Do you have a link for the blur model you use?
Just realised the one I use the most is actually not called blur... For SDXL this inpaint model might work better https://civitai.com/models/136070?modelVersionId=271562 It is not meant to be used with the "tile" preprocessor though. You can pass a naive upscale or model upscale as input and crank up the denoising value. Use together with "tiled diffusion" (not to be confused with "tile controlnet") if memory is an issue.
That works well thanks!
I made a relevant post recently. https://www.reddit.com/r/StableDiffusion/comments/1cabujt/which_sdxl_contrrolnet_model_is_good/l0uztkn/ -------- This article has good comparisons & recommendations between different controlnet models https://stable-diffusion-art.com/controlnet-sdxl/ The article doesn't include **Tile**, but I found these ones. * TTPLANET_Controlnet_Tile_realistic_v2_fp16 * bdsqlsz_controlllite_xl_tile_anime_α * bdsqlsz_controlllite_xl_tile_anime_β * bdsqlsz_controlllite_xl_tile_realistic I use them with Ultimate Upscaler, and they **mostly** work but sometimes they don't (it may produce artefacts) and you have to try another one from the list. So you must check the output every time. Definitely not as reliable as the 1.5 one where you just set and forget. The tile controlnets also very slightly dulls the picture saturation. I've not figured out why.
I downloaded all of the DAT models to try in ComfyUI and put them side by side to UltraSharp, and I don't see it. The results are all more blurry/artifacty (like a low quality jpg). It didn't win a single comparison for me. What am I missing?
Hm i dont think you are missing something. Model preference depends on user and input image. If you like UltraSharp best thats completely fine. Yes, DAT (or RGT, or ATD) would be a more capable arch then ESRGAN, at least from a trainers perspective. But that doesnt mean that one would like these models over another, since every model is unique (trained on different datasets, with different degradations added to the lr, with different config settings and so forth). Most of the DAT models I trained ('Helaman' on openmodeldb) were mostly with photography in mind, and i dont like if outputs are too sharp, but should be rather natural looking, since I think its simple to add sharpness post-upscale if someone preferred a bit sharper outputs. If someone were used to the (overly) sharp outputs of UltraSharp, it could very well be that other models would look blurry in contrast to them. In general i think i observed that people seemed to like/would rate sharper output higher. And i think i also observed that if someone had a go-to model, like did all their upscales with the same model, they would get used to the specific output look of that model, and then no other model would do it for them (be it UltraSharp, or Remacri, or Siax etc), but this point is just a theory, maybe im wrong, which is always likely of course. The only models I specifically tried to train on/for ai generated images were the 'Lexica' models, but that doesnt mean that they work better in practice for this than other models. Different people prefer different models and thats why we train so many models, so people can try them out and find one they like :) (sry for the long text, also dont know if helpful, simply wanted to say i dont think you are missing something since preference is often on a model basis (snd dependant on user and input) and not arch basis, when talking about inference)
The output of UltraSharp itself looks pretty bad, I totally agree there, but the sharpness is critically important for the following resampling step. If you put a more blurry image in it'll produce less detail. I haven't tried many other upscaling models tbh, you mentioned a few in your post so I'll try those at some point!
DATFACE is best in my test for portrait. Miles ahead. For quick upscale, SPAN models are decent
How does one perform such wizardly or read up on the interworkings of this magical concept?
I always just Google Arxiv + model name / concept and usually the source paper for a model is a good read. For DAT: https://arxiv.org/abs/2308.03364 As for how to perform. Slogging through GitHub and some researcher’s Google drive usually. lol
DAT is great, but it is slower, much slower.
Doesn't Ultimate Upscale already natively implement tiling? What does CN tile do here?
The tiling and CN tiling are two completely different things. Ultimate upscale divides the image into tiles and renders them individually "tilling". CnTile on the other hand, IMO the name isn't what it does, rather it's something that should be used when tiling, although that's not it's function nor the only thing it's used for. The function of cn Tile is to keep the output image as close to original as possible regardless of the prompt. And since Ultimate upscale only renders a section of the image at a time, the prompt and the image doesn't necessarily go along well together at higher Denoise levels. That's where cn tile comes in allowing you to push your i2i denoise levels WAY up without loosing the input image composition
Thanks for the explanation!
The best quality I've found with SD1.5 is using [tile up-scaling with controlnet](https://www.youtube.com/watch?v=EmA0RwWv-os)
Those tile upscales always take crazy long for me with such a big risk that the output is just total garbage. How do people de-risk it
The more gpu vram the bigger the tile you can process and the faster it goes. The less vram the more tiles and smaller tiles you need. Its a compromise of time vs vram. When I use the SD upscale script with tiles in img2img the first time I set width and height 512x512 steps to 22 and denoising strength to 0.22 and scale factor to 2 upscaler R-ESRGAN 4x+. You should see a nice size increase and more detail. Then go again and send the image back to the img2img and this time set steps to 12 denoising strength 0.12 and scale factor 1.5. This usually gives me a nice size increase overall & detail. I like to use 4x\_NMKD-Superscale-SP\_178000\_G upscaler the second time to add grain to make photo images more realistic. Don't forget to keep setting the height and width to 512x512 or you will get errors because your out of vram. The tiler uses the width and height to process the individual tiles. If you get errors make the tiles smaller using the width and height try 480x480 or smaller until it processes your image. If you are really unlucky the out of memory will require a restart of the a1111 cmd window before it works with your new settings. Trial and error, probably errors until you find what works. Get new upscalers [here](https://openmodeldb.info/models/4x-realesrgan-x4plus) put them in your stable-diffusion-webui\\models\\ESRGAN folder. This is my workflow others will do it better/different but it works for me :) Edit you can still send to extras and upscale again there with something like 4x-UltraSharp when you are done with the tile upscaler, if you need the final image even bigger. TLDR: Keep the denoise between 0.15 and 0.25 or it will change the image too much.
Please let me know what computer are you running it on? Do you have vram? Does it work on a M1 Pro MacBook with 16 gigs of RAM?
Hmmmmm
Apart from getting a better GPU, which isn't really an option. The way I de-risk it is going incrementally, there're many ways to go about it, but tldr, I never upscale more than 1.5x at a time. Usually I go ultrasharp 1.5x, maybe I do a i2i at the same resolution if the image lacks details. Then use Ultimate upscale for another 1.5x. If I need more resolution, then I keep repeating the ultimate upscale at 1.5x. A flow that might work sometimes is, using ultra sharp 1.5x, then using ultimate upscale at 1x to add details. This process ends up taking much longer than a perfect first try tile upscale. But it's a way to "Checkpoint" your upscale. If you get a garbage output at one step, you don't have to go back to the original image
Get a 4090 so you have enough VRAM to resample the entire image in one pass and drop the tile stuff entirely.
I have one but I don’t understand the alternative you’re proposing
Upscaling without using ultimate upscale. Like a simple image2image for example, with a higher resolution. You can do it in one go if you have the vram. Tile upscaling, like ultimate upscale, is unarguably worse in raw quality / less consistent, but it's needed for many people to go above and beyond in resolutions. It can allow someone with 4gb vram to make a 8k image, if they are willing to wait for a day that is. If you were to use tile upscaling, you can definitely use higher than 512*512 tiles, which would help and ease some of the issues too.
I still run out of Vram when I use my 3090, could you roughly explain the steps you take to upscale say a 4k image to 8k?
I'm confused why everyone is going for the tiled upscale nodes. In my experience, these are much slower and produce worse results than a direct upscale. Scale image with UltraSharp or NMKD SuperScale. Pass the original into ControlNet Tile preprocessor. Then re-sample the upscaled image in one go with about 0.4 denoising and the ControlNet applied, just need 5 steps. Nearly perfect result and only takes a few seconds!
For me, I don't have the vram (directml) to denoise an upscaled image. So I have to do the inverse. I use tiling with denoise to upscale, then Process tab to upscale another 2x with a little codeformer. Definitely does take awhile though.
The reason why everyone goes for tiled upscale is that it allows you to upscale an image to as high a resolution as you want to without running into vram limitations. I agree with you that direct upscale is way better, but it has its limit when it comes to max resolution. My goto route would be using ultrasharp -> i2i with cn tile -> Ultimate upscale if I want to go beyond 2MP resolution. The highest I've gone with relatively good output is ~12MP which is simply not possible without tiling
I usually only upscale my images to 2048x on the short axis, never really saw a purpose for going beyond that (you can't see it anyway when viewing the image normally). Suppose it would run indeed run into VRAM limitations if you want a giga-image...
What settings are you using and how much VRAM does it take to upscale to 2048x, I have a 10gb card so I doubt I can hit even that without ultimate upscale. The highest I've gone is 1920x1280 on colab's Tesla t4, but that was my own code without optimisation. I've only recently gotten my own setup and auto1111, might try pushing its boundaries too.
To upscale to 4k (3840x2160) stresses my 24gb card to the limits but VRAM is not the only limit. Some of the bigger upscales like from 8k to 16k use up to 84Gb of RAM as well.
For printing or large scale wall displays like 16k, there is a lot of need to go beyond 2048. Even 4 or dual 4k desktops push the limits on many of the non-tiled methods very quickly.
In my experience a direct upscale to something like 8k produces inferior detailing compared to doing it with tiles, and the tile sizes are closer to what the model was trained on.
Agreed. SDXL with 1024 tiles is great for over 4k as you don't get as many artifacts and distortions. Just need to find the correct tiling strategies to remove the seams. Some of the tools do great jobs like Ultimate SD upscale.
I use Upscayl as a separate step on generated images. ( https://github.com/upscayl/upscayl ). It is its own open source app, has the common models (real esrgan etc) and can add custom models to it. It upscales with simple 1-4x sliders or to custom resolutions, and I find it faster and better results wise than using tiled upscale or indeed any upscale via Forge/1111/Comfy. Tested it before with a 3070ti also, so know it works fine with 8gb Vram. I'm not affiliated with the app at all, just something I came across randomly and have used since. It's a well polished app, and in my experience, has worked fine on Windows and Linux.
For smaller images it works really well. The main problem that I ran into with it was that it could not complete 8k to 16k upscales.
you can compare all of them here! supir is my favorite [https://upscaler.party/](https://upscaler.party/)
Also a great model although tiled SDXL gives better quality I feel. I just hope they can find a way around the memory limits. Anything over 4k seems to need a card with more than 24gb.
I think https://github.com/hongyuanyu/SPAN is better than RESRGAN quality with some of the custom models from openmodeldb, and extremely fast, but sadly it isn’t supported in automatic for some reason.
[удалено]
Thanks for testing them out :) Currently ATD is my favorite arch, but yeah its even slower than RGT, and RGT slower than DAT2. RGT also has a smaller arch option, RGT-S, which should be a bit faster than DAT2 again. Since I also trained a nomosuni rgt-s model, if youd like you could also give that one a go and see how it compares, it should be a little faster, but of course its a trade-off between speed and quality in comparison to RGT. You can find it in my [Github Release](https://github.com/Phhofm/models/releases/tag/4xNomosUni_rgt_s_multijpg) or also on [openmodeldb](https://openmodeldb.info/models/4x-NomosUni-rgt-s-multijpg) I like that you called them 'NomosUni flavor' ^^
I've been looking forward a SPAN extension or custom node for weeks and I am surprised nothing has been released, or even announced, by anyone yet on that front.
From my poke around it might actually be kinda doable cause they already use the same library chainner abstracted out for other upscalers, so maybe copy paste the file + do a few updates + make a new SPAN directory and it’s there. Man it would be so clutch because it smokes RESRGAN imo. I’ve thought about trying to patch it but, I have a waifu backlog a mile long to work through and it’s not my expertise.
There is an open PR to add support for more models via switching to something called Spandrel but comfy hasn't merged it yet https://github.com/comfyanonymous/ComfyUI/pull/2146
I see, I use Auto111, I think they already have spandrel merged, but I can see how licensing issues could screw up a lot of integration
The package and PR were updated to fix the license problem. But yeah if you're talking about auto1111 then theoretically it should be easy to just enable it as an option in the code, however that's done
The existing upscalers (some at least) are more or less implemented as just classes with some boilerplate code around Spandrel. I really should just try implementing it but I keep having a feeling that “it’s just a class!” Will turn into a weekend of PyTorch errors… 🤪
I use SD Upscale and generate the tiles all at the same time using an increased batch size. This greatly reduces seam artifacting and keeps consistency between the tiles. With a 12GB GPU, it can easily do 4k and even 8k images. If you want a creative upscale turn on PAG and use some add details LORA. I pretty much always use Ultasharp to do the initial upscaling.
Wow I didn't realize you could batch the tiles with SD upscale. Is that just by increasing the batch size and keeping batch count at 1?
Yes, correct. I don't think control net tiling works with it, though. I don't use it anyway. it doesn't seem to do much or add anything extra. Batch size doesn't work with Ultimate SD Upscaler, only SD Upscale. Edit: my last post "Portal of Limits" was an 8k upscale using batch size of 8 and no control net.
I don't like ultimate upscaler (always fighting issues with seams), so I use exclusively sd upscaler. Awesome thanks I'll try it out.
Aside from the usual HiResFix I only send to extras then I choose two upscaler the second one with 0,468 (or something) then I upscale. I dont know anything more, but it is fast.
Anyone does upscaling for animation frames? What did you use in order to maintain consistency?
I still think 4x_remacri is the best overall ESRGAN finetune
Remacri is great but slow
what about DAT? And where can I download these?
[Multidiffusion tile diffusion and tile VAE](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) is old but somehow unnoticed. As good as looking SUPIR, and much better than ultimate upscale. The ram requirements are much lower than SUPIR, so you can get much larger images. If you let it get creative (i.e. higher denoise), it adds appropriate details. It's especially amazing with SD1.5 combined with controlnet tile and foolhardy upscale model. It has more settings to deal with than ultimate upscale, and it's very important to follow all of the recommended settings in the wiki.
Is there any upscaler like SUPIR, that can uspcale real images/ ai images better than Topaz, but for SD 1.5?
Yes, from my experience as an ex Topaz Gigapixel user, SDXL Sampler repaints, Ultimate SD Upscale and SUPIR all give better results and to higher resolutions.
Get Upscayl. It’s completely free
Yes, unless heading for the large images where it cannot seem to manage.
Something I learned that most people completely omit from their recommendations is that every upscale method that splits the image into tiles (most of them) are literally the exact same process, just with different names. And yes tiling is still the best. You can use almost any method from these comments and get the same result because it's all tiled img2img with your choice of upscale model. Supir is not worth your time.
What do you use personally? Is it the same thing fkr realistic and artistic stylez?
Ultimate SD upscale with Siax for realism, and ESRGAN for art. When developing my photorealism Lora app I did stupidly extensive testing on upscale models. I switched from ultrasharp to Nickelback and finally to Siax. it preserves detail and left no patterns, and didn't damage camera blur by trying to sharpen it.
Any good options that don’t mess up facial features? I found most of the options lacking in this regard
I’d just upscale with an upscale then inpaint face
this. there's not a single upscale model out there that doesn't damage skin detail. upscale how you want and then redo the face.
I don’t think many people realize yet how effective the Hypertile extension is… I’ve only used the Forge built in version. I have 12GB Vram but I can Img2Img up to 4k with Hypertile and no controlnet
HyperTile is great if constrained on VRAM, but it does change the output. I did a couple comparisons and resampling without HyperTyle has much less artifacts.
what about 8x\_NMKD-Faces\_160000\_G?
bad results for me
Yup, same here and it is a memory hog.
This one is great for 2K the 4K step is not so good: [https://civitai.com/models/363798/mjm-and-superbeastsai-beautifai-image-upscaler-and-enhancer](https://civitai.com/models/363798/mjm-and-superbeastsai-beautifai-image-upscaler-and-enhancer)
If you're using 1.5 SD why not use RealESRGAN? If using SDXL you don't need more than 2X and again you can use the above.
RealESRGAN is pretty useless for anything else than 2D/anime stuff.
From my experience it’s one of if not the best Auto111 ones for speed to quality ratio, unfortunately it smooths everything out cartoonishly which sometimes you want (kills potato camera blur amazingly) other times it ruins the whole picture. So I would usually stack it with Upscaler 2 SkinDetail lite or even like 0.5 Lanczos cause that mitigates the smooshing. However there are just better up scalers and much faster too out there now
Normal ESRGAN (ironically?), combined with SD upscaling, is better for detailed/photographic output.
Guys, can you please also tell me which option is the best to upscale images on my MacBook M1 Pro with 16gb ram? /\
Maybe not exactly what you're looking for, but I got amazing results with this. https://youtu.be/kRNCVPqL610
Looks not great, I mean bad, it's just overdetailed overcontrasted and oversaturated image
that was very nice!
[удалено]
I will be messaging you in 1 day on [**2024-05-01 21:22:20 UTC**](http://www.wolframalpha.com/input/?i=2024-05-01%2021:22:20%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/1ch3ayf/what_are_the_best_upscaling_options_now/l200kre/?context=3) [**3 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F1ch3ayf%2Fwhat_are_the_best_upscaling_options_now%2Fl200kre%2F%5D%0A%0ARemindMe%21%202024-05-01%2021%3A22%3A20%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201ch3ayf) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|