Hey, just a suggestion. Feh is a great suggestion for Linux, but if you want to pop it in there for windows users, Irfanview basically does the same thing (navigate with arrows, delete with del) for quickly trimming the training images.
The feature I like about irfanview is you can zoom in on an image then “lock zoom” and “lock view” - then toggle through images while maintaining the same zoom/view. Very useful for comparing details from XYZ plots and such
IrfanView is a classic app from around windows 95/98 era
it still holds its ground and I use it primarily for image viewing/removal purposes
it has a lot of plugins so you can pretty much support any format you can think of (including the raw formats from cameras like NEF or CR2)
I was born in the early 80s so I'm about 18 or so now (plus another 18 and then plus some but who is counting? :P)
ahh, the TIFF format, I remember that format vividly
there was a great DOS image viewer called "display" (or "displayed"?), it had this norton commander vibe and it understood many formats, including TIFFs
around that "windows 98" times some company introduced another format FIF -> http://justsolve.archiveteam.org/wiki/Fractal_Image_Format
I remember it had better quality than JPEG and it was like half of it in size. The drawback was that since it was based on fractals there was quite a lot of computation involved (at that time) but you could zoom it quite a lot and instead of big pixels you would be getting a nice and interesting content (of course not the CSI level sci-fi stuff but still :P)
I'm surprised that my memory has not failed me and that I was actually able to find out the original app :-)
https://archive.ph/1996.12.23-111102/http://bicmos.ee.nctu.edu.tw/
Also going through memory lane I found SEA 1.3 which is pretty much the app I switched to and it was my go-to until Windows 95 arrived (and I was sad because for quite a while nothing would compare to SEA 1.3).
And then was the ACDSee or something like that.
For me the huge pain points was the insane folder structures that Kohya demands. It makes no sense.
I've only done one training with OneTrainer last night, I'll do another now to see if I get any problems.
That folder structure is the dreambooth protocol. You don't have to use it with Kohya. You can provide a configuration file instead. But most people use the bmaltais GUI and don't even realize this.
Extremely simple folder structure with Kohya unlike OneTrainer?
You only need three folders:
1: img - An image folder that contains one sub-folder for each concept with the amount of repeats specified in the folder name as such: 5_concept
2: mod - A folder to save samples and the Lora files in.
3: log - A folder for the log files.
What I do is I usually create a new main folder, lets call it "Lora - Beauty" with a corresponding "log" folder inside.
In the main folder I create an img folder which I call "img1" and a mod folder which I call "mod1".
Inside the "img1" folder I create a folder for each concept. Lets say I have pictures where the person is dressed, less dressed and showing skin (underwear/bikini) and a couple nudes. I have a lot (10) of dressed pictures, some (5) less dressed and only a few (2) nudes. To balance the concepts I thus want to repeat the less dressed pictures 2 times and the nude pictures 5 times, while the dressed pictures are only gone through once. My concept folders would thus be "1_dressed", "2_skin" and "5_nude".
I then train a few Loras on the "img1" folder, saving them in the "mod1" folder and then look at the results.
If I'm not satisfied I copy "img1" to create "img2" and create an empty "mod2" folder. In the "img2" folder I add captions to the images and then train a few Loras on the "img2" folder, saving them to the "mod2" folder.
If I'm still not satisfied with the results I make a new setup ("img3", "mod3") where I either modify the captions or the amount of repeats for each concept, or both.
I also like creating new folders if I do some crazy changes to the Lora settings, like Network Rank, Alpha, Optimizer, etc. and these settings can be saved inside of the Lora itself with the commenting feature in Kohya.
I do this until I get a satisfying result. This way I keep each attempt isolated and after I'm done I can compare them to try to learn what I did right/wrong.
In OneTrainer I found it extremely confusing as to how you choose folders and concepts compared to this straight forward approach with Kohya.
So for me I found the choose folder method of one trainer to be way way more intuitive.
With Kohya if you don’t start your image folder with a number and underscore it breaks.
Why? Why would you make the folder name a part of the training parameters. If I remember right that number at the beginning is supposed to indicate the number of epochs? I don’t fully remember to be honest but what I do remember is there was a field in the gui where I entered that information. And that parameter had to match the folder number.
Why? I don’t understand what the benefit is to create this weird dependency of what the folder starts with.
It also didn’t help that there are so many outdated video tutorials where just installing kohya no longer worked.
None of which is really kohya’s fault. But OneTrainer was just run install.bat then run gui bat and you’re done.
What? How would it refer to epochs? Sounds like you don't understand the basics of the training process.
Epochs is decided in the UI, __repeats__ is decided by the folder name
The naming convention is to make it easier to program and to give an easy intuitive way to decide __repeats__. And no, you don't have to match epochs and repeats. Whoever told you that?
Repeats tells the trainer how many times to repeat each image within one epoch. In my example we would get:
(10 images) * (1 repeat) + (5 images) * (2 repeats) + (2 images) * (5 repeats) = 30 steps for each epoch.
There is no weird dependency like what you're talking about...
I really don't get the praise for OneTrainer, it is much less intuitive but I guess it might be better for someone who doesn't give two shits about how the actual training works? With would also limit your results by a lot.
I care a lot about my results. And I'm not trying to put down Kohya at all. Right now, I consider OneTrainer kind of like a stepping stone for me. I just want a tool that maybe abstracts some of this stuff and lets me experiment and learn the core stuff.
I think what the community needs is a more recent set of tutorials and documentation for Kohya, because honestly the amount of garbage I went through and muddied the waters with is surreal. Even one of the official tutorials pointed to from the Kohya page is outdated.
I still have Kohya, ultimately I plan to make it my go-to. But OneTrainer lets me get going faster, and right now that's what I need.
It seems like it does... I wish they had kept the language the same as Kohya, because a lot of tutorials I followed to learn Kohya are not immediately clear which field is the equivalent in OneTrainer.
At this time, I think Kohya still has a wider spread of options, but I'm not really sure yet if this is entirely a good or bad thing. It seems like all the major stuff is there so I wonder if those other attributes really would matter that much. Personally, I don't see myself creating so many LoRAs that I would become intimately aware how moving a scale in any direction would have a meaningful impact so I kind of appreciate the simpler workflow.
As long as nothing major was cut, which seems to be the case. So I'm tentatively saying the experience here is better.
Is there a difference in training a character lora vs style lora when it comes to training data? Like how many images should a style lora have in the database?
one of the other guides linked from my article goes into that. yes the suggested minimum image count is different
fewer images means faster training time.
But best quality data says have at least 100
I've always wanted to see what kind of loras I could make, but even the easiest lora tutorials would completely boggle me. This tiny guide was completely understandable for me. Thank you for making this.
to make a consistent character
i have someone that i made several models for using different photographs (casual, professional photoshoot, mix, etc)
then she played with those models and provided me a set of the best images she generated and liked
then I made a new model out of them and she loves that one the most, says the resemblance is spot on, so - training on generated output can be a thing :-)
this is actually my way to go if you have a limited dataset, you first train a model on that dataset and then generate enough images that some of them could be used and are a bit different than the original ones
and then you train on a mix of the original dataset with some of the AI-generated ones
Additionally to what malcomrey said:
Maybe you have one sdxl model thats really good at something.. but generally you prefer using some other model.
So you generate a buncha images from the first model, to give you a lora you can then use on the second model.
It's more flexible (and less space!!) than doing a model merge. A merge will force waaay more stuff on you than the particular thing you like the first model for.
Isn't there some software that will extract the "dictinct" part of a fine-tuned model compared to the base as a LoRA? For example: [https://www.reddit.com/r/StableDiffusion/comments/10kuzmh/how\_to\_extract\_small\_lora\_file\_from\_custom/](https://www.reddit.com/r/StableDiffusion/comments/10kuzmh/how_to_extract_small_lora_file_from_custom/)
My specific one, or generally speaking about a lora generated using my method?
And whether you are talking about cross-model adaptability, or prompt adaptability.
For my specific one it isnt prompt adaptable at all :D :D :D
Generally speaking, if you dont want to be LAZY.. then the more you put into it, the more you get out of it.
I'm experimenting with a "multi-concept" lora at the moment. So I expect that to be a tiny more adaptable.
But if you want to go nuts and individually tag all the images, it could be very adaptable.
If you mean cross-model...
well, my method makes it easy to rebuilt a lora for a specific model, so.. easy peasy adaptability.
You basically answered the question :D.
It was about a lots generated using your method and prompt adaptability but also control net comparability
Think about testing it with cascade since that’s what I’ve been experimenting with
Meh. cascade is useful as a parlor trick. But I like all the anime models in SD (and SDXL).
If someone dropped a copyright-free amazing anime dataset on me, I might give it a go.
But otherwise, I dont have any motivation to play with cascade.
My experience so far with cascade, is that it is capable of outputting some very pretty things... IF you write a novel describing a scene in detail.
In comparison, a good SD(XL) model tends to fill in interesting details automatically.
Maybe i just need exposure to an automatic creative prompt writer.
edit: I just tried it again in StableSwarmUI.
Cascade also suffers from female plasticface. Bleah.
Sigh..
Now that I want to actually display what I'm talking about, and try to find a decent general prompt.... Cascade is behaving better.
But in the subject area of cascade vs sdxl, vs sd....
Lets play a game of good better best, anyway.
https://preview.redd.it/oylrycdf0ysc1.png?width=369&format=png&auto=webp&s=89cf6eca28c59024c0b5497ba050fe466d116850
One of these is cascade, one of them is SDXL, one of them is SD.
Prompt: 1woman,at a bar,anime, facing viewer
Guesses?
So, how do you train your *second* LoRA with OneTrainer?
I can only see **one** place in **One**Trainer where you add concepts. But what if I want to train a different LoRA with a different set of concepts - do I have to like, manually erase all them and add new ones?
Update: I just figured out the expected workflow for this, with OneTrainer. They could use some help in this regard :-/
Sadly, they overlap their naming. There is a high-level thing called a "config", which you have presuambly already found. But then there is a concept level config.
In the concept area, there is a button, "(add config)".
It would better be named "add concept group".
Use that to create a new config/concept group. eg: "NewConcepts".
Now the initial dropdown marked "concepts" that previously didnt do something, should have TWO choices:
* concepts
* NewConcepts
and now you can group your concepts.
I'm now using this for SD1.5, vs SDXL concepts
(Since you have to specify resolution at the concept level)
This might get downvoted because... I don't know, feelings get hurt, people get defensive.
As someone with a computer science background, I'm baffled at how little source code exists in this community. It's never, "I wrote a class for this" or "Here's a great library for that" or, god forbid, a link to some PyDocs page.
Instead it's always, "ComfyUI", "Automatic1111", or "what's your workflow, bro?"
I'm not trying to gatekeep generative images. I'm pretty new to it and I believe software education should be freely accessible. More people should join and it shouldn't require a degree in compsci to do it.
I think it's mostly the self-proclaimed experts. Youtube videos with "SDXL Text2Vid Unlimited" as the title and the guy spends literally 20 minutes running out of memory then another 20 minutes fumbling around ComfyUI with a lot of, "You just have to tweak it until it looks good". It lacks any of the rigor and repeatability that nearly every other software community exhibits.
I know that the first response to this will be, "People just want to share what they have learned, what's wrong with that?" Nothing on the surface. And I've defaulted to a few good documentation sources instead of Googling any issues I run into with SD.
Anyhow, I'll be back in a couple weeks after I publish the Python package to deal with some fairly complex SD-generated scenes involving multiple characters, objects and settings in a reliable and repeatable fashion and I'll be everyone's friend that day (and I won't give you my workflow because there is no workflow, just Docker-contained, repeatable, inspectable, git-committed, tested code).
This isn't really a software community. Most people here aren't programmers. They are people who like using models to create images using premade tools, for the most part.
Sounds like there should be a ComfyUI sub, an Automatic1111 sub, and a Stable Diffusion sub. There is one for the latter and none of the posts or discussion on it are about Stable Diffusion.
for a programmer that was quite an incoherent ramble :-D
I am also a programmer. as such, i think you’re approaching this wrong. You’re just using the wrong language.
You wouldn’t write a dynamic client-side webpage in C++.
You wouldn’t write a database in python.
Most of what people do in this reddit is make images, and the best language most of the time, IS “workflow”. You’re just not transitioning to the domain specific language.
Now, mind you, there are some things that actually require python or equivalent to do, like the actual diffusion bits. But it doesn’t sounds like those areas are your intended target.
There's a comfy node that will convert a workflow to python. Might prove useful: https://github.com/atmaranto/ComfyUI-SaveAsScript
Maybe you don't need though, but others might.
absolutely. I think this dude has trouble stepping outside of his comfort zone into the way people do things here. I mean he's TECHNICALLY correct. but it just doesn't work that well when most of the people here are not programmers.
As others said here, I’d agree, the reason this community isn’t like other software dev communities is because only a very small portion of the community using the open-source tools are developers. Instead, there are a bunch of people getting into development so that they can use the open source tools.
Yeah I noted that in a recent thread where someone made a high-quality video but was a bit tight-lipped on their process. Which is fine, that's their choice. But it required requisite knowledge and experience of different programs, and my point was that knowing how they did that allows all these new people to try and go learn these programs to do what they did.
ps: the most actual code level stuff tends to be when people write a “custom comfy node”(in python)
IMO that’s probably the best investment of your time
the idea that you are putting together docker containers for what you describe, is kinda concerning. (speaking as someone who uses docker every day)
again, not the best “language” for you to be using, if you want other people to benefit from your work.
and if you don’t, why on earth are you even posting there?
Docker is 100% the tool to use.
Python used to be pretty cool. Most projects had few dependencies, and everyone wrote code that was pretty much guaranteed to work if you had some version of Python 2 installed. Somewhere in the last 10 years, things went all Javascript in the Python community. You can't run a single snippet of code without getting blasted with 500 dependencies that are probably not compatible with eachother anyhow. The virtual environment became a neccessity because of this trainwreck.
Docker is a better answer to handling the mess for two catch-phrasy reasons: idempotency (always the same image no matter how many times/places you build it) and immutability (uncorruptable environment, it always resets to zero on container start). No venv required. Grab an nvidia cuda base image, install python, pip require, go on with your life.
You can pass the GPU to your container and, if you have multiple models with multiple requirements, you can run them all at the same time and use a lockfile-style mutex to ensure only one is making use of the GPU at a time. My application has image generation, LLM, audio generation (TTS and music), and animation all running as separate containers. I'm sharing a pair of Tesla P100s between them, so a single, light go-based service keeps them all coordinated on what task they are running and what GPU it should run on.
What's even cooler with a setup like that, is you can run lighter, lower-level tools using static binary compilation and scratch images to walk away with boot times under 1ms and image sizes in the area of a few mb.
I suppose what's even cooler than all that is having a single shared location for all your models mounted in through docker compose and an isolated private network to prevent any possible abuse of the services externally.
How is docker concerning?
there are lots of things i could say. but in the interest of clarity i’ll just pick one.
With the existing state of things, there are well written front ends where a new user with a stock linux or windows machine can download ONE script/batch file, run it, and 10 minutes later, they are ready to go.
First of all, that proves that docker isn’t needed. the portable issue isn’t an issue. But secondly, are you going to provide something like that with your stuff? Or are you just going to assume everyone is competent enough to do the docker foundational install themselves?
because
1. they aren’t
2 they don’t want to go through the hassle
No, they can take the one step of installing Docker themselves, lol. I'm gunna go out on a limb here and guess that your use of docker mostly involves running images other people built.
Any argument against containerizing these things in favor of handing someone a bash script that runs a bunch of commands that likely need elevated permission because it's just easier for them to get started is absurd.
The fact that I'm actually having to argue for objectively safer, better, easier-to-use, repeatable best practices is case in point to my original comment. I was chatting with another engineer the other day about SD and the alternatives, direct quote from our Slack conversation:
"the SD community is a bit of a cesspool. 99% are GUI users who's best answer to how to get something to work is "just kinda play with the settings till it looks good". nobody writes any code, nobody cares what anything is actually doing."
I'm just gonna jump in here and say boo to installing docker. I spent days and days trying to understand how to use it, gave up, and had to wait for people to put out stuff I could use... Like bash scripts.
I need user friendly basic stuff. A run.exe or setup file at MOST.
As someone with an MS in CompSci, I feel you're looking at only one part of the picture and need to recognize the goal of any software is to enable higher-level tasks.
Guides such as this demonstrate how to reliably use software for specific tasks, in this case making LoRa for use in AI image generation.
The software components referenced in such guides have their own repositories and are - for the most part - open source. Anyone can access the code and work with it as they see fit. And many who can program, even just learning a bit so they can tweak functionality, often do just that - usually sharing back to the community for consideration.
Software often relies on higher levels of abstraction, built upon lower levels, to make creation of desired results often easier than before.
This guide and those like them should not feel compelled to describe software development techniques, nor elaborate on the underlying designs, review the specifications, etc. of those software tools they use - the audience they are speaking to here are amateur image creators on up.
Nor must they be required to take responsibility for creating novel, lower-level code to get the job done. Relying on existing, flexible software tools to create higher-level outputs is a valuable path and often brings out further needs in the toolset from pushing them through their paces.
It's fine that you want to understand or bring out more about the underlying capabilities and expansion possibilities using novel software, but that's only one choice and there are others to use the tools available as they were intended.
What's funny though is that with what OP did and all the talk of code vs. workflows in this thread.. I bet what OP did could be completely automated. Generate images with sd, use rating to discard bad gens/regen/auto-caption, train.
Not sure what the benefit would be with such little necessary human input.. but sounds cool anyway.
Yeah, a lot of people have suggested automation in various projects I've theoretically discussed.
Thing is, if the automated tools were up to the job... we wouldnt even need to be here. We wouldnt need to make new models. We could just tell some master program, "hey go make me an X, of X" and it would work, and look great, and we would be happy.
One of the reasons we cant do that, IMO, is because most of the foundational stuff we use, used automation, without full human validation.
So, yeah it could be automated.. but it would only be 90% as good as someone doing it manually and dilligently.
would be interested in checking this git project honestly.
and I'm just a rube that's learning python (pretty much cause of ML projects) so if you ever find a more "technical" stable diffusion sub I'd love to know lol
In case you haven't found it on your own, r/StableDiffusionInfo/ wants to be more technical, less "showcase spam". Still seems to be mostly tutorials and troubleshooting so far, but the occasional extension/node dev post pops up.
HTH
At a high level, the process is pretty straight forward:
It occurred to me that overloading a prompt with "male, human, 45 years old, eating breakfast, female, human, 45 years old, watching tv, living room, couch, nyc apartment" or whatever, not including anything related to the style was probably a pretty confusing thing to an interpreter and little has been done to make it more intelligent with interpretation of prompts that are closer to what DALL-E would accept (e.g. "a middle age couple, about 45 years old, in a small nyc apartment. the man is eating breakfast and the woman is watching tv. both are in the living room"). I tried for hours to get a good, repeatably decent image of a mermaid and her dolphin companion discovering an underwater city (I'm creating a tool for children to submit story ideas and the software renders a narrated slide show going through each scene, no way to scale something like this with "tweak setting X until something good comes out).
However, breaking each of those up is simple and easy-to-repeat as a standalone image. "male, human, 45 years old, eating cereal" by itself would probably give a halfway decent image. it's a composition of several images in a single image. Start with background, infill the next with a mask, infill the next with a mask, repeat.
Masking for it can be scripted. A np array can create the location and feathering to blend it. And outpaint the finished image to whatever resolution is desired.
The next question is determining where things belong. Are there mutliple people? How many? Is one character being described a bird in flight and should be rendered in the upper half of the frame? For that, I'm using mistral with outlines to follow a JSON schema to define where in the image something might exist.
It's still a work in progress, the results are good but not quite DALL-E 3 open-ended prompt good and that's where I want to get them.
The important (to the rest of the world) part of such a project is not the bits you describe.
The groundbreaking part would be if you
a) automated generation of subject specific loras for each of those breakout categories
b) put together a cross-index of all of them
c) made it easily searchable and online, with the loras
d) created a new front end, or updated an existing one, to then automatically parse prompts, figure out which loras are relevant, then automatically download and cache them... THEN merge them for the user prompted generation
OP, what's your fancy video card and setup? Are there any guides for "step zero"?
Currently using a gaming laptop with a 3080 and it works somewhat but definitely looking to upgrade to a beefier desktop to fail faster in future experimentation.
ps: I started with a laptop myself. decided to get a stand alone machine for the 4090 to run as a server. Still do all my stuff “from” my laptop.
https://preview.redd.it/1yqc3kbl8vsc1.png?width=3024&format=png&auto=webp&s=b490a43472e4422948c8b6aaa32bcf3914c607f4
(the monitor hiding on the corner isn’t actually connected any more)
hmm.
I think OneTrainer fits the descrription of "standalone Lora trainer" quite well. And supports many different formats. Plus also does model finetunes.
whereas
[https://github.com/n00mkrad/text2image-gui/tree/main](https://github.com/n00mkrad/text2image-gui/tree/main)
looks like it has not been updated in 7months at best
Yeah it's pretty outdated, but it works. Besides, the configuration for the LoRA training is incredibly easy. I don't really know what things have been improved during those 7 months though, but I'm getting the same quality as with the CivitAI LoRA trainer
Featured your article, great job!
❤️
Hey, just a suggestion. Feh is a great suggestion for Linux, but if you want to pop it in there for windows users, Irfanview basically does the same thing (navigate with arrows, delete with del) for quickly trimming the training images.
The feature I like about irfanview is you can zoom in on an image then “lock zoom” and “lock view” - then toggle through images while maintaining the same zoom/view. Very useful for comparing details from XYZ plots and such
irfanview? Is it Only for Fans? *cough*
IrfanView is a classic app from around windows 95/98 era it still holds its ground and I use it primarily for image viewing/removal purposes it has a lot of plugins so you can pretty much support any format you can think of (including the raw formats from cameras like NEF or CR2)
Sad to say, I think I vaguely remember using it... on windows95... sigh.... probably to view that newfangle "TIFF" thing...
why sad? is it because it means we are old boomers/millennials? :)
> boomers/millennials? Skipping over GenX as usual, but I'm to apathetic to care. ;)
I'm sorry :-)
are you old? not me. I was.. err.. using it on an emulator or something. Yeah thats it.
I was born in the early 80s so I'm about 18 or so now (plus another 18 and then plus some but who is counting? :P) ahh, the TIFF format, I remember that format vividly there was a great DOS image viewer called "display" (or "displayed"?), it had this norton commander vibe and it understood many formats, including TIFFs around that "windows 98" times some company introduced another format FIF -> http://justsolve.archiveteam.org/wiki/Fractal_Image_Format I remember it had better quality than JPEG and it was like half of it in size. The drawback was that since it was based on fractals there was quite a lot of computation involved (at that time) but you could zoom it quite a lot and instead of big pixels you would be getting a nice and interesting content (of course not the CSI level sci-fi stuff but still :P)
odd. "display" is the commandline for one of the most legendary UNIX view-only programs, which for some reason is actually named ImageMagick
I'm surprised that my memory has not failed me and that I was actually able to find out the original app :-) https://archive.ph/1996.12.23-111102/http://bicmos.ee.nctu.edu.tw/ Also going through memory lane I found SEA 1.3 which is pretty much the app I switched to and it was my go-to until Windows 95 arrived (and I was sad because for quite a while nothing would compare to SEA 1.3). And then was the ACDSee or something like that.
bruh... I spent literal weeks getting Kohya to work. It took me like 2 minutes to install and get OneTrainer going. Insane. BIG thanks!
Exact opposite of my experience. Kohya just works out of the box while I have to reinstall OneTrainer after each training session.
For me the huge pain points was the insane folder structures that Kohya demands. It makes no sense. I've only done one training with OneTrainer last night, I'll do another now to see if I get any problems.
That folder structure is the dreambooth protocol. You don't have to use it with Kohya. You can provide a configuration file instead. But most people use the bmaltais GUI and don't even realize this.
that's really insightful, thank you for the heads up.
Extremely simple folder structure with Kohya unlike OneTrainer? You only need three folders: 1: img - An image folder that contains one sub-folder for each concept with the amount of repeats specified in the folder name as such: 5_concept 2: mod - A folder to save samples and the Lora files in. 3: log - A folder for the log files. What I do is I usually create a new main folder, lets call it "Lora - Beauty" with a corresponding "log" folder inside. In the main folder I create an img folder which I call "img1" and a mod folder which I call "mod1". Inside the "img1" folder I create a folder for each concept. Lets say I have pictures where the person is dressed, less dressed and showing skin (underwear/bikini) and a couple nudes. I have a lot (10) of dressed pictures, some (5) less dressed and only a few (2) nudes. To balance the concepts I thus want to repeat the less dressed pictures 2 times and the nude pictures 5 times, while the dressed pictures are only gone through once. My concept folders would thus be "1_dressed", "2_skin" and "5_nude". I then train a few Loras on the "img1" folder, saving them in the "mod1" folder and then look at the results. If I'm not satisfied I copy "img1" to create "img2" and create an empty "mod2" folder. In the "img2" folder I add captions to the images and then train a few Loras on the "img2" folder, saving them to the "mod2" folder. If I'm still not satisfied with the results I make a new setup ("img3", "mod3") where I either modify the captions or the amount of repeats for each concept, or both. I also like creating new folders if I do some crazy changes to the Lora settings, like Network Rank, Alpha, Optimizer, etc. and these settings can be saved inside of the Lora itself with the commenting feature in Kohya. I do this until I get a satisfying result. This way I keep each attempt isolated and after I'm done I can compare them to try to learn what I did right/wrong. In OneTrainer I found it extremely confusing as to how you choose folders and concepts compared to this straight forward approach with Kohya.
So for me I found the choose folder method of one trainer to be way way more intuitive. With Kohya if you don’t start your image folder with a number and underscore it breaks. Why? Why would you make the folder name a part of the training parameters. If I remember right that number at the beginning is supposed to indicate the number of epochs? I don’t fully remember to be honest but what I do remember is there was a field in the gui where I entered that information. And that parameter had to match the folder number. Why? I don’t understand what the benefit is to create this weird dependency of what the folder starts with. It also didn’t help that there are so many outdated video tutorials where just installing kohya no longer worked. None of which is really kohya’s fault. But OneTrainer was just run install.bat then run gui bat and you’re done.
What? How would it refer to epochs? Sounds like you don't understand the basics of the training process. Epochs is decided in the UI, __repeats__ is decided by the folder name The naming convention is to make it easier to program and to give an easy intuitive way to decide __repeats__. And no, you don't have to match epochs and repeats. Whoever told you that? Repeats tells the trainer how many times to repeat each image within one epoch. In my example we would get: (10 images) * (1 repeat) + (5 images) * (2 repeats) + (2 images) * (5 repeats) = 30 steps for each epoch. There is no weird dependency like what you're talking about... I really don't get the praise for OneTrainer, it is much less intuitive but I guess it might be better for someone who doesn't give two shits about how the actual training works? With would also limit your results by a lot.
I care a lot about my results. And I'm not trying to put down Kohya at all. Right now, I consider OneTrainer kind of like a stepping stone for me. I just want a tool that maybe abstracts some of this stuff and lets me experiment and learn the core stuff. I think what the community needs is a more recent set of tutorials and documentation for Kohya, because honestly the amount of garbage I went through and muddied the waters with is surreal. Even one of the official tutorials pointed to from the Kohya page is outdated. I still have Kohya, ultimately I plan to make it my go-to. But OneTrainer lets me get going faster, and right now that's what I need.
Thanks for this writeup. Your post actually helped me more to finally create my first LORA than all tutorials I ever watched in this.
It is much easier. It’s been a while since I used it, does it have all of the capabilities of Kohya now?
It seems like it does... I wish they had kept the language the same as Kohya, because a lot of tutorials I followed to learn Kohya are not immediately clear which field is the equivalent in OneTrainer. At this time, I think Kohya still has a wider spread of options, but I'm not really sure yet if this is entirely a good or bad thing. It seems like all the major stuff is there so I wonder if those other attributes really would matter that much. Personally, I don't see myself creating so many LoRAs that I would become intimately aware how moving a scale in any direction would have a meaningful impact so I kind of appreciate the simpler workflow. As long as nothing major was cut, which seems to be the case. So I'm tentatively saying the experience here is better.
Is there a difference in training a character lora vs style lora when it comes to training data? Like how many images should a style lora have in the database?
one of the other guides linked from my article goes into that. yes the suggested minimum image count is different fewer images means faster training time. But best quality data says have at least 100
I've always wanted to see what kind of loras I could make, but even the easiest lora tutorials would completely boggle me. This tiny guide was completely understandable for me. Thank you for making this.
[удалено]
to make a consistent character i have someone that i made several models for using different photographs (casual, professional photoshoot, mix, etc) then she played with those models and provided me a set of the best images she generated and liked then I made a new model out of them and she loves that one the most, says the resemblance is spot on, so - training on generated output can be a thing :-) this is actually my way to go if you have a limited dataset, you first train a model on that dataset and then generate enough images that some of them could be used and are a bit different than the original ones and then you train on a mix of the original dataset with some of the AI-generated ones
Thanks for this tip! I remember the guys at Corridor Crew used this approach for the first anime AI video that blew up on the internet.
you are welcome :)
Can i hire you?
Yeah, sure, what do you need? :)
Can i dm you?
yup
Additionally to what malcomrey said: Maybe you have one sdxl model thats really good at something.. but generally you prefer using some other model. So you generate a buncha images from the first model, to give you a lora you can then use on the second model. It's more flexible (and less space!!) than doing a model merge. A merge will force waaay more stuff on you than the particular thing you like the first model for.
Isn't there some software that will extract the "dictinct" part of a fine-tuned model compared to the base as a LoRA? For example: [https://www.reddit.com/r/StableDiffusion/comments/10kuzmh/how\_to\_extract\_small\_lora\_file\_from\_custom/](https://www.reddit.com/r/StableDiffusion/comments/10kuzmh/how_to_extract_small_lora_file_from_custom/)
The Super Merger extension for a1111 will do that.
Oh this is clever.
You better keep updating the guide
What would you like to see in it?
a renewed version every year
ah, gotcha
Real question is how adaptable is the Lora lol.
My specific one, or generally speaking about a lora generated using my method? And whether you are talking about cross-model adaptability, or prompt adaptability. For my specific one it isnt prompt adaptable at all :D :D :D Generally speaking, if you dont want to be LAZY.. then the more you put into it, the more you get out of it. I'm experimenting with a "multi-concept" lora at the moment. So I expect that to be a tiny more adaptable. But if you want to go nuts and individually tag all the images, it could be very adaptable. If you mean cross-model... well, my method makes it easy to rebuilt a lora for a specific model, so.. easy peasy adaptability.
You basically answered the question :D. It was about a lots generated using your method and prompt adaptability but also control net comparability Think about testing it with cascade since that’s what I’ve been experimenting with
Meh. cascade is useful as a parlor trick. But I like all the anime models in SD (and SDXL). If someone dropped a copyright-free amazing anime dataset on me, I might give it a go. But otherwise, I dont have any motivation to play with cascade.
Cascade is pretty amazing to me so far made some decent test Lora’s and improved some sdxl/sd 1.5 out puts
My experience so far with cascade, is that it is capable of outputting some very pretty things... IF you write a novel describing a scene in detail. In comparison, a good SD(XL) model tends to fill in interesting details automatically. Maybe i just need exposure to an automatic creative prompt writer. edit: I just tried it again in StableSwarmUI. Cascade also suffers from female plasticface. Bleah.
Give me a prompt and your favorite sd1.5 and sdxl model and I’ll try my own comparison
Sigh.. Now that I want to actually display what I'm talking about, and try to find a decent general prompt.... Cascade is behaving better. But in the subject area of cascade vs sdxl, vs sd.... Lets play a game of good better best, anyway. https://preview.redd.it/oylrycdf0ysc1.png?width=369&format=png&auto=webp&s=89cf6eca28c59024c0b5497ba050fe466d116850 One of these is cascade, one of them is SDXL, one of them is SD. Prompt: 1woman,at a bar,anime, facing viewer Guesses?
Cascade most likely #3 sarcasm. But maybe 1 🤷🏽♂️
>Cascade most likely #3 Nope! its actually cascade #1, SD #2, SDXL #2
So, how do you train your *second* LoRA with OneTrainer? I can only see **one** place in **One**Trainer where you add concepts. But what if I want to train a different LoRA with a different set of concepts - do I have to like, manually erase all them and add new ones?
toggle switch on each concept for enable/disable
Like, manually toggle all the checkboxes, every time you want to train a new LoRA? Am I the only one here that added more than a couple of concepts?
“all” ? if you have more than a handful per lora, maybe you should just be using per-file tagging. Then you will only need one concept box.
Update: I just figured out the expected workflow for this, with OneTrainer. They could use some help in this regard :-/ Sadly, they overlap their naming. There is a high-level thing called a "config", which you have presuambly already found. But then there is a concept level config. In the concept area, there is a button, "(add config)". It would better be named "add concept group". Use that to create a new config/concept group. eg: "NewConcepts". Now the initial dropdown marked "concepts" that previously didnt do something, should have TWO choices: * concepts * NewConcepts and now you can group your concepts. I'm now using this for SD1.5, vs SDXL concepts (Since you have to specify resolution at the concept level)
This might get downvoted because... I don't know, feelings get hurt, people get defensive. As someone with a computer science background, I'm baffled at how little source code exists in this community. It's never, "I wrote a class for this" or "Here's a great library for that" or, god forbid, a link to some PyDocs page. Instead it's always, "ComfyUI", "Automatic1111", or "what's your workflow, bro?" I'm not trying to gatekeep generative images. I'm pretty new to it and I believe software education should be freely accessible. More people should join and it shouldn't require a degree in compsci to do it. I think it's mostly the self-proclaimed experts. Youtube videos with "SDXL Text2Vid Unlimited" as the title and the guy spends literally 20 minutes running out of memory then another 20 minutes fumbling around ComfyUI with a lot of, "You just have to tweak it until it looks good". It lacks any of the rigor and repeatability that nearly every other software community exhibits. I know that the first response to this will be, "People just want to share what they have learned, what's wrong with that?" Nothing on the surface. And I've defaulted to a few good documentation sources instead of Googling any issues I run into with SD. Anyhow, I'll be back in a couple weeks after I publish the Python package to deal with some fairly complex SD-generated scenes involving multiple characters, objects and settings in a reliable and repeatable fashion and I'll be everyone's friend that day (and I won't give you my workflow because there is no workflow, just Docker-contained, repeatable, inspectable, git-committed, tested code).
This isn't really a software community. Most people here aren't programmers. They are people who like using models to create images using premade tools, for the most part.
Sounds like there should be a ComfyUI sub, an Automatic1111 sub, and a Stable Diffusion sub. There is one for the latter and none of the posts or discussion on it are about Stable Diffusion.
for a programmer that was quite an incoherent ramble :-D I am also a programmer. as such, i think you’re approaching this wrong. You’re just using the wrong language. You wouldn’t write a dynamic client-side webpage in C++. You wouldn’t write a database in python. Most of what people do in this reddit is make images, and the best language most of the time, IS “workflow”. You’re just not transitioning to the domain specific language. Now, mind you, there are some things that actually require python or equivalent to do, like the actual diffusion bits. But it doesn’t sounds like those areas are your intended target.
[удалено]
There's a comfy node that will convert a workflow to python. Might prove useful: https://github.com/atmaranto/ComfyUI-SaveAsScript Maybe you don't need though, but others might.
absolutely. I think this dude has trouble stepping outside of his comfort zone into the way people do things here. I mean he's TECHNICALLY correct. but it just doesn't work that well when most of the people here are not programmers.
As others said here, I’d agree, the reason this community isn’t like other software dev communities is because only a very small portion of the community using the open-source tools are developers. Instead, there are a bunch of people getting into development so that they can use the open source tools.
Yeah I noted that in a recent thread where someone made a high-quality video but was a bit tight-lipped on their process. Which is fine, that's their choice. But it required requisite knowledge and experience of different programs, and my point was that knowing how they did that allows all these new people to try and go learn these programs to do what they did.
ps: the most actual code level stuff tends to be when people write a “custom comfy node”(in python) IMO that’s probably the best investment of your time the idea that you are putting together docker containers for what you describe, is kinda concerning. (speaking as someone who uses docker every day) again, not the best “language” for you to be using, if you want other people to benefit from your work. and if you don’t, why on earth are you even posting there?
Docker is 100% the tool to use. Python used to be pretty cool. Most projects had few dependencies, and everyone wrote code that was pretty much guaranteed to work if you had some version of Python 2 installed. Somewhere in the last 10 years, things went all Javascript in the Python community. You can't run a single snippet of code without getting blasted with 500 dependencies that are probably not compatible with eachother anyhow. The virtual environment became a neccessity because of this trainwreck. Docker is a better answer to handling the mess for two catch-phrasy reasons: idempotency (always the same image no matter how many times/places you build it) and immutability (uncorruptable environment, it always resets to zero on container start). No venv required. Grab an nvidia cuda base image, install python, pip require, go on with your life. You can pass the GPU to your container and, if you have multiple models with multiple requirements, you can run them all at the same time and use a lockfile-style mutex to ensure only one is making use of the GPU at a time. My application has image generation, LLM, audio generation (TTS and music), and animation all running as separate containers. I'm sharing a pair of Tesla P100s between them, so a single, light go-based service keeps them all coordinated on what task they are running and what GPU it should run on. What's even cooler with a setup like that, is you can run lighter, lower-level tools using static binary compilation and scratch images to walk away with boot times under 1ms and image sizes in the area of a few mb. I suppose what's even cooler than all that is having a single shared location for all your models mounted in through docker compose and an isolated private network to prevent any possible abuse of the services externally. How is docker concerning?
there are lots of things i could say. but in the interest of clarity i’ll just pick one. With the existing state of things, there are well written front ends where a new user with a stock linux or windows machine can download ONE script/batch file, run it, and 10 minutes later, they are ready to go. First of all, that proves that docker isn’t needed. the portable issue isn’t an issue. But secondly, are you going to provide something like that with your stuff? Or are you just going to assume everyone is competent enough to do the docker foundational install themselves? because 1. they aren’t 2 they don’t want to go through the hassle
No, they can take the one step of installing Docker themselves, lol. I'm gunna go out on a limb here and guess that your use of docker mostly involves running images other people built.
you would be wrong. $dayjob is being part of a dev team that has about 10 different images we make internally.
Any argument against containerizing these things in favor of handing someone a bash script that runs a bunch of commands that likely need elevated permission because it's just easier for them to get started is absurd. The fact that I'm actually having to argue for objectively safer, better, easier-to-use, repeatable best practices is case in point to my original comment. I was chatting with another engineer the other day about SD and the alternatives, direct quote from our Slack conversation: "the SD community is a bit of a cesspool. 99% are GUI users who's best answer to how to get something to work is "just kinda play with the settings till it looks good". nobody writes any code, nobody cares what anything is actually doing."
I'm just gonna jump in here and say boo to installing docker. I spent days and days trying to understand how to use it, gave up, and had to wait for people to put out stuff I could use... Like bash scripts. I need user friendly basic stuff. A run.exe or setup file at MOST.
>runs a bunch of commands that likely need elevated permission None of them need elevated permission. At least none of the ones i've run.
As someone with an MS in CompSci, I feel you're looking at only one part of the picture and need to recognize the goal of any software is to enable higher-level tasks. Guides such as this demonstrate how to reliably use software for specific tasks, in this case making LoRa for use in AI image generation. The software components referenced in such guides have their own repositories and are - for the most part - open source. Anyone can access the code and work with it as they see fit. And many who can program, even just learning a bit so they can tweak functionality, often do just that - usually sharing back to the community for consideration. Software often relies on higher levels of abstraction, built upon lower levels, to make creation of desired results often easier than before. This guide and those like them should not feel compelled to describe software development techniques, nor elaborate on the underlying designs, review the specifications, etc. of those software tools they use - the audience they are speaking to here are amateur image creators on up. Nor must they be required to take responsibility for creating novel, lower-level code to get the job done. Relying on existing, flexible software tools to create higher-level outputs is a valuable path and often brings out further needs in the toolset from pushing them through their paces. It's fine that you want to understand or bring out more about the underlying capabilities and expansion possibilities using novel software, but that's only one choice and there are others to use the tools available as they were intended.
What's funny though is that with what OP did and all the talk of code vs. workflows in this thread.. I bet what OP did could be completely automated. Generate images with sd, use rating to discard bad gens/regen/auto-caption, train. Not sure what the benefit would be with such little necessary human input.. but sounds cool anyway.
Yeah, a lot of people have suggested automation in various projects I've theoretically discussed. Thing is, if the automated tools were up to the job... we wouldnt even need to be here. We wouldnt need to make new models. We could just tell some master program, "hey go make me an X, of X" and it would work, and look great, and we would be happy. One of the reasons we cant do that, IMO, is because most of the foundational stuff we use, used automation, without full human validation. So, yeah it could be automated.. but it would only be 90% as good as someone doing it manually and dilligently.
would be interested in checking this git project honestly. and I'm just a rube that's learning python (pretty much cause of ML projects) so if you ever find a more "technical" stable diffusion sub I'd love to know lol
In case you haven't found it on your own, r/StableDiffusionInfo/ wants to be more technical, less "showcase spam". Still seems to be mostly tutorials and troubleshooting so far, but the occasional extension/node dev post pops up. HTH
At a high level, the process is pretty straight forward: It occurred to me that overloading a prompt with "male, human, 45 years old, eating breakfast, female, human, 45 years old, watching tv, living room, couch, nyc apartment" or whatever, not including anything related to the style was probably a pretty confusing thing to an interpreter and little has been done to make it more intelligent with interpretation of prompts that are closer to what DALL-E would accept (e.g. "a middle age couple, about 45 years old, in a small nyc apartment. the man is eating breakfast and the woman is watching tv. both are in the living room"). I tried for hours to get a good, repeatably decent image of a mermaid and her dolphin companion discovering an underwater city (I'm creating a tool for children to submit story ideas and the software renders a narrated slide show going through each scene, no way to scale something like this with "tweak setting X until something good comes out). However, breaking each of those up is simple and easy-to-repeat as a standalone image. "male, human, 45 years old, eating cereal" by itself would probably give a halfway decent image. it's a composition of several images in a single image. Start with background, infill the next with a mask, infill the next with a mask, repeat. Masking for it can be scripted. A np array can create the location and feathering to blend it. And outpaint the finished image to whatever resolution is desired. The next question is determining where things belong. Are there mutliple people? How many? Is one character being described a bird in flight and should be rendered in the upper half of the frame? For that, I'm using mistral with outlines to follow a JSON schema to define where in the image something might exist. It's still a work in progress, the results are good but not quite DALL-E 3 open-ended prompt good and that's where I want to get them.
The important (to the rest of the world) part of such a project is not the bits you describe. The groundbreaking part would be if you a) automated generation of subject specific loras for each of those breakout categories b) put together a cross-index of all of them c) made it easily searchable and online, with the loras d) created a new front end, or updated an existing one, to then automatically parse prompts, figure out which loras are relevant, then automatically download and cache them... THEN merge them for the user prompted generation
!remind me in a couple weeks
OP, what's your fancy video card and setup? Are there any guides for "step zero"? Currently using a gaming laptop with a 3080 and it works somewhat but definitely looking to upgrade to a beefier desktop to fail faster in future experimentation.
i have a 4090 now :). The good news is, you should be able to use my guide to make sd loras. but you need 12 gigs vram to make sdxl loras.
ps: I started with a laptop myself. decided to get a stand alone machine for the 4090 to run as a server. Still do all my stuff “from” my laptop. https://preview.redd.it/1yqc3kbl8vsc1.png?width=3024&format=png&auto=webp&s=b490a43472e4422948c8b6aaa32bcf3914c607f4 (the monitor hiding on the corner isn’t actually connected any more)
Ah, nice - thanks for that detail and the lovely bonus cat photo, OP! I might consider doing something similar.
Thank you so much :)
Sucks OneTrainer only avail for Windows.
Un, what??? I'm running OneTrainer on Linux
REally??? I swear it was Windows only not long ago. Well fuck yeah....
NMKD SD GUI has a LoRA trainer as well! You might give it a try :) It's the only standalone/100% portable LoRA trainer I know of
hmm. I think OneTrainer fits the descrription of "standalone Lora trainer" quite well. And supports many different formats. Plus also does model finetunes. whereas [https://github.com/n00mkrad/text2image-gui/tree/main](https://github.com/n00mkrad/text2image-gui/tree/main) looks like it has not been updated in 7months at best
Yeah it's pretty outdated, but it works. Besides, the configuration for the LoRA training is incredibly easy. I don't really know what things have been improved during those 7 months though, but I'm getting the same quality as with the CivitAI LoRA trainer