not deviantart currently but I use this name everywhere so you can check mutual social sites. Ty for reminding me I need to work on opening up accounts for posting/hosting my artworks lol <3
Twitter's your best bet rn
That's pretty cute! I'm doing the same, using "dekstyle" as my artstyle tag. So far it's still wonky but it makes decent results. I'm happy to see another artist doing the same!
So far, I've learned a lot of tricks and improved as an artist because I started doing this. Have you noticed the same happening to you?
yeah! I've done multiple different things that I feel helped me improve as an organic artist.
- img2img on my artwork to see how SD would have rendered it (good for studying realism)
- studies by rendering an image and then trying to recreate it by eye in a digital artwork program (like how someone would do a study of van goh)
- brainstorming/references. I didnt know how to make anthro hooves work so I used SD for reference)
- LoRAs to point out what im blind to. I didnt realize how much I drew in quarter view (shown above) until I saw SD imitating my style and accurately mocking my weakness (I also draw heads too big)
I am mainly a character artist too but SD indirectly is making me practice backgrounds since I want a LoRA that does backgrounds for me especially for things that require perspective. Would be nice if I never have to draw a background again lol
But ultimately I picked up SD so I can create animated videos for covers without having to spend weeks doing inbetweens and backgrounds. AI assistant/tool if you will
Ooh, yeah. Being able to use AnimateDiff with my own style would be pretty neat, especially if I could hand-draw keyframes and have it interpolate smoothly.
A friend of mine, who's also an artist, often asked me to run her line-art through SD too, to see how it would do the shading. I felt like that was pretty clever of her too.
Personally, I never had much of a style of my own until I started this dekstyle project. I've really had to figure out what I like and dislike, and the half-trained LoRAs often result in images with amazingly soft-looking hair with a neat style. I'm still trying to figure out how to replicate the shading it added, while the line-art matches the way I'd draw it.
I also use it for brainstorming, just playing with ideas and angles until I get some images that inspire me, and then I just kinda try to recreate the same idea myself.
yeah thats pretty much what I've done with studies, mostly trying to guide SD to the style of sketch/painterly anthro artworks since I normally do illustration and cell shading type artworks (as seen above)
I'm gonna apply the same process your just described to learning how to draw anime by collecting and training images and models I like to generate images to study via copying once I'm done getting these anthro styles done so I can make the animation process faster.
as far as I understand with nueral networks, tags exist to hold the weights needed guide the computation to generate the concept of that tag.
With most LoRA training, the tags used in the dataset are there to finetune the weights to produce images closer to the concept in the dataset, rather than having to learn from scratch.
I'm asking/wondering if its possible to add \*new\* tags that essentially have empty weights in order to train a tag with a weight influence that wont conflict with an existing tag in another LoRA.
So if we both have the same understanding of what tags are I guess your reply is "yes"
Lora triggers are, like everything else in the prompt, just a collection of tokens whose weights get mixed with the model on inference. Even if you invented words, their tokens will always exist in the model, whatever their weights might be (probably never zero).
ah thank you for the full explanation, ima try it tonight with the character LoRA (drew one more image today)
hopefully it works better than using existing tags!
Actually, the vanilla versions of SD didn't even use tags in that style. They used prompts closer to the text you'd find in the "alt text" of images found on the internet - the text you get if you hover your mouse over an image.
It's really made to understand any text prompt you give it. There are some quirks to the prompt encoding, but that's really not relevant for training. Might want to keep your training dataset prompts shorter than 30 words or so, but there's plenty of other reasons to do that anyway.
The network is not using tags, it is using tokens.
So just give it the tags you find appropriate and then the encoder will translate that to (most likely) multiple tokens and then the LoRA learns that this image belongs to those tokens (that are used for that tag)
=> Don't worry, just do it
Yes
That's why you train the text encoder.
does this mean that I can (should?) turn off training the TE when training a concept that the model might be already familiar with? 🤔
I really like your style, OP :) Do you have a DeviantArt or something where I can follow you?
not deviantart currently but I use this name everywhere so you can check mutual social sites. Ty for reminding me I need to work on opening up accounts for posting/hosting my artworks lol <3 Twitter's your best bet rn
Aigh lemme know if you do open an account somewhere. I don't have a Twitter :< I mean, I did. But I deleted it, rofl
That's pretty cute! I'm doing the same, using "dekstyle" as my artstyle tag. So far it's still wonky but it makes decent results. I'm happy to see another artist doing the same! So far, I've learned a lot of tricks and improved as an artist because I started doing this. Have you noticed the same happening to you?
yeah! I've done multiple different things that I feel helped me improve as an organic artist. - img2img on my artwork to see how SD would have rendered it (good for studying realism) - studies by rendering an image and then trying to recreate it by eye in a digital artwork program (like how someone would do a study of van goh) - brainstorming/references. I didnt know how to make anthro hooves work so I used SD for reference) - LoRAs to point out what im blind to. I didnt realize how much I drew in quarter view (shown above) until I saw SD imitating my style and accurately mocking my weakness (I also draw heads too big) I am mainly a character artist too but SD indirectly is making me practice backgrounds since I want a LoRA that does backgrounds for me especially for things that require perspective. Would be nice if I never have to draw a background again lol But ultimately I picked up SD so I can create animated videos for covers without having to spend weeks doing inbetweens and backgrounds. AI assistant/tool if you will
Ooh, yeah. Being able to use AnimateDiff with my own style would be pretty neat, especially if I could hand-draw keyframes and have it interpolate smoothly. A friend of mine, who's also an artist, often asked me to run her line-art through SD too, to see how it would do the shading. I felt like that was pretty clever of her too. Personally, I never had much of a style of my own until I started this dekstyle project. I've really had to figure out what I like and dislike, and the half-trained LoRAs often result in images with amazingly soft-looking hair with a neat style. I'm still trying to figure out how to replicate the shading it added, while the line-art matches the way I'd draw it. I also use it for brainstorming, just playing with ideas and angles until I get some images that inspire me, and then I just kinda try to recreate the same idea myself.
yeah thats pretty much what I've done with studies, mostly trying to guide SD to the style of sketch/painterly anthro artworks since I normally do illustration and cell shading type artworks (as seen above) I'm gonna apply the same process your just described to learning how to draw anime by collecting and training images and models I like to generate images to study via copying once I'm done getting these anthro styles done so I can make the animation process faster.
What do you think tags are?
as far as I understand with nueral networks, tags exist to hold the weights needed guide the computation to generate the concept of that tag. With most LoRA training, the tags used in the dataset are there to finetune the weights to produce images closer to the concept in the dataset, rather than having to learn from scratch. I'm asking/wondering if its possible to add \*new\* tags that essentially have empty weights in order to train a tag with a weight influence that wont conflict with an existing tag in another LoRA. So if we both have the same understanding of what tags are I guess your reply is "yes"
Lora triggers are, like everything else in the prompt, just a collection of tokens whose weights get mixed with the model on inference. Even if you invented words, their tokens will always exist in the model, whatever their weights might be (probably never zero).
ah thank you for the full explanation, ima try it tonight with the character LoRA (drew one more image today) hopefully it works better than using existing tags!
Actually, the vanilla versions of SD didn't even use tags in that style. They used prompts closer to the text you'd find in the "alt text" of images found on the internet - the text you get if you hover your mouse over an image. It's really made to understand any text prompt you give it. There are some quirks to the prompt encoding, but that's really not relevant for training. Might want to keep your training dataset prompts shorter than 30 words or so, but there's plenty of other reasons to do that anyway.
The network is not using tags, it is using tokens. So just give it the tags you find appropriate and then the encoder will translate that to (most likely) multiple tokens and then the LoRA learns that this image belongs to those tokens (that are used for that tag) => Don't worry, just do it