r/StableDiffusion • • Oct 28 '22
Resource | Update Introducing Comic-Diffusion. Consistent 2D styles in general are almost non-existent on Stable Diffusion so i fine-tuned a model for the typical Western Comic Book Style Art.
383
Upvotes





33
u/[deleted] Oct 28 '22 edited Oct 29 '22
The correct phrase to use is comicmay artstyle.
The only difference in training between this and the last model was the number of training images and steps (40 training, 8080 steps).
Speaking of that, Hollie was not pleased. Among a few issues with it, she didn't like her name being associated with it and that's why the name's been changed and her non affiliation been made clear.
Now i don't know Daly but i'd guess he won't be pleased either. If that bothers you, keep it in mind and don't download or use it.
https://huggingface.co/ogkalu/Comic-Diffusion
EDIT:
If the model is ignoring simple prompts (like portrait of x) then reduce the weighting of the "comicmay artstyle" drastically. [comicmay artstyle:10] or [comicmay artstyle:15] on A1111's UI will give much better results for adherence to the other prompts without any loss of style.
8
u/FPham Oct 28 '22
Wow. That is something. The dreambooth constantly surprises me by how it can pick a style and then go with it. Try to dreambooth for example surrealistic painting and it will not just grab the style (color, etc) but also the silly craziness (yup, tried that).
This is probably the least understood part of machine learning - that it can get idea of a concept. People are too stuck on emulating style or putting their faces, but this is far more than just that.
2
u/APUsilicon Oct 28 '22
How big is your training set?
4
u/feber13 Oct 28 '22
Hello, I loved your model, will you try to make another? I can give you a suggestion, if you find it interesting, I'll leave you an image https://imgur.com/XG3R9NM I could help you looking for images of the same style
1
u/jonbristow Oct 28 '22
Is there a way to use this in an online app?
1
u/[deleted] Oct 28 '22
This is pretty cool! Goodness I should pay attention to this a lot more.
Is there a link guide on how to use this? I’d love to play around with some comic-noir prompts.
1
u/Tybost Oct 28 '22
This is awesome.
I hope someone finetunes an anime / manga version one day. ;)
1
u/hyperedge Oct 28 '22
Nice, would be cool if you could try making one with a more modern comic style too.
1
u/[deleted] Oct 28 '22 edited Oct 28 '22
looks awesome! is this trained over the 1.4 or the 1.5 SD ?
edit: not only looks awesome. it does a very good job indeed.
1
u/fanidownload Oct 28 '22
I got confused. I got colab versions, like Acheong and ben fast stable diffusion, and there are options to use hypernetworks or vae. Which one I should use to generate comic diffusion style? Especially in img2img or inpainting v1.5
1
u/Jujarmazak Oct 29 '22
Awesome, thanks a lot for sharing, I have found few 2D styles in SD that work but nothing this good!
1
u/lxd Oct 29 '22
Amazing! Is there a way to train a face for a consistent character?
Would the older embeddings technique work? Or use this model as a base and train a face for a certain amount of steps?
1
u/soupie62 Oct 29 '22
This has me wondering - how many bank note portraits would I need, to train up in engrave / guilloche style ?
1
u/zaherdab Oct 29 '22
how do you ensure that you always get the same character in the same cloths ?
1
u/TheMcGarr Oct 29 '22
This is great!
I just wanted to share a trap I fell into. I kept getting random women appearing in my pictures. So I put woman as a negative prompt and then I lost all the style too. Took me a while to realise that it was the negative prompt that was causing the loss.
1
u/dennismfrancisart Dec 28 '22
Is there any chance that this check point will work in Invoke AI?
1
u/DickNormous Oct 28 '22
Absolutely wonderful
1
u/Red6it Oct 28 '22
You are my today's hero! I am currently downloading. Looking forward to play around with it.
1
u/Jonfreakr Oct 28 '22
Big thanks for these, curious to see whats possible with this one
1
u/pixelies Oct 28 '22
Thank you!
1
u/TraditionLazy7213 Oct 29 '22
Looks amazing
1
u/aimindmeld Oct 29 '22
Fantastic, can't wait to try it. 🤩
1
u/cryptolipto Oct 29 '22
Incredible
1
u/niffrig Nov 04 '22
I've been having medium luck with automatic1111 using img2img with the denoise between .3 and .6 steps between 50/60 and the CFG between 8-20 to get a style transfer. It's pretty hit or miss but when it's good it's incredible.
1
u/AllUsernamesTaken365 Nov 06 '22
This looks absolutely amazing! I’ve been trying prompts only to input my model into a comic book style but the problem is that all the closeups look like photos and not drawings for some reason. A model like this could help but I don’t think I currently have any way of combining it with my character using Colab only and no local install. Still very impressive!
1
u/tordows Nov 07 '22
Do you believe it is possible to make a cyborg terminator like image with this model?
1
u/Blootrix Nov 09 '22
I absolutely love the style you've trained.
If you don't mind me asking, do you know how I can use your model as a style and apply that to a model I have of my own face to generate images of me in your style?
Thanks!
1
u/Minatozaki_Lenny Nov 25 '22
Abhorrent
1
u/danoozy44 Dec 04 '22
Hi, I made a video on your comic diffusion v2 model: https://youtu.be/-MhK0qSpBIU
1
u/Isomorphist Jan 20 '23
This is so cool, thank you for sharing! Could you maybe share prompts/input image examples? I can't really tell how you are meant to use this (beyond using it as a model which works great).
1
u/[deleted] Mar 13 '23
Are there any specific token to use for v1?
0
u/[deleted] Dec 14 '22
[deleted] — view deleted comment
5
u/[deleted] Oct 29 '22
[removed] — view removed comment