Posed scenes to Stable Diffusion

BC Rice

I'm not sure if a thread exists yet on this (it probably does?), but with things like invoke Ai via Stable Diffusion (along with Midjourney), I thought it might be fun to finally cross that threshold that so many have been trying to cross using Daz for the past decade-plus: the ability to pose a scene and then render out a professional comic panel, and eventually a page. 
 

The best we've had access to in the past has been photoshop layering (or even tracing models) to middling results. With the intervention of Ai on these posed shots, it might be the advancement we've been waiting for. 
 

Maybe in the future we can get into some animation stuff with a combination of mocaps, but I thought we could start with some comic book panels and see what we can come up with. 
 

If you've already begun this process, feel free to post some images and links to your page work. I'm really excited to see what we can do. :)

SnowSultan

I've done a lot of testing in regards to this.   ;)

Midjourney/Nijijourney mix reference images with each other or the prompt, so it's difficult to get an exact copy of the pose from your 3D reference render. They're the best quality by far, but pretty unpredictable if you have a specific idea in mind. I put one of my interior renders through it (a medieval bedroom), and it pretty much made the same thing with or without the reference pic if I just described the bedroom in the prompt. The biggest problem with these AIs is that they're heavily censored - you can't even use 'pinup' or 'voluptuous' in your prompt, and they will reject any reference image that might even be remotely mature (even bikini images get rejected).

In my opinion, NovelAI's image generator does the best job at retaining the pose, hairstyle, clothing, or other elements of a 3D render reference image. It has a more limited dataset though, and I've found it fairly hard to make things easily other than sexy women (no censorship there, as it's based on Stable Diffusion).

Raw Stable Diffusion generators, like Automatic1111 or InvokeAI give you a lot more control, but I find it almost impossible to get really good results out of them. They're good for editing with inpainting and outpainting, but the Journeys leave SD in the dust as far as quality and composition. Custom models like Anything V3 can give really nice results, but they're also quite specific and you're not likely to get consistently good generations for all of the various elements of a comic.

 

I'm also interesting in seeing any comics done with AI, but I think we're still quite a ways off from being able to get reliable and pleasing results.