Loading...
> >

Is there a plan to generate 3D mesh?

yuyu.atemyuyu.atem Posts: 350
edited January 10 in Daz AI Studio

Hi! From what I understood, I have the impression that the IA can generate only 2D images. Could you tell me if it is planned to improve it to generate 3D mesh, please? And also to texture existing 3D mesh?
I think it would be a great features to add to the IA?

Thank you in advance!

Post edited by yuyu.atem on

Comments

  • FSMCDesignsFSMCDesigns Posts: 12,767

    I highly dount that any such tool will come from DAZ considering how far behind they are with normal AI, but there are AI tools that exists that can do this.

    try 3daistudio.com, or meshy.ai

  • yuyu.atemyuyu.atem Posts: 350
    edited January 8

    Oh, great, I didn't know them, thank nyou very much! laugh

    From your experience, could you tell me what are the strengths and weaknesses of 3daistudio.com and meshy.ai? Before to subscribe to when of them, I would like to be able to compare them, but I have the impression that meshy.ai's website is more "explicit" than 3daistudio.com's website...

    Thank you in advance!

    Post edited by yuyu.atem on
  • FSMCDesignsFSMCDesigns Posts: 12,767

    yuyu.atem said:

    Oh, great, I didn't know them, thank nyou very much! laugh

    From your experience, could you tell me what are the strengths and weaknesses of 3daistudio.com and meshy.ai? Before to subscribe to when of them, I would like to be able to compare them, but I have the impression that meshy.ai's website is more "explicit" than 3daistudio.com's website...

    Thank you in advance!

    I haven't tried either one, just have them bookmarked for future reference. Check YT for any videos they might have

  • yuyu.atemyuyu.atem Posts: 350

    Ah, ok, thank you very much!

  • wsterdanwsterdan Posts: 2,372
    edited January 10

    I haven't used 3daistudio yet, but I've generated a hundred or so meshes using Meshy.ai; overall, the meshes are usually fairly light on detail. That said, I am using it for taking simple sketches to 3D model form for printing as miniatures or running my DAZ Studio-rendered characters through a custome style model to give them the comic/toon look I like, then running *that* through Meshy for a printable figure.

    I usually start with a quick toon character render like this:



    Running this directly through Meshy gives something like this (it generates 4 samples at a time, this is just one):

    I usually run my DAZ renders through OpenArt with my custom style model to get something like this;

    I usually run a dozen or two through to get one or two that I like and if I need somethine specific (e.g. if I wanted the original logo to show instead of the modified one) I can update it before runnign it through Meshy.

    Meshy generates stuff like these:

    I'd donated my 3D printers to a local school when my wife and I downsized to a smaller house, but I specifially wanted to test the idea of using something Meshy to generate smal figures or gaming miniatures so I bought a small, cheap 3D printer for about $100 for my desktop. The prints have all been done using the Malyan Delta Mini printer (zero failures in 30+ prints):

    My plans for 2025 are to take the Meshy-generated figures into Nomad Sculpt on my iPad to add detail and extra features, print them and then clean and paint them. I might go with a multi-colour 3D printer instead and just print them in colour.

    DAZ_UB.jpg
    2048 x 2048 - 426K
    DAZ_UB-Mesh.jpg
    806 x 1340 - 89K
    OA_UB_01.jpg
    1536 x 1536 - 666K
    OA_UB_Mesh_01.jpg
    1282 x 1292 - 102K
    OA_UB_Mesh_02.jpg
    1282 x 1292 - 103K
    ManyPrints01.jpg
    1606 x 314 - 161K
    MeshTest01.jpg
    768 x 1024 - 119K
    MeshTest02.jpg
    768 x 1024 - 141K
    MeshTest03.jpg
    503 x 509 - 61K
    Post edited by wsterdan on
  • ArtiniArtini Posts: 9,623

    Great achievement, @wsterdan.

    It seems, like a lot of fun.

     

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412
    edited January 10

    OK for 3D printing but for rendering those meshy UV's are ...interesting

    A_medieval_man_in_ela_0103132212_refine.png
    2048 x 2048 - 4M
    A_medieval_woman_wear_0103131359_refine.png
    2048 x 2048 - 3M
    Post edited by WendyLuvsCatz on
  • wsterdanwsterdan Posts: 2,372

    WendyLuvsCatz said:

    OK for 3D printing but for rendering those meshy UV's are ...interesting

    For sure; I'm surprised that they actually work, but I've used some of the 3D characters generated this way as animated characters for game develpment, both as 2D and as 3D characters. The first ones I did I used OpenArt to convert the 2D to 3D, then animated them in Adobe's Mixamo. OpenArt dropped the image-to-3D option, but Meshy actually allows you to animate their generated characters in a sort of limited Mixamo format (limited only in the number of animations you can apply). You can always export the models and *then* animate them in Mixamo.
    As mentioned, my plan is to open them in my iPad's Nomad Mobile app (sort of a ZBrush knock-off, but dirt cheap) and repaint them as I add detail.

  • wsterdanwsterdan Posts: 2,372
    edited January 11

    A quick example, here's the artwork I fed OpenArt to make the 3D model:

    And here's the character with a spellcaster motion from Mixamo applied and uploaded as an animated GIF (which, of course, the forum doesn't support and only uploads a single frame); for my game, I'd plan on upgrading the texturing and motin and either using it as a 3D animated characater or as a number of animated GIFs. EIther way, with a little detailing and texturing in either ZBrushr or Nomad Sculpt I could get decent characters for a game.

    Front Spell Caster.gif
    556 x 640 - 496K
    Redhead Forward Cleaned.jpg
    2048 x 2048 - 682K
    Post edited by wsterdan on
  • yuyu.atemyuyu.atem Posts: 350

    wsterdan said:

    I haven't used 3daistudio yet, but I've generated a hundred or so meshes using Meshy.ai; overall, the meshes are usually fairly light on detail. That said, I am using it for taking simple sketches to 3D model form for printing as miniatures or running my DAZ Studio-rendered characters through a custome style model to give them the comic/toon look I like, then running *that* through Meshy for a printable figure.

    Thank you very much! If i understand well, I guess you used the text to 3D feature to generate them. Did you ever test the image to 3D? I generetated some beautiful pendents with Daz AI Studio (I show you a screenshot above), and I am looking for a way to genrate a 3D meshes from them, but from what you showed, I am affraid that the resulting mesh in meshy would have not enough details... Do you think it could be worth to try with meshy? Do you know some AI that could make the job in a better quality?

    Thank you in advance!

  • ArtiniArtini Posts: 9,623

    wsterdan said:

    WendyLuvsCatz said:

    OK for 3D printing but for rendering those meshy UV's are ...interesting

    For sure; I'm surprised that they actually work, but I've used some of the 3D characters generated this way as animated characters for game develpment, both as 2D and as 3D characters. The first ones I did I used OpenArt to convert the 2D to 3D, then animated them in Adobe's Mixamo. OpenArt dropped the image-to-3D option, but Meshy actually allows you to animate their generated characters in a sort of limited Mixamo format (limited only in the number of animations you can apply). You can always export the models and *then* animate them in Mixamo.
    As mentioned, my plan is to open them in my iPad's Nomad Mobile app (sort of a ZBrush knock-off, but dirt cheap) and repaint them as I add detail.

    Thanks for the mention of https://nomadsculpt.com/ which is also available for Android devices.

     

  • wsterdanwsterdan Posts: 2,372

    yuyu.atem said:

    Thank you very much! If i understand well, I guess you used the text to 3D feature to generate them. Did you ever test the image to 3D?

    All of the samples were image to 3D, none were text to 3D. I've only tested Meshy.ai and when it was available, OpenArt's image to 3D. As FSMCDesigns mentioned, there's 3daistudio.com, and others that I haven't tried. If you do a Google search (e.g. image to mesh tools) you'll find there's lots of others. For something like your jewelry sample, you might also try some simple 3D modeling or editing software that will extrude or displace a flat mesh using your image.

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412

    there is triposr you can run on your own PC, but I have had no luck installing it either in Automatic 1111 or Pinokio

  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412
    edited January 12

    another one

    https://github.com/Stability-AI/stable-point-aware-3d

    may be of interest to ComfyUI users

    I might try the Hugging Space demo later when on my PC (am on iPad by my fan hatinng summer)

    Post edited by WendyLuvsCatz on
  • Bunyip02Bunyip02 Posts: 8,747

    WendyLuvsCatz said:

    another one

    https://github.com/Stability-AI/stable-point-aware-3d

    may be of interest to ComfyUI users

    I might try the Hugging Space demo later when on my PC (am on iPad by my fan hatinng summer)

    Know the feeling with summer temps !!!

  • wsterdanwsterdan Posts: 2,372

    I feel for you guys. I thought about your suffering in the heat this morning while I shovelled snow when it was -23° C.

    laugh

  • yuyu.atemyuyu.atem Posts: 350
    edited January 13

    wsterdan said:

    All of the samples were image to 3D, none were text to 3D.

    Ah, ok, I misunderstood! Thank you very much!

    For something like your jewelry sample, you might also try some simple 3D modeling or editing software that will extrude or displace a flat mesh using your image.

    I would prefer something able to "understand" the object structure rather to only create an extrusion... For example, here, I would like that the pendent has an actual bird shape, not only a flat bird... (I don't know if what I amm saying is understandable...)

    Post edited by yuyu.atem on
  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412
    edited January 13

    https://huggingface.co/spaces/stabilityai/stable-point-aware-3d

    the SPAR3D demo

    just gave me curved cutouts with the images I tried

    Post edited by WendyLuvsCatz on
  • wsterdanwsterdan Posts: 2,372

    yuyu.atem said:

    wsterdan said:

    All of the samples were image to 3D, none were text to 3D.

    Ah, ok, I misunderstood! Thank you very much!

    For something like your jewelry sample, you might also try some simple 3D modeling or editing software that will extrude or displace a flat mesh using your image.

    I would prefer something able to "understand" the object structure rather to only create an extrusion... For example, here, I would like that the pendent has an actual bird shape, not only a flat bird... (I don't know if what I amm saying is understandable...)

    What you're saying is totally understandable. You don't want a simple extrusion of the outline resulting in a flat surface, you want the extrusion to use the source image's light and dark areas (e.g. if you were to convert the image to greyscale) extruded or "displaced" at matching heights to form a matching, detailed top surface. You can do this with displacement mapping, but the detail gets tricky because the colours in an image sample like your pendant don't necassarly line up with dark and light areas that would match low and high parts of the surfaces. I think it might be hit and miss as to what parts wind up high or low and I doubt you'd be happy with the results.

  • yuyu.atemyuyu.atem Posts: 350
    edited January 15

    wsterdan said:

    What you're saying is totally understandable. You don't want a simple extrusion of the outline resulting in a flat surface, you want the extrusion to use the source image's light and dark areas (e.g. if you were to convert the image to greyscale) extruded or "displaced" at matching heights to form a matching, detailed top surface. You can do this with displacement mapping, but the detail gets tricky because the colours in an image sample like your pendant don't necassarly line up with dark and light areas that would match low and high parts of the surfaces. I think it might be hit and miss as to what parts wind up high or low and I doubt you'd be happy with the results.

    Ah, ok, tahk you very much!

    WendyLuvsCatz said:

    https://huggingface.co/spaces/stabilityai/stable-point-aware-3d

    the SPAR3D demo

    just gave me curved cutouts with the images I tried

    Thank you very much! Maybe it comes from the settings, but the mesh doesn't seem in good qulatity... sad However, it "Remove background" tool is interesting: I have an image on which I tried to remove backagrond with DALL E from ChatGPT, and it didn't work... I tried to remove the same image background with SPAR3D, thanks to its "Remove background" feature, and the preview looks pretty good! Do you know if it is possible to download the actual background removed image? I could download only its preview...

    Thank you in advance!

    Post edited by yuyu.atem on
  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412
    edited January 15

    yuyu.atem said:

    wsterdan said:

    What you're saying is totally understandable. You don't want a simple extrusion of the outline resulting in a flat surface, you want the extrusion to use the source image's light and dark areas (e.g. if you were to convert the image to greyscale) extruded or "displaced" at matching heights to form a matching, detailed top surface. You can do this with displacement mapping, but the detail gets tricky because the colours in an image sample like your pendant don't necassarly line up with dark and light areas that would match low and high parts of the surfaces. I think it might be hit and miss as to what parts wind up high or low and I doubt you'd be happy with the results.

    Ah, ok, tahk you very much!

    WendyLuvsCatz said:

    https://huggingface.co/spaces/stabilityai/stable-point-aware-3d

    the SPAR3D demo

    just gave me curved cutouts with the images I tried

    Thank you very much! Maybe it comes from the settings, but the mesh doesn't seem in good qulatity... sad However, it "Remove background" tool is interesting: I have an image on which I tried to remove backagrond with DALL E from ChatGPT, and it didn't work... I tried to remove the same image background with SPAR3D, thanks to its "Remove background" feature, and the preview looks pretty good! Do you know if it is possible to download the actual background removed image? I could download only its preview...

    Thank you in advance!

    no but this one is pretty good

    https://www.remove.bg/ 

    I just use the lowrez one to create a mask in Gimp for the full sized images

    Post edited by WendyLuvsCatz on
  • yuyu.atemyuyu.atem Posts: 350
    edited January 15

    WendyLuvsCatz said:

    no but this one is pretty good

    https://www.remove.bg/ 

    I just use the lowrez one to create a mask in Gimp for the full sized images

    Great, it seems perfectly working, thank you very much! However, I am not sure to understand pricing: aparently, to download it in HD, I have to crete an account, but if I understand well, there is no free plan, is it that? And if i don't pay, I am allowed to download only one image? Or does it mean that I have a free image by month?

     

    I am surprise because here, it is written "free":

    Thank you in advance!

    Post edited by yuyu.atem on
  • WendyLuvsCatzWendyLuvsCatz Posts: 38,412
    edited January 15

    no idea about pricing 

    I only download the free lorez image

    just save the one generated in the browser 

    I use that in Gimp witn my original image

    import as a layer, resize layer to match

     select transparent bit by colour

    merge layers and cut

     

    Post edited by WendyLuvsCatz on
  • yuyu.atemyuyu.atem Posts: 350

    Ah, ok, tha,k you very much! laugh

  • wsterdanwsterdan Posts: 2,372

    You can also try the one-free tier on OpenArt.ai. Besides generating 2D images (they no longer have the image-to-3D option) there are edit tools, one of which is to remove the background, another is a very good Upscale option (better than my Topaz Gigapixel) and a third is to extend an image; there's also other tools. You only get 40 credits for joining, but you can get another 50 if you also join their Discord server.

Sign In or Register to comment.