New Face MoCap Coming

124

Comments

  • wolf359wolf359 Posts: 3,837
    edited December 2020
    @shadowhawk1

    I wont defend little timmy's Apple.

    A trillion Dollar Corporation that makes luxury Brand product for affluent consumers.

    However you dont have to "replace" your current
    phone with a new Iphone 12 as there are people, over in the Reallusion community, using cheaper second hand IPhone X(10) models (without phone or Data service on them) to utilize live face capture plugin for Iclone 7.

    There are other means to create lipsync for a G8 figure
    such as Daz mimic live, or running the 32 bit version of Daz studio or even the free script to use the free Papagayo lipsync utility,

    However if you insist on a camera based facial mocap system for Daz studio/Genesis 8 , right now Facemojo for the Iphone is your only option...live with it.
     

    On the matter of "overpriced  apple applications that
    dont work on other systems"


    Well CG professionals on Windows dont seem to care
    As long as they have Autodesk,SideFX,Blender, UE4 etc

    so let apple have their litte closed garden
    We live in a world full of choices & alternatives. 

    Post edited by wolf359 on
  • wolf359 said:

    I wont defend little timmy's Apple.

    A trillion Dollar Corporation that makes luxury Brand product for affluent consumers.

    However you dont have to "replace" your current
    phone with a new Iphone 12 as there are people, over in the Reallusion community, using cheaper second hand IPhone X(10) models (without phone or Data service on them) to utilize live face capture plugin for Iclone 7.

    There are other means to create lipsync for a G8 figure
    such as Daz mimic live, or running the 32 bit version of Daz studio or even the free script to use the free Papagayo lipsync utility,

    However if you insist on a camera based facial mocap system for Daz studio/Genesis 8 , right now Facemojo for the Iphone is your only option...live with it.
     

    On the matter of "overpriced  apple applications that
    dont work on other systems"


    Well CG professionals on Windows dont seem to care
    As long as they have Autodesk,SideFX,Blender, UE4 etc

    so let apple have their litte closed garden
    We live in a world full of choices & alternatives. 

     

    Just purchased an Iphone X for £234.  Should be able to recoupe fair bit on my Pixel 3a.

     

  • shadowhawk1shadowhawk1 Posts: 2,200

    Kevin Sanderson said:

     

    You don't have to replace your cell phone. Just buy an unlocked used one. You're just using the TrueDepth camera and app. You can use an iPad with a TrueDepth camera as well.

     I guess that is easy for you to say if it helps justify your sales numbers doesn't it? Just how many people are going to be able to afford another cellphone that is only useful as a camera? I really would have thought that daz would be smarter than this and use a mocap system that is more friendly to their customers pocket.

  • would be nice if they actually sold a standalone a depth camera for use with computers as webcams with suitable software as people do sit at their desktops and do stuff too and it would be another way for Apple to squeeze some profit from the Windows users,

  • GordigGordig Posts: 10,190

    shadowhawk1 said:

    Kevin Sanderson said:

     

    You don't have to replace your cell phone. Just buy an unlocked used one. You're just using the TrueDepth camera and app. You can use an iPad with a TrueDepth camera as well.

     I guess that is easy for you to say if it helps justify your sales numbers doesn't it? Just how many people are going to be able to afford another cellphone that is only useful as a camera? I really would have thought that daz would be smarter than this and use a mocap system that is more friendly to their customers pocket.

     All Daz did was offer a product for sale in their store, unless you have evidence that they're preventing non-Apple-based mocap products from being released.

  • shadowhawk1shadowhawk1 Posts: 2,200

    wolf359 said:

    I wont defend little timmy's Apple.

    A trillion Dollar Corporation that makes luxury Brand product for affluent consumers.

    However you dont have to "replace" your current
    phone with a new Iphone 12 as there are people, over in the Reallusion community, using cheaper second hand IPhone X(10) models (without phone or Data service on them) to utilize live face capture plugin for Iclone 7.

    There are other means to create lipsync for a G8 figure
    such as Daz mimic live, or running the 32 bit version of Daz studio or even the free script to use the free Papagayo lipsync utility,

    However if you insist on a camera based facial mocap system for Daz studio/Genesis 8 , right now Facemojo for the Iphone is your only option...live with it.
     

    On the matter of "overpriced  apple applications that
    dont work on other systems"


    Well CG professionals on Windows dont seem to care
    As long as they have Autodesk,SideFX,Blender, UE4 etc

    so let apple have their litte closed garden
    We live in a world full of choices & alternatives. 

    @wolf359 Now that is the beauty of our world, I don't have to "live with it" as you say I can much more easily save my money by not purchasing a phone still costing in the $200 range that I don't want or need just to use it for a program that I will not buy because someone feels the need to play favorites. 

     

    I guess ol' P.T. Barnum was right.

  • shadowhawk1shadowhawk1 Posts: 2,200

    Gordig said:

    shadowhawk1 said:

    Kevin Sanderson said:

     

    You don't have to replace your cell phone. Just buy an unlocked used one. You're just using the TrueDepth camera and app. You can use an iPad with a TrueDepth camera as well.

     I guess that is easy for you to say if it helps justify your sales numbers doesn't it? Just how many people are going to be able to afford another cellphone that is only useful as a camera? I really would have thought that daz would be smarter than this and use a mocap system that is more friendly to their customers pocket.

     All Daz did was offer a product for sale in their store, unless you have evidence that they're preventing non-Apple-based mocap products from being released.

    OK, where are you seeing me accusing them of that? If that is all you have taken away from what I have said so far, please go back and reread the comments again. I am merely questiong the fact that daz is selling a program that requires the purchase of an item im most cases is still in the $150+ range that would have no other relevant need other than their program and they are passing it off like it is no big deal. Selling a program that uses Blender, Photoshop, Zbrush and others is one thing, they can all be used as stand alone products to create other things. requiring the pruchase of a cellphone simply too utilize its camera for their program is a completely different tangent all together.

  • wolf359wolf359 Posts: 3,837
    edited December 2020

    @shadowhawk1

    Daz accepted a Genesis 8 product from a PA that 
    requires some external hardware& software dependencies
    for that small minority of DS users who actually animate with Speaking Characters.


    Just as they released "bridge plugins" for the small miniorty of DS users who can afford $1200 USD a year for an Autodesk Maya subscription or whatever MAX costs per year

    or nearly $4000 USD for a C4D perpetual license
    or several hundred per year for a C4D sub.

    the facemojo plugin itself is $279 USD.

    and what does a decent Iray capable GPU costs these days??
    I would not know  as I am an AMD man using Blender cycles/EEVEE for rendering.

    Are you a Daz studio animator?? 
    how have you been doing your lipsync for your animations
    before Facemojo?

    If upgrading beyond the core features of Daz studio for animation is too expensive for people they can alway use Diffeo and migrate to Blender which has a facial motion tracking option that uses ordinary cell phone video.... after some tedious face rig setup of course.


    There are many ways to light Europe.

     

    Post edited by wolf359 on
  • shadowhawk1shadowhawk1 Posts: 2,200

    wolf359 said:

    @shadowhawk1

    Daz accepted a Genesis 8 product from a PA that 
    requires some external hardware& software dependencies
    for that small minority of DS users who actually animate with Speaking Characters.


    Just as they released "bridge plugins" for the small miniorty of DS users who can afford $1200 USD a year for an Autodesk Maya subscription or whatever MAX costs per year

    or nearly $4000 USD for a C4D perpetual license
    or several hundred per year for a C4D sub.

    the facemojo plugin itself is $279 USD.

    and what does a decent Iray capable GPU costs these days??
    I would not know  as I am an AMD man using Blender cycles/EEVEE for rendering.

    Are you a Daz studio animator?? 
    how have you been doing your lipsync for your animations
    before Facemojo?

    If upgrading beyond the core features of Daz studio for animation is too expensive for people they can alway use Diffeo and migrate to Blender which has a facial motion tracking option that uses ordinary cell phone video.... after some tedious face rig setup of course.


    There are many ways to light Europe.

     

    And every one of those programs you mentioned are a stand alone product that can be used to create other content apart from studio. I get the fact that occationally a product will need to have a secondary requirement to operate. What I am calling into light is the glib way in which daz is justifying the requirement to purchase an entire periferal and not a program. 

  • I soon gave up on OpenPose in Blender devil I did try

    I have no objection to DAZ selling this and dearly too because I simply won't be buying it, the iPhoneX or the app but if Apple wanted to tempt me with a depth Webcam that just might find it's way in a box to my door.

    I wouldn't buy an Android solution either, my phone is strictly a phone and South Australian Covid tracing with it's QR code won't even get me carrying it on me (fortunately you can still use a pen on a sheet of paper at stores)

  • Kevin SandersonKevin Sanderson Posts: 1,643
    edited December 2020

     

    shadowhawk1 said:

     I guess that is easy for you to say if it helps justify your sales numbers doesn't it? Just how many people are going to be able to afford another cellphone that is only useful as a camera? I really would have thought that daz would be smarter than this and use a mocap system that is more friendly to their customers pocket.

    I'm not selling it. I haven't even bought it yet. But it sure beats the crazy work involved for lipsync. Daz is just selling it for people who want it. If you want to do the advanced stuff you have to pay for it. Right now the only way to get this before the plugins and iPhone was to get iClone and its plugins which cost more and then figure out how to get the stuff back into DS.  I only have an Android phone. Nothing at this level for Android yet.

    Post edited by Kevin Sanderson on
  • IvyIvy Posts: 7,165
    edited December 2020

    This is my 2 bits & the resources I know about for lipsynch I have book marked in case I needed something for lipsynch in one of my projects in the future..

    For my needs  I just use daz studio 32 byte version that comes with Mimic or lipsynce. what ever they are calling it these days. I am still using it on 4.10 32bit.    i do not require much lipsynch in my animations to invest in something major.  But there is some pretty good alternatives

    If you have a xbox or use Interactive Kinetic on your PC?  if so you can use Kinetic for great facial capture. downside is  you need to buy the little face censors.  but it will work on windows and Apple and I believe Linux too.   But the censors are not all that cheap and the more details in your motions you want the more you will need of the censors. but you can get them here pretty reasonable. the censors come with compatible software. https://brekel.com/face_v2/

     and I know some users at Animations World https://www.awn.com/  ( one of my favorite animations resources sites BTW ) they are using  F-clone a markerless face captures software with some pretty good results. But its a pretty hefty price though.  its does not use censors or special hardware or require a iphone and it does work for daz studio as well.  works off any web cam or Kinetic capture as well.   But the cost is like $135 for the indy version which is what you will really need.& what keeps me from buying it    https://f-clone.com/

    Then you can always try   VUP on steam. I see a lot of people on YouTube seem to like it.  you sign up for steam and only pay for the time you use it.   I tried the demo of it and thought it a bit clumsy to use & did not transfer to or support DUF files either  Daz 32bit  lipsynch was better for my needs https://store.steampowered.com/app/1207050/VUPVTuber_Maker_Animation_MMDLive2D__facial_capture/

    Post edited by Ivy on
  • I have actually stopped using dialogue in most of my videos altogether devil

    just a few expressions and a saved mimic file for some facial movement 

    no actual speech

  • IvyIvy Posts: 7,165

    @WendyLuvsCatz

    I found  animations translate better in other countries if you can make your animation story without Speech , So I am with you on that

  • wolf359wolf359 Posts: 3,837

    @shadowhawk1

    It is not Daz that is preventing others from developing
    true depth cameras and software for face tracking or "playing favorites" to Apple.

    They are doing what the  rest of Character based CG industry is doing and levering what is already on the market to help sell Daz content

     

     

    And there are indie filmakers filming entire movies in 4K with their Iphones, so  this cynical narrative that an iphone X+ is only good for facial mocap is a bit self serving IMHO.


    @Ivy is correct ,as is Kevin sanders.

    The Moment you decide you need  to have animation features for G8,not already available  in Daz studio, be prepared to pay in terms of money or labor or both. 

  • wolf359wolf359 Posts: 3,837

    Not for Daz studio but...... 

  • shadowhawk1shadowhawk1 Posts: 2,200

    wolf359 said:

    @shadowhawk1

    It is not Daz that is preventing others from developing

    Honestly, is this all you are taking away from what I have said? At no point have I said or implied that daz is in any way preventing others from creating their own versions of this software. Starting to feel like I am playing cards with my sisters kids. Please stop reading into my words.I have stated multiple times that myu issue is selling software that requires the purchase of an iphone. Not once have I said they were limiting or stonewalling anyone from making something similar. 

    :facepalm:

  • TheMysteryIsThePointTheMysteryIsThePoint Posts: 3,027
    edited December 2020
    Deleted
    Post edited by TheMysteryIsThePoint on
  • This software is of interest to those who have, or might buy, a suitable iPhone. If you do not have and will not be purchasing a suitable iPhone then the software is not for you. Please drop the argument, it serves no purpose and is disrupting the thread for those for whom the software is relevant.

  • nonesuch00nonesuch00 Posts: 18,320

    This upcoming omniverse SDK & such apps derived from it coming from nVidia in the next year should make all those frustrations mostly a thing of the past.

  • Funny, I did a search on Digital Domain's site for Masquerade 2.0 and returned no results. Doesn't matter, the software probably costs thousands to license anyway.

  • mpyre_aoa said:

    Paying for an "expensive" iPhone isn't just about buying the phone. It's a lot cheaper than a lot of other facial animation tools. If you change your mindset from "buying an expensive apple phone" to "investing in my animation", then it may make more sense to do so. I've done everything that I could to stay away from Apple products. Been on Android since forever, and even tried the Windows phone (a very sad experience). But, when Apple came out with the iPhone, and it's facial animation feature, I saw it as a tool that I can use for personal, and business.

    I am budgeting for my iPhone upgrade from X-Max to 12-Max Pro.  Also budgeting for a new watch, iPad, and my first MacBook Pro.  I could get the watch and either the phone or iPad now if I want to (fully budgeted for them), but haven't spent the money yet because I think I'd rather spend the money on the MBP first; particularly if the M1 comes out for the 16" MBP in the next month or two.   My old phone, watch, and tablet are all working fine and I don't have the MBP yet, so there's that.  And really, I don't want it for the apps; it's because I want to write apps for iOS or iPad OS, and I need a Mac for that.

    But I'll probably go ahead and get Logic for it, plus a couple of other tools.  After all, why spend 3.5K and NOT be able to do stuff on it?

    But the i9 will still be my gaming laptop, and my old i7 desktop will continue to be my main creativity and gaming rig until I finally pull the trigger on an upgrade for it.

  • So the original video is VERY VERY GOOD.  Wow, we have enough realism to make characters more relatable.  Amazing, and I'm excited about this.

  • Kevin SandersonKevin Sanderson Posts: 1,643
    edited December 2020

    Subtropic Pixel said:

    So the original video is VERY VERY GOOD.  Wow, we have enough realism to make characters more relatable.  Amazing, and I'm excited about this.

     Yep if you mean the first video in this thread it was done by Free Nomon and his version is now called Facemotion with his original morphs (not in Face Mojo now) and the updates. He has a YouTube channel.

    Post edited by Kevin Sanderson on
  • wolf359wolf359 Posts: 3,837

    @nonesuch00

    I recently recevied an E-mail from NVIDIA
    about thier new "Omninverse" Digital content system.


    It appears to be a 3D content market based on the distributing /delivering assets in Universal scene description format(USD)
    that is supported by the major 3DCC's including Blender.

    I will not post any links as it will likely be a competitor for Daz/CC3/Poser as there will be no limitations on being able to get high quality human models ,created in high end programs, imported with perfect fidelity for use in your own projects via USD.


    I did not find any references to an SDK for developing facial mocap systems or similar on the website.

    Camera based facial mocap system are nice but present the same problem as Daz mimic live in that a single artist ,working alon,e will have a difficult time incorporating facial performances from remote voice actors.


    I know of one person ,in the Reallusion community, who has his remote voice actors record thier lines and send him the audio.

    He then has to match thier tempo/cadence while recording his own face capture with his Iphone (not an easy task BTW)

    and then line up the external voice tracks with the rendered Iclone Avatars In his nonlinear video editor.


    Blender already has a means to track facial motion onto a CG Character from any cell phone video but the custom setup is excruciatingly complex & tedious.

    I pray that someone creates a script to automate the face rig setup
    for any FBX imported Character


     

     


      

  • drzapdrzap Posts: 795
    edited December 2020

    wolf359 said:

    @nonesuch00

    I recently recevied an E-mail from NVIDIA
    about thier new "Omninverse" Digital content system.


    It appears to be a 3D content market based on the distributing /delivering assets in Universal scene description format(USD)
    that is supported by the major 3DCC's including Blender.

    I will not post any links as it will likely be a competitor for Daz/CC3/Poser as there will be no limitations on being able to get high quality human models ,created in high end programs, imported with perfect fidelity for use in your own projects via USD.


    I did not find any references to an SDK for developing facial mocap systems or similar on the website.

    Camera based facial mocap system are nice but present the same problem as Daz mimic live in that a single artist ,working alon,e will have a difficult time incorporating facial performances from remote voice actors.


    I know of one person ,in the Reallusion community, who has his remote voice actors record thier lines and send him the audio.

    He then has to match thier tempo/cadence while recording his own face capture with his Iphone (not an easy task BTW)

    and then line up the external voice tracks with the rendered Iclone Avatars In his nonlinear video editor.


    Blender already has a means to track facial motion onto a CG Character from any cell phone video but the custom setup is excruciatingly complex & tedious.

    I pray that someone creates a script to automate the face rig setup
    for any FBX imported Character


     

     


      

     

    Omniverse is very exciting technology.  Nvidia also plans to soon introduce software into Omniverse that will do AI lipsync like you never seen before (look up Audio2Face)!  And then there is this:  https://zivadynamics.com/beta/faces ;      It'll be expensive, for sure, but I'll be on it like white on rice.

    Post edited by drzap on
  • nonesuch00nonesuch00 Posts: 18,320
    edited December 2020

    wolf359 said:

    @nonesuch00

    I recently recevied an E-mail from NVIDIA
    about thier new "Omninverse" Digital content system.


    It appears to be a 3D content market based on the distributing /delivering assets in Universal scene description format(USD)
    that is supported by the major 3DCC's including Blender.

    I will not post any links as it will likely be a competitor for Daz/CC3/Poser as there will be no limitations on being able to get high quality human models ,created in high end programs, imported with perfect fidelity for use in your own projects via USD.


    I did not find any references to an SDK for developing facial mocap systems or similar on the website.

    Camera based facial mocap system are nice but present the same problem as Daz mimic live in that a single artist ,working alon,e will have a difficult time incorporating facial performances from remote voice actors.


    I know of one person ,in the Reallusion community, who has his remote voice actors record thier lines and send him the audio.

    He then has to match thier tempo/cadence while recording his own face capture with his Iphone (not an easy task BTW)

    and then line up the external voice tracks with the rendered Iclone Avatars In his nonlinear video editor.


    Blender already has a means to track facial motion onto a CG Character from any cell phone video but the custom setup is excruciatingly complex & tedious.

    I pray that someone creates a script to automate the face rig setup
    for any FBX imported Character


    Well give me a few minutes to find it...

    I got that same email & have installed it even though I as of yet been unable to to buy a GeForce RTX 30X0 series video card. The functionality I was talking about is the AI assisted animations they breifly demoed in a video. They had changed an animation in a medieval type scene with catapults in the video.

    NVIDIA Omniverse - A New Era of Collaboration and Simulation - YouTube

    They mention "Interactive robot teaching with 3D depth camera" (as part of the Isaac SDK). There are various 3D Depth cameras as cams for your computer USB ports but I'd recommend going to the nVidia developer forums and inquiring for as much details as you can get before buying one. 

    Anyway, as you saw, the "apps" that are available so far in the Omniverse is not the app that was mentioned in earlier nVidia video I was refering too. The app I remembered from the nVidia site page is called "Machinima" and apparantly hasn't been released yet.

    Machinima : A New Storytelling App For Gamers | NVIDIA

    or

    Machinima : A New Storytelling App For Gamers | NVIDIA (same link, sort of)

    To find that page in the future if you want, the search term I used was "omniverse ai pose" in either duckduckgo, bing, or google.

    You'll notice that the admittedly sparse details given by nVidia sound almost exactly like what your friend is doing but much simplified and streamlined.

    Post edited by nonesuch00 on
  • Im just over here hoping for full body motion capture / tracing from a movie.

  • Faux2DFaux2D Posts: 452

    The Genesis 3 & 8 Face Controls product already has everything that you need for facial motion capture. The Dynamixyz software itself is very expensive but there is a free trial version on their site. The main advantage is that you only need a recording of an actor's face to capture the motion capture data. Nothing fancy like a depth camera the iPhone has, even 720p works just fine. The main disadvantage is the learning curve, don't expect a plug-and-play style solution. I attached a file going into more detail for the Dynamixyz to Daz pipeline.

    If you are interested in motion capture I do recommend trying out the Dynamixyz to Daz pipeline, or at least reading the documentation + the video tutorial and going through one test session with Dinamixyz's Performer. This will make you better understand the challenges studios have (AAA studios) when dealing with facial motion capture.

    I also saw someone here mention Masquerade 2.0. By looking at the quality of animation from big productions, Masquerade 2.0 is a clear winner with Dnamixyz coming in second. As an example check out Avengers: Endgame. Thanos' facial animation was created with Masquerade, Smart Hulk's facial animation was created with Dynamixyz. Taking a more critical look at the scenes with the 2 of them, when the Hulk talks notice how shy the camera is to show his lip-sync. Thanos' lip-sync is top notch however, you can even do lip-reading on the animation.

    Depth cameras are great but that's not even half the battle of getting good mocap data. Dynamixyz for instance doesn't need them, neither does Masquerade. Both solve the depth issue by using two cameras at once.

    pdf
    pdf
    Dynamixyz to Daz DOCUMENTATION.pdf
    94K
  • drzapdrzap Posts: 795

    MoonCraft3D said:

    Im just over here hoping for full body motion capture / tracing from a movie.

    https://getrad.co/?utm_source=facebook&utm_medium=post&utm_campaign=anime&utm_content=unityviewer

Sign In or Register to comment.