Victoria 6 Base - Mimic Plug in for Carrara Pro 8.5 Experiments
Joepingleton
Posts: 746
View the animation on YOUTUBE.COM by clicking here.
This is my first test of the Mimic Plug in for Carrara Pro 8.5 with Victoria 6. I thought I would try it out since they were giving her away for free this weekend.
This version uses the Genesis 2 DMC file and has no extra modifications applied. I wanted to see how it worked with all the default settings. It looks like it will need a lot of tweaking.
The poem is "Time does not bring relief (Sonnet II)" by Edna St. Vincent Millay
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Exactly, as you said. I would like to see a better, re-worked facial animation with explanations of how how made the modifications. I would appreciate that very much.
That is the next step argus1000
My plan is to try some of the techniques I have been playing with on Genesis 1 figures and applying them to Genesis 2 figures. Once I get something that works, then I can pass along what I did to get the results.
I would like to invite anyone who is experimenting with this type of work to contribute to this thread, so we can all learn from each other. Plus this would give us a central place to use a reference.
View the animation on YOUTUBE.COM by clicking here.
This is my second test of the Mimic Plug in for Carrara Pro 8.5 with Victoria 6. This version is based of the default settings with a few tweaks.
I tried some of the techniques from my previous lip sync speed tests. These include adding 4 NLA tracks (brows, mouth, blinks and eye movement). The brows track has three NLA clips (Brow UP, Brow Down, Brow Squeeze). The mouth track has two NLA clips (Smile, Frown). The Blink Track on has one NLA clip used at an average of 5 times per second. The eye track has four NLA clips (eye right, eye left, eye up and eye down).
I also went through the Lip sync from Mimic and removed any NLA clips that lasted less than a frame to try to get rid of the Lip pops that occurred in the first test.
I think it looks much better than the default settings and only added less than a half hour to the editing time.
What do you think?
Much better. The brows and eyes are better. I still think the lips pop a little too much though. It's like the phonemes are coming too fast..
Yeah I am thinking that removing the spaces between phonemes may get rid of the pop
What do you mean: "removing the spaces between phonemes"?
When Mimic creates the phonemes NLA clips in the timeline there are spaces between them. My idea is to make all the phonemes NLA clips line up to blend between them better.
View the animation on YOUTUBE.COM by clicking here.
This is my third test of the Mimic Plug in for Carrara Pro 8.5 with Victoria 6. This version is based of the the test 2 modifications plus I removed the spaces between the phonemes NLA clips generated by Mimic in the timeline.
It seems better, what do you think?
So you change the size of the space between the phonemes, is that right? (pic 1)
I see see lip popping though. Have you though of making the phonemes editable (pic 2), opening them (pic 3) and changing the value of the visemes (pic 4)?
oddly and sort of on topic, I cannot get the supplied dmc file to work
I made one myself I put on sharecg using mimic pro, default dmc swapped out with G2F head morphs and Poser saved cr2 which works for me
the official Daz one will not.
Yes I did like you show in pic 1. The popping could still be coming from some of the short NLA clips. My plan was to get rid of those next.
I hadn't tried what you suggested in pics 2 -3. I was trying to keep the process as simple and quick as possible. But I think your on to something and will have to try it out.
That's strange since I only used the one that came with the free V6 this weekend. Are you using Mimic for Carrara to apply it?
In my opinion Mimic can be a little too animated and lip articulate. Part of this is the shadow buffers (you could adjust the shaders to make the tongue/teeth less visible), but also imo people's mouths just move less. Especially in animation..., I built up a figure from scratch by programming the Mimic settings myself, and I just repeated most of the vowels and consonants. The mouth became less distracting overall... I later went back and added a few more mouth shapes that didn't look right.... I suppose it could be different for each character, some more animated, some more stoic... or depending on the mood. Anyway in the end I felt Mimic was giving me too much and my eyes always watched the mouth, but once I had the mouth doing less it sort of just became more "natural" and less distracting... That's all I use Mimic for otherwise it feels really arbitrary and animatronic™.
I have been meaning to do some longer dialog conversations and use Puppeteer to do the expressions and looks, so there could be more of an organic blend across the expressions, rather than point to point of keyframe/NLA but to be honest I haven't done it beyond just experiments. I have a custom blink NLA that I drop in their own track, usually in pairs so the character does a blinkblink, or just before the eyes change focus or begins a sentence seems natural (for a woman)... Men I make blink slower and less often, but I don't know why it just seems right...
I haven't worked out what to do with eyes. Mimic just slides them around like dead marbles which is creepy, LOL. Sometimes they are part of the expression and sometimes they need to keep looking at something.... I need more Puppeteer practice probably.... I also have tried some tweeners to get eyes to flutter a little anime style (also big superwhite highlights look like anime for "I am having an emotion now"....
I also started a Group over at Cafe to hold some of my lipsync research. There's some articles there, and links to tutorials and similar...
http://carraracafe.com/group/?gid=3
Thanks for the input Holly its very informative.
I like your idea that Mimic produces too many clips. That is why I am slowly getting rid of them in these experiments. Just to see how that effects the overall feel. I was never been able to figure out how to do the stuff from the Dynamic Visemes link you posted in my other thread a long time ago.
Also Mimic wasn't animating her eyes in the tests I have done, other than the blinks. Maybe I am doing something wrong.
Puppeteer does look like a better way to get a more organic look. It's a totally different way of working than I am use to. But it's probably closer to real performances since you can just do a new take if you don't like the results.
I think any automated process will have its draw backs versus specific key framing by someone who knows what they are doing. I'm lazy, my experiments up to this point have been just trying to get a quick minimal acceptable lip sync in the shortest amount of time. All these tests take less than a half hour.
I'd love to see results from your experiments. You always have such great stuff. :)
WOW great link. Thanks for sharing the knowledge.
All my stuff was just bs tests, lol... I actually tried to switch over to CrazyTalk which is an interesting approach, but especially because now they are into selling all these preset expressions and idle animations.... It also has a system for recording puppeting, in multiple passes - the Carrara equivilent might be to control eyes and focus in one pass, smile and frown in another pass.... Each a separate puppeteer session going into a separate NLA track.... or something... Anyway it made me rethink how I could animate in Carrara more expressively.... Also there is no comparing the render quality, CrazyTalk is a really simplistic mesh... Carrara is a zillion times better.
All my stuff are BS tests too. Mainly figuring out what not to do. ;)
I never heard of crazy talk and will have to look into it.
I like your idea of using puppeteer since the Muppets are one of my favorite things and nobody ever questions there lip sync.
I've been basing my lip sync experiments off the ideas in this book "Stop Staring: Facial Modeling and Animation Done Right"
http://www.amazon.com/Stop-Staring-Facial-Modeling-Animation/dp/0470609907/ref=sr_1_11?s=books&ie=UTF8&qid=1390261757&sr=1-11&keywords=lip+sync
He hates automated lip sync and has a quick method that I have been trying to emulate. Its basically that there are three mouth positions (closed, open, wide). four brow positions (neutral, up, down, squeeze), five eye positions (neutral, right, left, up, down). I have been using mimic because I can't bring myself to do mouth charting.
View the animation on YOUTUBE.COM by clicking here.
This is my fourth test of the Mimic Plug in for Carrara Pro 8.5 with Victoria 6. This version is based of the the test 3 modifications plus I removed some more phonemes NLA clips that were only a frame or two long.
Did I remove too many?
That looks pretty good. Now I am less focused on the mouth and can see some of the eyebrow things... I still think her eyes are a little too solid and smooth, Maybe a little more EASE IN/OUT in the tweeners? idk, it is all pretty cool!
Monologs are hard, lol.
I think the mouth looks really good in that last one. I'm not sure about the eye movement. If it were me, I might video tape myself or some someone reciting either that poem, or just having their own monologue- Something they don't have to read as they're saying it and pay attention to what the eyes are doing.
I like having the eyes point at target helpers myself. The all you do is move the helpers to make the eyes move.
No, it doesn't seem like you removed too many phonemes. There is no more lip popping. Although she doesn't pronounce each of her syllables now, this is your best try yet.
I wonder if this lip popping is not related to the mesh resolution (or subdivision level). Like: more lip popping if there is not enough resolution?
Thanks for the feedback,
I was afraid I removed too many. It did get rid of the popping.
I was trying to use NLA's on the eyes to keep things simple and build a library I could use over and over. These test use them to try to make it look like she is thinking about what she was saying so they are timed to her words.
I used a point at target in my previous thread for genesis 1 [ http://www.daz3d.com/forums/discussion/37225/ ] and never liked what I was getting. But I may have been moving it too much in those versions. I like the idea of a target because it gives finer control of the movement. I'll try that next test.
I watched a lot of actors and they moved there eyes only when they wanted to make a point. Jack Nicholson is a master at it. Every movie I see of his now, all I can focus on is his eyes and brows. I try not to video tape myself as reference because I found out when doing facial motion capture I am a horrible actor. Plus it goes against my idea with these experiments of speed of production.
Monologs are hard, but if the resulting technique works on them, it should work with cuts and other people.
I think the lip popping has to do with the length of the phonemes, because they seem to go away when I get rid of the quick phonemes . I probably should pay more attention to the ones I am removing (I just removed any of the real short ones) and make sure she still has the start and end phonemes around the silences. Maybe also check that and emphasized words start and end correctly too. This could also possibly be fixed if I used phonetic text when I import the audio with mimic. In these test I only used the actual text file of the poem.
I also found out that it pays to normalize your sound file in a sound editor before you use it in Mimic. The lips movements become more accurate.
And I use the Truespel phonetic spelling (www.truespel.com) for my text file.
Thanks for the tips Argus1000
Too bad Mimic Pro for Carrara doesn't have the radio buttons "Delay" and "Animation interval" that Mimic Live for Daz Studio has. That would be as great help.
That would be nice. Have you done any lip sync with Mimic Live for Daz Studio?
It should import into Carrara. I use to use Mimic Pro standalone in DAZ and got good results. It was a lot of steps to get it into Carrara, so I have been trying the Carrara plug in.
No, I don't have LipSync Live for Daz Studio yet. But I used the Lip Sync standard feature in Daz Studio 32-bit and, although it gives good results in Daz Studio, it doesn't import well into Carrara. Better use Mimic Pro for Carrara.
I re-read the Carrara manual and it says that the more space you have between phonemes on the timeline, the less lip popping you're likely to have. I guess the trick is to shorten the phonemes so as to leave more space between them, and maybe change the place or value of the key frames INSIDE the phonemes themselves. I'm investigating this idea at the moment.
Thanks for the Information.
I thought I would post a link to all my mimic experiments here (both DAZ Studio and Carrara), just in case someone would find my other experiments interesting.
YouTube Lip Sync Experiments