Yea, I turned off limits and then it was pretty gross, lol! Twisted really bad. Its hard to tell orientation when they are that twisted. Anyway, at least now I know how to use the interface. It took a few tries to realize I had to click the figure in the scene tab and not the stick figure prior to transfer.
Is the download updated with the changes or not yet? I was going to grab it for a test.
here's
mcjKinectKitSecretPreBetaVersion_v2
i still have to make tests on different figures and on DS1,2,3,4
1 - the collar-bone animations are now excluded from the transfer to the figure
2 - the server now writes mocap archive files(txt) in its own folder instead of in DazStudio's folder
3 - the “Seated” mocaps are still not really supported
4 - minor DS3 compatibility enhancement
5 - the left arm over-twisting issue is possibly fixed
6 - long operations now display a progress bar with interrupt-ability
it's important to avoid installing the v2 drivers which are for the new kinects
after installing this, i unplugged and re-plugged the kinect
Windows detected thr presence of the kinect
it installed a bunch of drivers, maybe 8 drivers, it took a few minutes to complete
when you start the mcjKinectServer001.exe application, there's should be a sparkling red light coming out of the kinect,
i would guess that's an infra-red laser that peppers the room with infra-red dots and makes skeleton detection possible
if you stand, say, at least 8ft from the kinect you should be detected and appear onscreen
note that in there's also the "Kinect for Windows Developer Toolkit v1.8"
which is not required, and is quite big to download, but it contains
sample applications ( including 3D scanning! ) http://www.microsoft.com/en-ca/download/details.aspx?id=40276
it's important to avoid installing the v2 drivers which are for the new kinects
after installing this, i unplugged and re-plugged the kinect
Windows detected thr presence of the kinect
it installed a bunch of drivers, maybe 8 drivers, it took a few minutes to complete
when you start the mcjKinectServer001.exe application, there's should be a sparkling red light coming out of the kinect,
i would guess that's an infra-red laser that peppers the room with infra-red dots and makes skeleton detection possible
if you stand, say, at least 8ft from the kinect you should be detected and appear onscreen
note that in there's also the "Kinect for Windows Developer Toolkit v1.8"
which is not required, and is quite big to download, but it contains
sample applications ( including 3D scanning! ) http://www.microsoft.com/en-ca/download/details.aspx?id=40276
Thanks for your quick and well explained response I have reinstalled using your method now works great thanks for your help and your program this will help me a lot and save me a lot of time
Very nice progress, thanks for all the work you have done in making the Daz community what it is today. We are very grateful.
thanks!
i hope this will help me do more youtube videos where Amy presents ( mostly stand up ) the old and new scripts
animating a stand-up animation can take days
so i wont need to re-use sword-wielding or boxing BVHs like in
i dont know yet if it will be combined with skeletal mocap
it's facial tracking, i dont know if it will ( or can ) be combined with skeletal mocap
so you could do facial animations in a second pass
the set of facial features tracked ( by the SDK's libraries )
is a bit limited ( see list below )
the eyelids didn't work in my tests but
maybe it's a question of setting the kinect resolution
var coef_UpperLipLift = i++; // 0.110862
var coef_JawDrop = i++; // 0.179137
var coef_MouthStretch = i++; // 0.197998
var coef_BrowLower = i++; //-0.220958
var coef_MouthCornerLift = i++; // 0.069177
var coef_OuterBrowRaiser = i++; // 0.098132
var coef_UpperEyeLid = i++; // 0.000000
var coef_LowerEyeLid = i++; // 0.000000
Hmm.......I was somehow unsubscribed? Looking good! Beta 3 downloaded, and no plans this evening, so I should be able to give it a good testing! :)
Did you have a process that you have to smooth out any herky-jerky animation? (using some of your other scripts?) If so, i would love to see your steps.
Hmm.......I was somehow unsubscribed? Looking good! Beta 3 downloaded, and no plans this evening, so I should be able to give it a good testing! :)
Did you have a process that you have to smooth out any herky-jerky animation? (using some of your other scripts?) If so, i would love to see your steps.
for moderately short animations i also use mcjCycleFilter, either the "Filter" (Fourier based) or the "smooth" "Synth"
using mcjCycleFilter's "Current" "Synth" you can reduce the swing of a rotation or morph
note that using mcjCycleFilter's Filter you can turn a somewhat-looping animation into a perfectly looping animation https://sites.google.com/site/mcasualsdazscripts/mcjcyclefilter
here you can see the X and Y gradients of the image
( i actually used a daz script to test this ! but this will be done in the KinectServer, way faster )
OK, so I am testing tonight...when I just run the Server (beta 3) standalone write to file, then I import it, it reads as "seated"? I dont have seated checked, and I can see the whole figure, but it reads in that way via the client? Any ideas?
OK, it seems to be the beta3 Client, because I can load the Beta 2 Client and it reads all the recorded mocap as "not seated", as expected.
Another thing I noticed is the thigh bends don't seem to match what I do, but I notice that on the stickman, so maybe that's a Kinect thing. When I sit down, the thighs are only slightly slanted out, like I am leaning back on something.
Its fun though! :) I noticed some limitation of Kinect are spinning around...and turning sideways
OK, so I am testing tonight...when I just run the Server (beta 3) standalone write to file, then I import it, it reads as "seated"? I dont have seated checked, and I can see the whole figure, but it reads in that way via the client? Any ideas?
OK, it seems to be the beta3 Client, because I can load the Beta 2 Client and it reads all the recorded mocap as "not seated", as expected.
Another thing I noticed is the thigh bends don't seem to match what I do, but I notice that on the stickman, so maybe that's a Kinect thing. When I sit down, the thighs are only slightly slanted out, like I am leaning back on something.
Its fun though! :) I noticed some limitation of Kinect are spinning around...and turning sideways
thanks i'll look at those issues
in version 3, the moment the kinect reports a loss of track of the hip node,
it treats the mocap as "seated"
i'll change this in the next version
the microsoft drivers really dont do well with turn-arounds
eventually, i want to re-write the server but based on OpenNI
because it seems to cope with turn-arounds https://www.youtube.com/watch?v=sWADWpHIubI
OK, so I am testing tonight...when I just run the Server (beta 3) standalone write to file, then I import it, it reads as "seated"? I dont have seated checked, and I can see the whole figure, but it reads in that way via the client? Any ideas?
OK, it seems to be the beta3 Client, because I can load the Beta 2 Client and it reads all the recorded mocap as "not seated", as expected.
Another thing I noticed is the thigh bends don't seem to match what I do, but I notice that on the stickman, so maybe that's a Kinect thing. When I sit down, the thighs are only slightly slanted out, like I am leaning back on something.
Its fun though! :) I noticed some limitation of Kinect are spinning around...and turning sideways
i did a quick non-seated test and it seems to work
note that i normally check-mark the server's "delayed" check-box,
so when the capture really starts i'm fully in view
thanks! its funny, I was just looking at the OpenNI stuff after I was testing this. I could not get it working with MMD yet, so I uninstalled it all, and am going to try from scratch again tomorrow. I have some MMD VMD's that I ran through LiveAnimation to BVH and then imported, and they came out real good in DS.
thanks! its funny, I was just looking at the OpenNI stuff after I was testing this. I could not get it working with MMD yet, so I uninstalled it all, and am going to try from scratch again tomorrow. I have some MMD VMD's that I ran through LiveAnimation to BVH and then imported, and they came out real good in DS.
it's quite possible that the Microsoft drivers interfere with the OpenNI (primesense?) ones
though i think i saw someone who was using the MS SDK plus the OpenNI one plus the OpenCV all at the same time
still studying options for face tracking
the 100 2D points tracked by the kinect ( displayed here in DS3 using an mcjBillboards120 figure by the way )
i'm trying to see is i can get more motion measurements than the 6 animation/action units supplied by the SDK
i will probably not support gaze/iris tracking
but i'd like to add blink/eyelids tracking ( instead of faking eyelid animation based on the eyebrows and jaw )
Comments
Yea, I turned off limits and then it was pretty gross, lol! Twisted really bad. Its hard to tell orientation when they are that twisted. Anyway, at least now I know how to use the interface. It took a few tries to realize I had to click the figure in the scene tab and not the stick figure prior to transfer.
so today i will make the figure posing more intelligent
for example, forbid foreArm side-side rotation
and probably the shin side-side rotation
maybe i'll borrow some of my mcjAutoLimb code for this
fig 2 that's more like it !
there's still minimal side-side rotations on the arms but i'll leave it like that for now
at least the arm is not over-twisted anymore
----
issue which will be fixed: figure is positioned 154 cm too high
----
figure 3 for the benefit of anyone wanting to write their own Client
the format of the mocap .txt files
progress report
done
- (hopefully) fixed the over-twisted left arm issue
- progress bar during long operations with the possibility of interrupting it
- warns user if no figure is selected
todo
- bend-offset on the feet and neck
- warn user that locked bones or limits may interfere with poses
- test with
- Norm toony ( the zero-posed arms differ from the norm ),
- TheGirl ( zero-posed legs differ from the norm )
- Victoria 4
- Genesis
Is the download updated with the changes or not yet? I was going to grab it for a test.
here's
mcjKinectKitSecretPreBetaVersion_v2
i still have to make tests on different figures and on DS1,2,3,4
1 - the collar-bone animations are now excluded from the transfer to the figure
2 - the server now writes mocap archive files(txt) in its own folder instead of in DazStudio's folder
3 - the “Seated” mocaps are still not really supported
4 - minor DS3 compatibility enhancement
5 - the left arm over-twisting issue is possibly fixed
6 - long operations now display a progress bar with interrupt-ability
https://sites.google.com/site/mcasualsdazscripts4/mcjkinectkit001
little problem here Aiko's root bone is about at the same location as Aiko's hip bone
so that will be fixed somehow in the official 001 release
in the meantime, you could move the root bone approximately 144 cm down along the Y Axis before transfering the pose(s)
------
Fig 2, Fig3 - the official release will include an "offset" to tweak the figure's hip position
Fig 4 - Neck Bend adjustment because the kinect's skeleton's neck is always bent forth
HI server/Kinect not detecting actor the green light is on .on the Kinect ,server states waiting actor any suggestions
is it an 'old' xbox 360 type of kinect?, because my 'server' application only supports those
also you need the special power/USB adapter ( and it must be plugged .... you never know :) )
on mine when the kinect is not in use, i see the green light turn on/off about twice per second
i suspect this only happens because the drivers are properly installed
for my setup, my Windows had .NET 4.x installed, which is a pre-requisite for the installation of the drivers
then i installed the Kinect for Windows drivers,
the "Runtime" installer installs only the drivers needed to make applications work
http://www.microsoft.com/en-ca/download/details.aspx?id=40277
the full sdk is for programmers and contains libraries and header files and the Runtime installer
http://www.microsoft.com/en-ca/download/details.aspx?id=40278
it's important to avoid installing the v2 drivers which are for the new kinects
after installing this, i unplugged and re-plugged the kinect
Windows detected thr presence of the kinect
it installed a bunch of drivers, maybe 8 drivers, it took a few minutes to complete
when you start the mcjKinectServer001.exe application, there's should be a sparkling red light coming out of the kinect,
i would guess that's an infra-red laser that peppers the room with infra-red dots and makes skeleton detection possible
if you stand, say, at least 8ft from the kinect you should be detected and appear onscreen
note that in there's also the "Kinect for Windows Developer Toolkit v1.8"
which is not required, and is quite big to download, but it contains
sample applications ( including 3D scanning! )
http://www.microsoft.com/en-ca/download/details.aspx?id=40276
well the hopes are high ( for the 2nd or 3rd time i know ), for a stable-ish release this weekend
is it an 'old' xbox 360 type of kinect?, because my 'server' application only supports those
also you need the special power/USB adapter ( and it must be plugged .... you never know :) )
on mine when the kinect is not in use, i see the green light turn on/off about twice per second
i suspect this only happens because the drivers are properly installed
for my setup, my Windows had .NET 4.x installed, which is a pre-requisite for the installation of the drivers
then i installed the Kinect for Windows drivers,
the "Runtime" installer installs only the drivers needed to make applications work
http://www.microsoft.com/en-ca/download/details.aspx?id=40277
the full sdk is for programmers and contains libraries and header files and the Runtime installer
http://www.microsoft.com/en-ca/download/details.aspx?id=40278
it's important to avoid installing the v2 drivers which are for the new kinects
after installing this, i unplugged and re-plugged the kinect
Windows detected thr presence of the kinect
it installed a bunch of drivers, maybe 8 drivers, it took a few minutes to complete
when you start the mcjKinectServer001.exe application, there's should be a sparkling red light coming out of the kinect,
i would guess that's an infra-red laser that peppers the room with infra-red dots and makes skeleton detection possible
if you stand, say, at least 8ft from the kinect you should be detected and appear onscreen
note that in there's also the "Kinect for Windows Developer Toolkit v1.8"
which is not required, and is quite big to download, but it contains
sample applications ( including 3D scanning! )
http://www.microsoft.com/en-ca/download/details.aspx?id=40276
Thanks for your quick and well explained response I have reinstalled using your method now works great thanks for your help and your program this will help me a lot and save me a lot of time
Very nice progress, thanks for all the work you have done in making the Daz community what it is today. We are very grateful.
thanks!
i hope this will help me do more youtube videos where Amy presents ( mostly stand up ) the old and new scripts
animating a stand-up animation can take days
so i wont need to re-use sword-wielding or boxing BVHs like in
https://www.youtube.com/watch?v=7NGVSbhLd_Q
since many don't do animation, i'll try to add snapshot / posing functionnality
good news i think i fixed most problems with the "seated mode" mocaps
once i complete this and take care of cases where the kinect loses track of some bones
i will release the official version 001 !
impending release of the third secret pre-beta version ... today Sunday evening
** Video ** ** Video ** ** Video ** ** Video ** ** Video ** ** Video ** ** Video **
test of the newly implemented "seated" mode ( it was very shake, so i only kept 10 frames per second )
https://www.youtube.com/watch?v=wnw1wqfgAwQ
it's a well kept secret but i uploaded
mcjKinectKitSecretPreBetaVersion_v3
i still have things to test later tonight
note : by default the Hip Offset setting is 120 (cm) , this worked well for the Aiko3 figure under DS3
but for Genesis2/DS4 it needs to be 0
note also, it's better to "set Limits off" on the figure before transfering the animation from the stickMan
https://sites.google.com/site/mcasualsdazscripts4/mcjkinectkit001
1 - fixed : collarbone now excluded
2 - fixed : mocap files written in same folder as server's .exe
3 - fixed : the “Seated” mocaps supported
4 - fixed : minor DS3 compatibility
5 - fixed : over-twisted arms issue
6 - new : bend offsets for the neck and feet
7 - new : option to lock feet side-side and feet twist
todo : improve manual
todo: better handling of bad kinect poses
todo: keep or discard the list of bones participating in the pose transfer .... for now it's purely decorative
note : i improved the kinectkit manual a little bit
---
teaser .... that's facial motion capture
video !! https://www.youtube.com/watch?v=vHd2wHhu274
i dont know yet if it will be combined with skeletal mocap
it's facial tracking, i dont know if it will ( or can ) be combined with skeletal mocap
so you could do facial animations in a second pass
the set of facial features tracked ( by the SDK's libraries )
is a bit limited ( see list below )
the eyelids didn't work in my tests but
maybe it's a question of setting the kinect resolution
var coef_UpperLipLift = i++; // 0.110862
var coef_JawDrop = i++; // 0.179137
var coef_MouthStretch = i++; // 0.197998
var coef_BrowLower = i++; //-0.220958
var coef_MouthCornerLift = i++; // 0.069177
var coef_OuterBrowRaiser = i++; // 0.098132
var coef_UpperEyeLid = i++; // 0.000000
var coef_LowerEyeLid = i++; // 0.000000
Really liking the work you're doing. It's much appreciated! Hope my little utils can measure-up someday.
Hmm.......I was somehow unsubscribed? Looking good! Beta 3 downloaded, and no plans this evening, so I should be able to give it a good testing! :)
Did you have a process that you have to smooth out any herky-jerky animation? (using some of your other scripts?) If so, i would love to see your steps.
i use mcjDecimate, it reduces the number of keyframes,
https://sites.google.com/site/mcasualsdazscripts/mcjdecimate-for-ds-12-3
for moderately short animations i also use mcjCycleFilter, either the "Filter" (Fourier based) or the "smooth" "Synth"
using mcjCycleFilter's "Current" "Synth" you can reduce the swing of a rotation or morph
note that using mcjCycleFilter's Filter you can turn a somewhat-looping animation into a perfectly looping animation
https://sites.google.com/site/mcasualsdazscripts/mcjcyclefilter
using the SceneGraf plugin, you can manually move or delete keyframes
For DS3 https://sites.google.com/site/mcasualsdazscripts/scenegraf
For DS4 https://sites.google.com/site/mcasualsdazscripts2/mcjscenegrafds45
for the legs ( keep feet in place ) i use mcjAutolimb or mcjAutoLimb2014
https://sites.google.com/site/mcasualsdazscripts/mcjautolimb
https://sites.google.com/site/mcasualsdazscripts3/mcjautolimb2014
often after "pinning" feet in place using mcjAutoLimb
i force feet to remain oriented the same way using mcjkeeporient
https://sites.google.com/site/mcasualsdazscripts/mcjkeeporient
also often used for animation work
https://sites.google.com/site/mcasualsdazscripts/setinterpolation-for-ds2-3
https://sites.google.com/site/mcasualsdazscripts4/mcjmirrorlimbanim
https://sites.google.com/site/mcasualsdazscripts2/mcjlookatthecamera
https://sites.google.com/site/mcasualsdazscripts2/mcjholdo
https://sites.google.com/site/mcasualsdazscripts/mcjmaketarget
https://sites.google.com/site/mcasualsdazscripts4/mcjrepeataction
https://sites.google.com/site/mcasualsdazscripts3/mcjparent
http://www.inb.uni-luebeck.de/publikationen/pdfs/TiBa11b.pdf
that's a German university paper about gaze / iris tracking
i may use this method to add eye tracking to the kinect's head/face tracker
here you can see the X and Y gradients of the image
( i actually used a daz script to test this ! but this will be done in the KinectServer, way faster )
and someone here implemented it https://www.youtube.com/watch?v=aGmGyFLQAFM
Very nice, this should definitely improve my old head tracking script. looking forward to the development.
Thanks for sharing.
if you have a webcam
there's an interestin in-browser ( javascript ) gaze tracking demo here
http://aw204.host.cs.st-andrews.ac.uk/camgaze.js/examples/smiley_face.html
OK, so I am testing tonight...when I just run the Server (beta 3) standalone write to file, then I import it, it reads as "seated"? I dont have seated checked, and I can see the whole figure, but it reads in that way via the client? Any ideas?
OK, it seems to be the beta3 Client, because I can load the Beta 2 Client and it reads all the recorded mocap as "not seated", as expected.
Another thing I noticed is the thigh bends don't seem to match what I do, but I notice that on the stickman, so maybe that's a Kinect thing. When I sit down, the thighs are only slightly slanted out, like I am leaning back on something.
Its fun though! :) I noticed some limitation of Kinect are spinning around...and turning sideways
thanks i'll look at those issues
in version 3, the moment the kinect reports a loss of track of the hip node,
it treats the mocap as "seated"
i'll change this in the next version
the microsoft drivers really dont do well with turn-arounds
eventually, i want to re-write the server but based on OpenNI
because it seems to cope with turn-arounds https://www.youtube.com/watch?v=sWADWpHIubI
i did a quick non-seated test and it seems to work
note that i normally check-mark the server's "delayed" check-box,
so when the capture really starts i'm fully in view
thanks! its funny, I was just looking at the OpenNI stuff after I was testing this. I could not get it working with MMD yet, so I uninstalled it all, and am going to try from scratch again tomorrow. I have some MMD VMD's that I ran through LiveAnimation to BVH and then imported, and they came out real good in DS.
an old favorite
https://www.youtube.com/watch?v=dKhLy3kRcq8
it's quite possible that the Microsoft drivers interfere with the OpenNI (primesense?) ones
though i think i saw someone who was using the MS SDK plus the OpenNI one plus the OpenCV all at the same time
my Breckel to bvh to DazStudio test https://www.youtube.com/watch?v=wYEv7gtg68w
i also was able to get the MMD plugin to work but didnt post the results
back then i looked for vmd to bvh converters but didn't get far
would be nice to be able to export dynamic hair animations
that type of dynamic hair/clothes effects, similar to what secondLife uses ... is in my backlog of projects :)
still studying options for face tracking
the 100 2D points tracked by the kinect ( displayed here in DS3 using an mcjBillboards120 figure by the way )
i'm trying to see is i can get more motion measurements than the 6 animation/action units supplied by the SDK
i will probably not support gaze/iris tracking
but i'd like to add blink/eyelids tracking ( instead of faking eyelid animation based on the eyebrows and jaw )
For the VMD to BVH, this worked great! (link) Just selected the character, Import VMD, export BVH...
thanks !
i think there's many MMD animations that can be found in youtube and www.nicovideo.jp/