First mcjMidiLoad Beta Available - Midi File loader script for Daz Studio

Imago

Where can I get the kit, even incomplete?
I want to try it to verify a thing that buzzes in my head since I heard about this script...

mCasual

a little 90fps + motion blur + 30fps output test video

http://mcasual.x10.mx/rarara2.webm
or
http://mcasual.x10.mx/rarara2.avi

Keiron

Hi Casual

Great close up, movement is very good

Excellent well done

mCasual

new test/demo video :)

https://www.youtube.com/watch?v=KpZe6WmncHs

MilosGulan

Wow :) magic ;)

Keiron

Hi Casual

Total respect for your talent

Great Demo and that's one super cool Piano

Keep up the great work

Kindest wishes

Keiron

mCasual

currently playing with DS4.7 and i had to tend to teleblender3 but this weekend i do intent to give another go at the midi project !

notably the 2 scripts which let you load the midi file and animate instruments didnt change much last week meaning they're probably ready for a release, the piano playing script ( just a helper script it wont do all the work ! ) will probably take longer to finalize

first i need to adapt Aiko shoes to Victoria 6

mCasual

nearest objective is

- make sure the midi loader works well
- have a visual animated feedback in the scene for all the midi instruments and drums used

later

- have the script that turns the loaded midi data into an animation of Anything in your scene!

and finally

- post the pianist animation helper script ( which will
not be a turn-key solution,
just some time saver script, so it takes you 40 hours
instead of 160 hours to animate that keyboardist in your music video

Figure 1 - this will probably become an animated prop

Figure 2 - I was thinking of having a prop like that for each midi channel,
( there's VU Meter sticks hiding behind each drum name )
but in the end it's not a big advantage over the Parameters Tab

mCasual

i think, each non-percussive instrument will have 1 node, and 1 property per note , only the notes used in the music piece, not the 128

and each percussive instrument ( channel 10 ) will have 1 node per drum, only drums used in the music piece

so it will be easy to keep track of each instrument by selecting the node and watching the keyframe markers on the Daz Studio Timeline

Keiron

Hi Casual

Looking Great

I've made a basic Stage using Planes

I'm making a dancing crowd at the moment LOL

Great job

mCasual

that's the upcoming mcjMidiVU prop ( daz/poser/carrara )
there's 128 morph channels, and each one controls the position of the red VU bar for that note/drum
it will serve as an in-scene visual guide during the production of music videos

for non-percussive instruments, the prop simply has to be relabeled

( and a first occasion to use one of the 260 new mcjDoorFrames ! )

will be released here -----> http://www.daz3d.com/forums/discussion/55316/

Keiron

Hi Casual

Many thanks for the mcjMidiVU prop it's very very cool

Not sure how to make it do anything yet, LOL

I saw your script, could I ask how to use please, just for testing

Very well done, Excellent Job

I loaded both into the scene came in as mcjMidiVU and mcjMidiVU (2)
They started off all Black, but a quick play with the Surface and Parameters soon sorted them out

I've downloaded all your great frames too

mCasual

keiron said:

Hi Casual

Many thanks for the mcjMidiVU prop it's very very cool

Not sure how to make it do anything yet, LOL

I saw your script, could I ask how to use please, just for testing

Very well done, Excellent Job

I loaded both into the scene came in as mcjMidiVU and mcjMidiVU (2)
They started off all Black, but a quick play with the Surface and Parameters soon sorted them out

I've downloaded all your great frames too

guess i should've tested the prop in DS4, i did all my tests in DS3 ...

your DS4 is probably in per-pixel-shading preview mode
the per-pixel-shading mode has some advantages ( chromes for example )
but doesnt seem to handle other surface settings correctly
you can turn it on/off in DS's Edit/Preferences/Interface menu

i'll post a new version of the prop that looks better when loaded in pixel-shading mode

the little demo script... doesn't work anymore because the prop changed since then

i posted an updated downloadable version of it on the mcjMidiVU thread

mCasual

the basic building block of the midi animation kit

the midi-loader script

will also "know" how to animated the mcjMidiVU props

mCasual

yay

first success at loading a midi file using the upcoming mcjMidiLoader script

and immediately applying it to an mcjMidiVU prop

which is intended as visual cues for someone building a music videos

one could attach it to a view camera
wander around the scene
scroll through the timeline
add visual events that will happen when a note is played

fig 1 the 100 first frames of Pink Floyd / Sid Barrett 'Bike'

( truth is an important motivation was that i need a quick way to verify that
the midi loader produces tightly sync'ed animations and test many music pieces )

mCasual

Big video demonstration !
https://www.youtube.com/watch?v=884VgF7c1AY

so, now that we have the mcjMidiVU props, i can quikly test
mcjMidiloader and make sure it doesnt misreads the midi temps and BPMs

it also let me notice that the drum beats are so short that
i must add special code to make sure that quantization doesnt destroys some of them

Keiron

Hi Casual

Very well done

Keep up the great work

I see what you mean about the drum note on and off it's very fast. "The brain and limbs are wonderfull things". Hopefully it's not missing MIDI notes as some KYB visuals aren't lighting up. I checked my Keyboard and it has a percussion from Octave 1 through to Ocatve 8, is it seeing all the percussion from 0 to 127? also I noticed my KYB has Drums on Channel 10 and 9. I suppose you could have two drum sets playing LOL

Just had a look at a Drum Riff in Magix Samplitude it looks like the MIDI DRUM note on and off times are all the same but have different velocities. I assume it's relying on the Percussion sample times to do the job in audio terms. Perhaps you could add a visual delay for each Percussion Instrument with the default set so you can see most of them

Looking through the MIDi sequence the Note on and Off seem to complete between 2- 6 Note on and Note off's

In this sample the keyboard clock is not synced to the PC clock so the beats do not hit the exact time line

Thank you for the updated mcjMidiVU prop looks fine now

atticanne

Fantastic so far. I'm really excited about this project.

mCasual

AtticAnne said:

Fantastic so far. I'm really excited about this project.

advancing that project this evening with the foolish hope of posting the midiLoader :)

mCasual

keiron said:

.....the exact time line

when quantize is On and i receive a note-Off too early, i'll place it on the next frame or between 2 frames

if i'm not mistaken it's not that the drumming is faster than 30 beats per second, it's just that the drum doesn't need to be sustained, electronically speaking so they send the NoteOff immediately after sending the NoteOn event

in Figure 2 we can see we can safely detect when a Note-Off would over-write a note-On event and shift it to 1 frame later
and anyway, what's important it that the note-On event arrives on-time

Figure3 - the solution i'll retain for now,
when quantization is on
the NoteOff events are all delayed by 1 frame ( 1/30th of a second usually )
so a note-On will never be overwritten with a note-off,
but it the drumming is super fast, the note-off events may get eaten by a note-ion event
so it will become a series of Note-On events

mCasual

New Demo which doesnt miss a beat !

and i transferred some of the midi data
to a zebra head bend animation
and to Amy's "mouthSpeak" morph

https://www.youtube.com/watch?v=UqaLgpuOfNc&feature=share

mCasual

now that i know it works with this midi file
tomorrow, i almost can promise i'll post the first version of midiLoader
which lets you

1 - ( with the help of mf2t.exe) convert a midi music file into a text file
2 - load this file in your daz studio scene as a midi node, with up to 16 midi channel nodes
3 - transfer midi channel animation to the very specialized free mcjMidiVUpiano or mcjMidiVUdrums props

so that gives you the basic visual cues needed to build an animation

and tomorrow also i will give you at least 1 script
which lets you transfer some of the animation to a Figure joint or morph

in the demonstration video,
https://www.youtube.com/watch?v=UqaLgpuOfNc
the zebra's bend node is activated each time
a note higher than midi-note 62 is played on the Lead_2_Sawtooth channel

Amy's MouthSpeak morph is activated anytime the saxophone plays a note
and deactivated each time the note is released
which for an unknown reason makes her miss some notes

Imago

Casual said:

New Demo which doesnt miss a beat !

and i transferred some of the midi data
to a zebra head bend animation
and to Amy's "mouthSpeak" morph

https://www.youtube.com/watch?v=UqaLgpuOfNc&feature=share

This is exactly what I was hoping for this script!
Thanks! ^.^

Keiron

Hi Casual

Rock and Roll, very well done

"Amy’s MouthSpeak morph is activated anytime the saxophone plays a note
and deactivated each time the note is released
which for an unknown reason makes her miss some notes"

It might be because note on and note off are out of sequence

note 1 on mouth open
note 2 on mouth open
note 3 on mouth open
note 1 off mouth closed
note 2 off mouth closed
note 3 off mouth closed

note 1 on mouth open
note 1 off mouth closed
note 2 on mouth open
note 2 off mouth closed

note 1 on mouth open
note 2 on mouth open
note 2 off mouth closed
note 3 on mouth open
note 1 off mouth closed
note 3 off mouth closed

There is usually some controls like a Sustain Pedal, Breath Controller or Pitch Bend etc, that can hold a group of notes on for longer, detect changes in air pressure , or change / bend the Pitch of a note as in a guitar, you may be able to use that + note info to control the mouth , and possible use the note changes to change some face features

You can see some of these control values here
Table 3
http://www.midi.org/techspecs/midimessages.php

Which instrument is selected
http://www.blitter.com/~russtopia/MIDI/~jglatt/tech/midispec/bank.htm

Perhaps you could use the 1st note on to set Mouth to blow into the SAX, "followed by a hold your breath time or when model falls over running out of breath, then a quick open and close to take a breath until all notes off and a pause or breath controller detected. using the notes on and off to press the keys of the SAX

mCasual

i did this very quickly at the last minute
the idea was to show that the drum beats were now all visible

i had Amy and the zebra just standing there in the scene
so i used the fastest way to just make them move a little

yes it's like you said the notes are straddling

with the script i used , note0 ..... imposes its on-off events on the head/mouth
then note1 ..... up to note 127, without regard to what the previous note did

i must add the note influences using an 'or' operation instead

i dont think i'll cover the controllers. pedals, pitch bends, after-touch and velocities

mCasual

since it was a bit faster i'll combine the basic transfer mechanism in the mcjMidiLoader script

so the steps to animate a woofer which happens to have a push-pull morph would be

1 - by your own means convert your midi file into a wav file.
i use PolySoft's Free Midi To MP3 converter and Audacity )
The converter is hosted at cnet be careful and avoid installing the optional stuff that's usually bundled with it

2 - in Daz Studio's Edit menu, set this wav file as the scene's audio track

3 - select the woofer

4 - launch the script

5 - convert the midi file into a text file

6 - load that text file

7 - select channel 10. the percussion channel

8 - select channel 10's Bass Drum

9 - select the woofer as the target node

10 - select the woofer's push-pull morph

11 - click the assign button

But it's not ready yet, i have to put in place the transfer function,
for example, you could want the Mouth-speak morph to be 0%
when there's no note and 75% when the note is at full velocity
or you could want the Y-Scale of an elastic trumpet to be 100% when
the note is off and 300% when it's on at full velocity
or you could want any velocity over 0% to be treated as 100%

--------------
fig 2 - there's the transfer function ( eventually, scripting code will be allowed )

mCasual

dress animated entirely with mcjMidiLoader

i just have to make the note-range thing effective

the Val-Min / Val-Max thing works !

i have to make the "expanded' trick and the '%' trick work

and that's a wrap .... before midnight !? maybe !

mCasual

First Beta Release of mcjMidiLoad !!!

https://sites.google.com/site/mcasualsdazscripts5/mcjmidikit

Note just now i tried animating Aiko's arm and failed
which means that this beta version 001 possibly works only on props
ok i found out why and will post version 002 incessantly
( i was searching non recursively for bones in the figure's child nodes instead of the bones )

i will start writing the manual right now, so come back regularly to that
web page to see the manual progress and to get the updates !

the crash-course:
get mf2t.exe
select something you want to animate
start the script
tell mcjMidiLoad where mf2t.exe is installed
use mcjMidiLoad (and mf2t) to convert a midi file into a text file
load that text file
select the midi channel that will drive the animation
select the notes or select a range of notes that will drive the animation
select the property of morph of your target object that will be animated
specify what value you want that property to have when the note is off and
specify what value you want that property to have when the note is on
click the assign button

mCasual

- v 002 - v 002 - - v 002 - v 002 - Update - mcjMidiLoad - v 002 - v 002 - - v 002 - v 002 -

version 001 was unable to apply animations to bones other than the hip !

version 002 succesfully created the foreArm animation, driven by the Bass Drum of a midi file
which you see below in the animated gif

NOTE: For Now Morph channels animation on figures works for Daz Studio 3 but wont work for DS4 !

( i'm almost always in DS3 ... sorry ... version 003 will work on figures for DS4 ... very very soon )

WARNING NOT Working At All with DS4 - version 003 will, later in the night ! ! !

mCasual

WARNING NOT Working At All with DS4 - version 003 will, later in the night ! ! !

the reason being possibly that DS4.7 apparently doesnt want to treat times as numbers unless we tell it to

how ridiculous is that ! :

var t = prop.getKeyTime( j ) + 0;

also had to do

var tick = Scene.getTimeStep() + 0;

there you go, animated foreArm bend and animated head morphs using mcjMidiLoad running in DS4.7Pro

this version will be posted in the next 10 minutes !

( just got to remove some debugging traces )

oops actually got to fix the case wherein there is no pre-existing midi node in the scene !