Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Lols.
Based on Onix's response, in this contex, morphs with "high connectivity".
Wasn't there a script to check load times between morphs, where it looks at the timestamps between each entry and reports the ones taking the longest?
Anyway you can do this manually by looking at the logs rigth after loading a figure.
That's a great idea. Time to make some plots.
Another interesting insight into the Daz3D recursive looping that causes slow load times. Playing around, I created a smoothing on the classic Ronen briefs to deal with poke-thru. I cranked it up so that the time to create the smoothing was about a minute. To my surprise, the smoothing starts but releases the user interface so you can continue to rotate the character while it is working. Well, that's interesting. So, I later tried copying morphs from the main character to the briefs by using the excellent "Fit Control" product, and lo and behold, it took FOREVER to copy the morphs. Ahhh, I see what is happening. Every time you touch a morph, Daz3D fires off the asynchronous process to do a fit. Every single morph. I can see how this could explode in your face if you have a lot of morphs. The only fix I can see would be for Daz to implement a DO_NOT_FIT system variable that could be set to TRUE at the beginning of a long operation such as loading a figure or transferring morphs, then turned off once the operation is complete.
"The easiest way to speed up a computer is to have it do less"
So, can we define a "bad" morph? Yes, I think we can. A "bad" morph is any morph that causes the automatic fit routine process to run. The degree of "badness" will be determined by circumstance depending on how many fitted objects are associated with the morph. This is why slow load times are so random, it "depends".
I would obviously love for DAZ to fix the issues, but in the meantime I'm also interested in user fixes.
I was thinking there were 2 issues:
* A lot of morphs (can be "inactive") associated with a figure; could be the number of morphs as well as how many others they reference. Causes long scene/figure load times. Also seems to delay "zero" operations (egs., setting a T/A pose is a lot faster than zeroing the pose).
* A lot of items "fit to" a figure (hidden or otherwise). This affects loading/previewing poses or an imations, any kind of posing or realtime manipulation.
But maybe what you're saying is that they are both the same problem? "Fit to" basically generates a bunch of morphs that reference other morphs, so it is adding or amplifying the first problem.
In my custom script workflow, I guess I've been partially mitigating this like so: auto-disable "fit to", do what you need to do, enable "fit to". This functions like a DO_NOT_FIT for those special cases of fit morphs. And no, it's not enough to temporarily turn off autofollow because there's still fitting that goes on that never shows up in the hidden morph list.
So what we need is something like this, but for all morphs. Just delay or do updates in parallel while the user works. Though I don't know if that explains why inactive (0 value) morphs on a figure bloats the load time... Maybe it still needs to load and process all potentially connected morphs.
I wrote an external script that calculates the "connectivity" of every morph in my G8F folder (the number of other morphs they connect to, recursively). I want to see if those contribute more to lags.
I'm thinking I want to "flatten" morphs (obj export->morph loader) for those cases so there is only a single merged morph in an otherwise complex figure, and see if that helps.
Some testing & results.
Test 1: All morphs except DAZ defaults are "hidden" in the G8F folder (they're marked as usual Windows "hidden" folders)
Base G8F loads in 3 sec
Test scene w/ base G8F + 20 fitted items (no other morphs active):
Test 2: All morphs I have installed are unhidden in the G8F folder
Base G8F loads in 5 min 45 sec. I don't know what to make of it from the log. This jump here is 2 or so of those minutes:
I can't see anything really wrong with those warnings. They're DAZ store morphs and they're referencing optional products I don't have (and aren't required). It may be that DAZ doesn't really log the other calculations it does, so I don't know if I can map the log timestamps to specific offending morphs.
I also don't know if I can get a clean connection with the number of morphs referenced by each morph (the connectivity). Some of the most "connected" morphs are DAZ original character morphs & expressions, so maybe I'll try hiding only those and see if it brings down the load times.
Test scene w/ base G8F + 20 fitted items (no other morphs active; same test scene as Test 1 & it was saved with only the base G8F morphs in Test 1):
Note that this is all with the exact same scene, geometry & texture resources as Test 1.
My auto unfit/refit script has been letting me do posing & animations almost as though I didn't have all those extra morphs installed, but it doesn't actually help G8F loads, scene loads and the extra stuff DAZ has to do after closing Studio.
Fits totally DO amplify the problem from having a bunch of unused morphs. Without those morphs, autofit items are actually pretty decent. So that's where the real problem is.
Some combo of hiding unused morph folders on a per-project basis, and unfitting/fitting items before any sort of manipulation, may be a practical way to deal with this until DAZ comes out with an update. If they ever! This has been around for a while.
UPDATE: I classified my morphs by the number of morphs they referenced (including themselves), and hid enough morphs to remove 50% of the number of morph references. This seemed to drop all load and manipulation delays by about 40% in Test 2. This is consistent with TBorNot said. I think the reason it didn't improve load times by 50% is because there is some overhead associated with the number of morphs, irrespective of the number of other morphs they reference.
Here you go. It is messy, but it's standalone. Not the fastest script in the world because it enumerates all the things it has to do each time, but that was the fastest way I could make it standalone.
At the very top, you'll find a bunch of options. By default, it will "unfit stuff", but you can set "fit_state = 1" to re-fit them. You can hardcode those options and save different copies of the script & attatch them to custom actions.
Lemme know if you find any bugs. It does know to process groups, statics & rigid nodes in addition to figures and clothing, and it should ignore geo-grafts.
EDIT: File won't attatch, so here it is: https://pastebin.com/eyDWMcA4
any instructions how to do this?, I'm dumb since 1972
Zilvergrafix said:
Turns out I can upload .zip files, so here you go. Unzip those to anywhere in your content folder structure (like any other script): you can launch them from your Content Library, or right click-->create custom action which will make it visible in window->workspace->customize to add to a menu/toolbar and bind a shortcut key.
All the .dsa files are copies of each other with only the first few lines changed. Bad coding, but I'm lazy to make a GUI :)
Files are:
unfit_visible.dsa unfits visible items. By default, unfit items will also be hidden (there is an option in the script to disable this behavior).
refit_visible.dsa repopulates the "fit to" field for visible items (and unhides them if they were hidden in the previous script). You'd use the first 2 scripts before/after any detailed manipulation/morph operation to speed it up.
unfit_invisible.dsa unfits items that aren't visible (so they no longer slow you down).
subd_smooth_off_visible.dsa sets visible items to base resolution and turns of mesh smoothing. It doesn't change the actual SubD level or smoothing amount; just disables it.
subd_smooth_on_visible.dsa reverses what the above script does
subd_smooth_off_invisible.dsa does the same thing as "subd_smooth_off_visible", but only does it for invisible items.
If no items are selected, the scripts will run on the entire scene. If you select a group or figure, then they'll run only on their children.
Hope that helps. Lemme know if there are any special requests, it is easy to modify.
thank you very much! going to follow your steps.
I'm playing around with another way to speed all this up. The issue with constantly fitting/unfitting items is that it is SLOW. Faster than lagging each edit, but not seamless. Even if you bind hotkeys to on/off scripts.
The alternative script (synchronize.dsa) does the following when run on figures:
First time, it will be slow since it adds an invisible figure to the scene. This will bloat scene load times too if you save it that way, but it won't bloat VRAM usage since it is hidden.
But from that moment on, any manual property changes to the original figure will not instanty propagate to the fit items. It will only "update" fit items when the synchronize script is run. I think this may be a way to get to what TBorNot suggested and is faster than completelly unfitting/refitting each time.
Wow! Thanks alot Grim!
Sorry. Real Life called me away, until today.
Tested the first script, without reading further.
Potential. THEN saw your zip!
It all works. Thank you for xmas!
I am suprised/not surprised that not more people say thank you. Shakes head. People.
The unfit vis/unvis * remove smooth works quite well.
With 3 chars on screen, the unfit/del smooth makes it so posing the char is manageable. Not near as good as blender, but at least there is a decent response!
May just go back to texture-shaded with your scripts. Thanks!
The speed isn't bad. If you have a bunch of animating to do, this is way faster than what we had.
Can make another script to unfit/unsmooth all in one script, and another to put them all back as it were.
Look at the first few lines: it is easy to just merge the variables to combine functions. Lemme know if you want me to explicitly mention the setup for some combo you want.
Thanks Grim !
Am busy off-on next few weeks. So will look forward to combining them then.
Essentially want 2 scripts. (1) All off & (2) all on as it was. Need everything off for best animations, at times anyway.
You have already done pretty much everything.
If i can't manage a few intro lines, i should be wacked with the garlic board ;)
If for someone else who appears grateful, and asks for (1) & (2),well I wouldn't say no to saving a bit more time.
Like this? Attached.
subd_smooth_fit_off_visible, subd_smooth_fit_on_visible, subd_smooth_fit_on_invisible, subd_smooth_fit_off_invisible
You are the bestest!!! Thanx alot!
Did rename to gOFF_ & gON_ (vis&unvis) to make it fit my 1440p screen layout. But that was it!
Refit could be tad faster. But so what! Anim diff is manageable now and not a wait for IK bone (like r.hand) to start moving, other than smidge, and then it just goes.
Really is totally appreciated! Thank you!! Hope u win a prize. ;)
Yeah, speeding up refit is what I’m experimenting with. The “synchronize” script from a few posts ago may be a (cludgy) way to do it. But there are other advantages to using a fit clone, like manually controlling the extent of autofit. So I think it’s worth exploring.
Will bookmark this thread in-case you find a "faster" way to refit. Will find it probably within a week or to a day.
From my previous quick trials, the clone route worked. May play yet when have more free time again with "cludgy" vs fit-clone.
TBH, didn't take a comparison look yet, but from the quick tests I did then, totally get why you write about 2 styles. Do remember 1st seemed alot faster despite being more scripts.
Be curious what you settle on. Speed may be worth it too?
DAZStudio was taking between 2 and 3 minutes with my DAZ content to merge the "Base G8 Female" actor into an empty scene (i.e. start DAZStudio then just "merge..." the character). I separated my DAZ content into 38 installation directories based on figure, props vs wardrobe vs pose etc and, by excluding those directories from the "current" CMS settings, managed to establish that pretty much all that time comes from the characters I have bought. Even with the various morph packages included the basic G8F loads in 10-15 seconds; about 12 times faster.
At present I've taken the 38 separate directories and come up with a base set that supports the scenes I have. It turns out that I've bought quite a lot of stuff I don't use... The basic insert is now down to 3-4 seconds. I just need to go through the scenes I have moving characters from my various "Figures" folders to the "Figures in use" folders and catching any of the morph packages I actually use (I know the principle ones and those are already loaded by default.)
It's quite a bit of work; I had to start by de-installing all of my DAZ content and it took me several days to a week to sort through it and install it by what are sort-of DAZ categories... However the benefits of loading a DAZ actor in a few seconds rather than a few minutes seem really worthwhile.
The way DAZ could fix this is by not having those morph dials; maybe someone out there wants 50% Victoria and 50% Edie, but surely it isn't worth slowing every load down by a factor of 10 or more to accomodate mix'n'match of character shapes. Maybe better to put that in a multi-morph loader tool.
Here is the morph loader tool I'm working on:
I made my own "categories" of morphs (as seen in the screenshot). Each category is a .txt file in the same folder as the script with a list of morph names. If a category check box is selected, those morphs are loaded. There is a button to automatically enable any other morphs with non-default values in the scene file, with a deep-search of all referenced morphs, so the selected scene always loads without any missing morph errors.
I normally check some of those boxes, setup props, characters, etc., then re-run the script with just posing & expression morphs enabled. This makes posing/animation work much faster!
Edit: As I posted this, I realized I didn't need those categories & check boxes in the first place. All this tool needs is a scene with whatever figure(s), and any desired morphs assigned some non-default value. The scenes can be saved with different sets of morphs "in use" and themselves act as a way to manually categorize morphs.
PowerLoader reborn :)
It has to be a non-default value anywhere in the timeline but yes, that would fix a whole load of problems. (I'm not sure if morphs can be changed in the timeline, but poses and expressions can).
If I load a scene and some of the pose controls (and most likely morphs too) are not present I get the "product not installed" list and the scene loads without them, as would be expected. If I save this scene it removes the missing poses/expressions and, maybe, morphs even if they have non-zero values. At this point the scene has been changed and given that the change may be a deleted non-zero value somewhere down the timeline it's easy to fail to notice.
It would be useful to have a script that "cleans" the whole timeline by removing morphs/poses/expressions which have zero values throughout the timeline. I believe this would eliminate totally spurious "product not installed" reports; anything in that list would now be something that is genuinely required to correctly load the scene.
I just tested with an animatable morph and whatever method I'm using seems to figure out if it is used anywhere in the timeline. The assumptions I'm making are:
jbowler said:
Right now I don't modify the scene file, but it's not hard for me to remove references to morphs that are not detected as "used". So far, I've been erring on the side of marking morphs as "used" when in doubt, to avoid any missing morph error messages.
Both the last two strike me as bugs in the morph. There was a recent report somewhere here by a user who had apparently ended up with a rogue morph which morphed every character; so even an old scene would be changed. I don't know if anyone ever worked out if it was a morph loading with a non-default value or a morph with a default value that changed things but I'm pretty sure that either must be wrong.
Most things I've tried have no real noticeable effect below 1% but I have seen DAZStudio report "changed" values for X/Y/Z transforms at the hip level simply as a result of scaling of the character, so there is certainly FP error in there and what you are doing sounds to me like the right approach.
That's the behavior I would expect.
Yes, cleaning up scene files, and pose timelines, is a separate, distinct, operation.
Dunno is this info can lead to some improvement but:
I did a video of removing morphs to accelerate G8 load time, but don't worry I'm not publishig that video here for now.
my question for developers is this...
I remove my entire morphs (as an experiment), now I load some of my custom characters an Daz asks for missing morphs
is that a problem?, NO, nope, I removed manually, and they are safe on another Hard Drive, this is an experiment, I remark that.
my question is, IF Daz Studio knows what morphs do I need to load correctly my figure, listing them with no delay in nothing,
why not using that info in create a FINAL FILE (example, when your figure does not need more morphs available for modifications, my figure is perfect already) when we create Subsets or Character Presents or Shaping Presents with no need to load my 23Gb of Genesis8 morphs and only load that list of morphs needed to load my customized figure?, a final file is not an invent of mine and I know some softwares uses this kind of settings, dumb of me I can't remember which software I saw that feature.
Yes it sounds confuse and my rotten English can be more confused and very bad explaining situations but let me show an example, Imagine you do/create/replicate RYU from Street Fighter using only Genesis 8.1 male morphs and your result is this:
Obviously I don't need to make other modifications, I've made a perfect RYU replica with G8.1M not using original videogame assets.
and such file does not need my ton of other morphs because I don't need another morph for Ryu but the neccessary ones, that is a final file character, if such file format or feature would exists in DS no more loading your entire morph library, you only would load your subset or character Ryu in any scene and voila. a final file format only will recall your list of morphs neccessary to create my RYU. gotcha? capisce?, understood?, quedo bien claro lo que intento explicar?
One solution could be a variation of the old morph injection technology, where the morph automatically gets injected into the character as soon as you set the dial to anything but zero, and gets removed again as soon as it's zeroed. Or maybe a checkbox on each morph dial for injecting/removing the morph.
I have done some previous investigation into figure load time at here.
My main findings are:
That is what is happening now. It's setting up the channels and links that make it work that takes the time.
Did some more poking.
The cache file is basically most of the files in "data\DAZ 3D\Genesis 8\Female\Morphs\...." mashed together with one important exception. The bulk of the data in those files are of two types: morph vertex delta and formula. The important exception is that morph vertex deltas are not included in the cache file and is loaded on demand when you change the slider. Formula data still make into the cache and can slow things down significantly.
This lead to one important observation: the load time is proportional to the number of files in the morph folder but not necessarily file size. If the file contain large amount of delta that is not included in the cache it will be fast to load. On the other hand if the file contain lots of formulas it will be very slow to load.
Edit:Also the morph file could be compressed which would make it smaller than it actually are.