Investigating Character Load Time

volpler11volpler11 Posts: 29
edited October 2020 in Daz Studio Discussion

I have been digging into daz character load process to try to make loading character faster. It is widely know that the more morphs you install the slower daz will load a character.

Character load process:

Using process monitor from Microsoft, it is possible to get a log of files daz accessed during the character load process with timestamps.

Here is a rough break down of the character loading process. I have tried to start this load on the hour to make the timestamp calculation easier. My computer is a bit slow but the sequence of events are good illustration of the process

0-46s unknown need to investigate further.
46.35s-52.19: start loading base character file, textures, uv ect
50.42s: access windows registry and a bunch of tcp call (accessing daz database?).
52.19s-141.89s: processing G8F cache file located at "AppData\Roaming\DAZ 3D\dson\cache\DAZ 3D\Genesis 8\"
141.89-148.90s: querying and access morphs located in"\data\DAZ 3D\Genesis 8\Female\Morphs\" in your content library
148.90s-151.07s loading 3rd party UV files.
151.07s-236s maining loading eyelashes.Daz might also be doing other things here since loading G8F dev load doesnt really reduce the amount of time needed.

Daz log report around 202s loading time, so its about right. 

Cache file:

As you can see bulk of the time is spent processing G8F cache. That file is a json text file containing prepossessed morphs value, see also DAZ dson file spec for asset_info, modifier, and scene object. Because the file is in text format, the text parsing process is a lot slower that the file read speed. It should be CPU limited rather than SSD speed limited. 

When adding new product or first installing daz, that file need to be generated and the character load process will be a lot slower as daz need to do a complete write.

By using some scripting, its possible to parse that file and count how many morphs a particular product have and uninstall the worst offenders. Note size of the json for each morph is variable. Factor include the number of formula it have. Just counting the number of asset in the json file is not sufficient. I used the number of characters for each product as a rough guide on the load time cost.

Improving loading time:

Uninstalling 4 of the most slowest to load product cut the load process by almost 1min from 202s to 148s. The cache file is reduced from 165MB to 129MB. Unfortunately some of the most expansive package to load left for me are zev0's morphs that I actually use. Note currently I dont have many characters installed.

Uninstalling package with HD morphs will might speed up with the morph loading phase as they end to be very large.

Might eventually write a python script to help find packages that are slow to load, or maybe someone else could share one. Currently my script is written in matlab and not founding the characters very accurately.

Daz dev could probably improve the character load time by improving the json parsing time. Maybe switching the cache to binary rather than text format.

process monitor: https://docs.microsoft.com/en-us/sysinternals/downloads/procmon
dson objects:http://docs.daz3d.com/doku.php/public/dson_spec/object_definitions/start

Edit: Dev load.duf is slightly faster than normal Genesis 8 basic female.duf. By installing a few more packages, managed to reduce the load time to about 1min.

Post edited by volpler11 on

Comments

  • ZilvergrafixZilvergrafix Posts: 1,385

    Interesting! i did a zero start with a single character and it takes minutes of message warnings and less than 30 seconds to load, after the almost 3 minutes of warnings of course, I dunno what causes this, the messages states something that I don't know how to fix, I have my runtime AND DazStudio in drive D not C.

    Thats the messages appearing and consuming minutes...

    My character is not heavy morphed, all that you see is all that she conforms, below after "static surface" is only ichigo hair posing and styling parameters

     

     

     

  • nicsttnicstt Posts: 11,715

    Reducing teh number of characters and morphs will reduce load times as will (I think) reducing or eliminating the number of conflicts.

  • ZilvergrafixZilvergrafix Posts: 1,385

    I'm going to do a clean install in C drive with no references to my D Runtime.

    (now in progress)

  • Why the 'solution' to something in Daz need be ... 'uninstall' Daz...? 

    :/

  • nicsttnicstt Posts: 11,715
    edited October 2020

    Because the more morphs and characters you have the longer it takes to process. Personally I'd eliminate conflicts first as I'm convinced they take up time, but have no evidence to support it.

    In effect, you can read a page faster than you can read a book. Computers are the same, even with parallelism, the more there is the longer it takes.

    Post edited by nicstt on
  • brvsnbrvsn Posts: 213
    edited October 2020

    well actually there is a way to check how much time conflicts waste. not 100% accurate because it's based only on daz studio log and I'm not sure everything is logged..

    anyway I use a program to read and analyze logs: there are many different ones; the important thing is that it needs to have the option to calculate time between log lines; I use a small and open source program called Tailviewer. open the daz studio log with it and check how much time passes between logs.. you'll find something like this

    everything above 4 seconds is something to double check. If I can't solve the issue (I'm not really an expert) I install the product into a different folder that I load into daz with CMS Sets only when needed

    Post edited by brvsn on
  • PerttiAPerttiA Posts: 10,024

    Why the 'solution' to something in Daz need be ... 'uninstall' Daz...? 

    :/

    It doesn't, and especially with long loading times it has no effect, since the problem is not DS the program but the installed content.

  • PerttiAPerttiA Posts: 10,024
    brvsn said:

    well actually there is a way to check how much time conflicts waste. not 100% accurate because it's based only on daz studio log and I'm not sure everything is logged..

    I am sure everything is not logged.

    When I had problems with loading times, I first eliminated the warnings and then started removing characters and morphs that I wasn't using.

    Every now and then I got a new warning that wasn't related to anything that was removed, for example typing errors inside the remaining morph files.

  • brvsnbrvsn Posts: 213
    PerttiA said:
    brvsn said:

    well actually there is a way to check how much time conflicts waste. not 100% accurate because it's based only on daz studio log and I'm not sure everything is logged..

    I am sure everything is not logged.

    When I had problems with loading times, I first eliminated the warnings and then started removing characters and morphs that I wasn't using.

    Every now and then I got a new warning that wasn't related to anything that was removed, for example typing errors inside the remaining morph files.

    yea but that may also be related to the fact you loaded a different character with different requisites (and so different derivated morphs allowed) like G3 or G8 or V7, Darius 8 and so on..

  • PerttiAPerttiA Posts: 10,024
    brvsn said:
    PerttiA said:
    brvsn said:

    well actually there is a way to check how much time conflicts waste. not 100% accurate because it's based only on daz studio log and I'm not sure everything is logged..

    I am sure everything is not logged.

    When I had problems with loading times, I first eliminated the warnings and then started removing characters and morphs that I wasn't using.

    Every now and then I got a new warning that wasn't related to anything that was removed, for example typing errors inside the remaining morph files.

    yea but that may also be related to the fact you loaded a different character with different requisites (and so different derivated morphs allowed) like G3 or G8 or V7, Darius 8 and so on..

    No, that's not the case. I am talking about loading the development version that has no character and/or morphs activated, but as DS still reads each and every character/morph installed for that base figure, it should list ALL warnings every time but apparently doesn't.

  • RobinsonRobinson Posts: 751

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

  • ZilvergrafixZilvergrafix Posts: 1,385

    Maybe we need a way to save an asset like "For faster loading, do you want to save this file like a FINAL version?" and avoiding load all the available morphs not needed for because any final save asset is not being modified.

  • PerttiAPerttiA Posts: 10,024
    Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    It isn't doing any caching, if it was, the cached data could be utilized when loading the next character with the same base figure.

    When loading a character, DS reads all the characters/morphs for creating the dials and the depencies between the dials and the morphs, when a dial is actually used, DS goes and reads the file again for deltas

  • PerttiAPerttiA Posts: 10,024

    Maybe we need a way to save an asset like "For faster loading, do you want to save this file like a FINAL version?" and avoiding load all the available morphs not needed for because any final save asset is not being modified.

    That would need Daz to re-think and re-write the whole concept of how the morphs are added to a figure, it used to be different with the older generations (V4 and older) but having to inject all the morphs separately every time wasn't fast either.

  • TBorNotTBorNot Posts: 370

    There is also the possibility that writing to the log file is actually causing the problem.  Do not underestimate the time for a file open, write, and close for each line.  I linked my log file to /dev/null (Mac) and it's more happy now.

  • Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    This is correct.

    PerttiA said:
    Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    It isn't doing any caching, if it was, the cached data could be utilized when loading the next character with the same base figure.

    When loading a character, DS reads all the characters/morphs for creating the dials and the depencies between the dials and the morphs, when a dial is actually used, DS goes and reads the file again for deltas

    So this isn't correct.

    Maybe we need a way to save an asset like "For faster loading, do you want to save this file like a FINAL version?" and avoiding load all the available morphs not needed for because any final save asset is not being modified.


    Rob says Watch the ski..change log, once the next full release is out.

    TBorNot said:

    There is also the possibility that writing to the log file is actually causing the problem.  Do not underestimate the time for a file open, write, and close for each line.  I linked my log file to /dev/null (Mac) and it's more happy now.


    The log is held in memory until the system, is free to write it - that's why it isn't always up to the problem step on a crash or the like.

  • PerttiAPerttiA Posts: 10,024
    Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    This is correct.

    PerttiA said:
    Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    It isn't doing any caching, if it was, the cached data could be utilized when loading the next character with the same base figure.

    When loading a character, DS reads all the characters/morphs for creating the dials and the depencies between the dials and the morphs, when a dial is actually used, DS goes and reads the file again for deltas

    So this isn't correct.

    Maybe we need a way to save an asset like "For faster loading, do you want to save this file like a FINAL version?" and avoiding load all the available morphs not needed for because any final save asset is not being modified.


    Rob says Watch the ski..change log, once the next full release is out.

    TBorNot said:

    There is also the possibility that writing to the log file is actually causing the problem.  Do not underestimate the time for a file open, write, and close for each line.  I linked my log file to /dev/null (Mac) and it's more happy now.


    The log is held in memory until the system, is free to write it - that's why it isn't always up to the problem step on a crash or the like.

    Richard, care to explain what kind of caching is done when loading a figure.

  • If I understand the problem, the more assets you have installed, the longer it takes to load a G8 figure. If the solution is to place a lot of your morphs in a non-loading folder, or to uninstall them, there are two big problems:

    1. Uninstalling is fine for items you purchased from Daz (although installing is often ridiculously slow), but if they are from another source--and there are many sources out there--you can't simply reinstall them when needed. If you keep them in a separate folder, then you would have to load and unload the item to your library, after finding all the components (i.e. morphs, textures, geometry, etc.) which is very time-consuming. The other option is to place them in a Daz library and load the library--this has the unfortunate downside of making it difficult to reopen the file after it has been saved and closed. You have to retain the folder on opening Daz or reload it before you open the saved file or your scenes will come in with numerous components missing. This just slows things to a crawl.

    2. The solution to these problems usually involves removing, uninstalling, or offloading troublesome morphs. So far, I have found at least 4 morphs mentioned in the numerous posts about loading times, Animate2 settings that should be changed, hidden files that should be removed, and all sorts of suggestions that are either unworkable or incredibly time-consuming. Offload the morphs and there are still characters in the people menu when they're removed. It's like trying find a number of needles in a haystack and then putting them back exactly where you found them. If Daz is in the business of selling assets and making commissions on asset sales, it is immensely counterproductive to have software that gets slower the more items you buy, and that requires exhaustive workarounds to minimize this slowing.

    I would like to see Daz Studio prosper--we all benefit from their success. Its business model seems to be a good one that is constantly being undermined by the software's deficiencies. If the purpose is to sell more assets, but those assets slow the software to a crawl, there's a huge fly in the ointment. If nothing else, I think fixing this Catch-22 should be Job 1 for the engineers (Job 2 being autosave). I have to go now...I think the DS file I've been loading for the last hour may finally have opened.

     

  • brvsnbrvsn Posts: 213
    edited October 2020

    1. Uninstalling is fine for items you purchased from Daz (although installing is often ridiculously slow), but if they are from another source--and there are many sources out there--you can't simply reinstall them when needed. If you keep them in a separate folder, then you would have to load and unload the item to your library, after finding all the components (i.e. morphs, textures, geometry, etc.) which is very time-consuming. The other option is to place them in a Daz library and load the library--this has the unfortunate downside of making it difficult to reopen the file after it has been saved and closed. You have to retain the folder on opening Daz or reload it before you open the saved file or your scenes will come in with numerous components missing. This just slows things to a crawl.

    yea third parties contents are annoying from that perspective.. but there are workarounds, like manually making DIM packages for them as well or using a tool I recently discovered reading the forum: Daz3D Manager Tool - that basically lets you load and unload morphs by searching them even by names. very handy.. it moves selected morphs to a custom folder (you can restore anytime) but it does not move the materials, so you can still use them..

    only limitation is that you can only select one content folder at time, so if you for example install daz content into a folder and R*Y ones into a different folder you can only pick one from the two

    Post edited by brvsn on
  • volpler11volpler11 Posts: 29
    edited October 2020
    Robinson said:

    The point of the cache is to speed it up.  Imagine how slow it would be without it.  

    Correct first time loading a character after installing all the morphs is at least 3x slower because daz have to build the cache file.

    Robinson said:

     It's probably not the JSON parsing that's slow, it's all the things that happen during it, so I doubt making a binary blob would speed it up that much.  You can load a couple of hundred megs into memory extremely quickly these days.  What it's actually doing will remain a mystery without the source code.

    It is actually the JSON parsing that is slow. Loading the file is very fast but converting all the text stored in the file to data is very slow and is probably single threaded.

     

    Post edited by volpler11 on
  • volpler11volpler11 Posts: 29
    edited September 2021

    -

    Post edited by volpler11 on
  • SevrinSevrin Posts: 6,309
    brvsn said:

    1. Uninstalling is fine for items you purchased from Daz (although installing is often ridiculously slow), but if they are from another source--and there are many sources out there--you can't simply reinstall them when needed. If you keep them in a separate folder, then you would have to load and unload the item to your library, after finding all the components (i.e. morphs, textures, geometry, etc.) which is very time-consuming. The other option is to place them in a Daz library and load the library--this has the unfortunate downside of making it difficult to reopen the file after it has been saved and closed. You have to retain the folder on opening Daz or reload it before you open the saved file or your scenes will come in with numerous components missing. This just slows things to a crawl.

    yea third parties contents are annoying from that perspective.. but there are workarounds, like manually making DIM packages for them as well or using a tool I recently discovered reading the forum: Daz3D Manager Tool - that basically lets you load and unload morphs by searching them even by names. very handy.. it moves selected morphs to a custom folder (you can restore anytime) but it does not move the materials, so you can still use them..

    only limitation is that you can only select one content folder at time, so if you for example install daz content into a folder and R*Y ones into a different folder you can only pick one from the two

    If you use Content Package Assistant, you can install products from other stores with DIM, and can uninstall them just like products purchased at Daz, and reinstall them using the same zip file.  You won't get Smart Content but the installing/reinstalling part is pretty painless.

  • Tomorrow I'm going to move my morph directories over to a slice of my RAM disk, will let you know if that improves my load times. After I do an A/B comparison, will use the python script that volpler11 mentioned above.

    I'm honestly wondering if my case is user error as it appears that my cache is setup under 'C:/Users/<user>/AppData/Roaming/DAZ 3D/dson/cache' but my disk IOPS are crushed when I'm loading models... does the cache location need to be included in the Content Directory Manager? I need to do some further investigating to see if those files are even being read during Figure load as it seems as though the cache is not being utilized.

Sign In or Register to comment.