Stop genesis from loading morphs

24

Comments

  • s_j_gregorys_j_gregory Posts: 104

    Please excuse the typos and misspellings.  I would note a couple of observations.  Both installations are sticky - they retain the 'foot print' in RAM even after the content is deleted.  I think the deifference in these results is not trivial.  If for instance one was to look at the last iteration, the old system shows an increase of about 500 MB when Dayton is added and an additional 300 MB when Cherry is added.  The new system shows an increase of 2.3 GB of RAM when Dayton is added and and additional 1.3 GB when Cherry is added to the same scene. 

    A couple of asides:

    1.  There is a marked difference even with Basic DAZ default characters: G3F and G8F

    2.  For a long time I was seeing V8 slider and Elizabeth slider activated whenever I installed any G8F.  I think I have solved this partly - but perhaps not completely given these results - but based on observation there are noticable differences in appearance between DT Cherry on the old system and DT Cherry on the new system.  There are what would appear to be morphs appled that do not show as currently used.  I reinstalled Elizabeth and V8 and no longer have their sliders activated in all G8F loads but there is a difference in appearance. 

    3.  But then, none of this explains what is going on with the G3F cases.  While not as large a difference it is not trivial either.

    I do have a lot of morph packages.  And they are not all installed on the old system.  Are they enough to explain this.  Even with 32 GB of RAM, a single G8F character that needs some posing and adjusting, as an example, to use a timeline to fit a garment or hair at these foot prints is not fun.  And my results would also suggest that using a basic character for setup is not going to solve the issue.

     

    Thanks again for the response.   

     

  • TheKDTheKD Posts: 2,696
    edited May 2020

    When you say old system and new system, you are meaning using two different computers? With the same exact version of DS?

    DS, freshly opened for me uses 401MB RAM according to task manager. Loaded G8F Base, took 4 minutes to load, task manager shows 5302MB load. After delete it shows 5303MB load lol.

    Meanwhile, blender over there chilling with 976MB load, with a scene I am building that has almost 400k polygons in it so far lol.

    Post edited by TheKD on
  • s_j_gregorys_j_gregory Posts: 104

    It is.  Perhaps had I waited longer the results would have been different. But interesting it occurs with both systems as well.  This though is far easier to manage than the RAM issue because that is only a metric indicating an entirely sluggish performance for a large number of processes - say save keyframe for example - due to the size of the file.   (my 4 GB RAM older system outperforms my 32 GB newer system)

  • s_j_gregorys_j_gregory Posts: 104

    TheKD,

    Yes, two systems. 

    The old system is old.  It has 4 GB of RAM and an Intel Core 2 Quad Q6600.  No GPU to speak of. 

    The new system is really just newer.  It too is old.  It has 32 GB of RAM and an Intel i7 4790k.  It has a GTX 660 TI and an RTX 2070 for GPUs.  Above where I mention G3F or G8F, like you, they are 'Base'.  I am really surprised, and disappointed, you are experiecing the same issue as I and even worse.  I was hoping I had something that was able to be fixed.  I still find the massive difference between old and new hard to explain with only morph dials. 

    Oh, and yes they are both using 4.12.  Actually the newer one is using 4.12.1 as I did a fresh install as part of my trouble shooting.

    Do you have a lot of morph packages?  Is the SubD other than 1?  Is it loading with the default MAT?  Sorry for the basics but these are all threads I looked at for an explanation and I keep coming back to morphs. 

  • Richard HaseltineRichard Haseltine Posts: 102,291
    TheKD said:

    That's odd, you would think after deleting the figures, it would go back to around the baseline idle, but doesn't

    The deletion is on the Undo stack, so still taking up memory.

  • Richard HaseltineRichard Haseltine Posts: 102,291

    TheKD,

    Yes, two systems. 

    The old system is old.  It has 4 GB of RAM and an Intel Core 2 Quad Q6600.  No GPU to speak of. 

    The new system is really just newer.  It too is old.  It has 32 GB of RAM and an Intel i7 4790k.  It has a GTX 660 TI and an RTX 2070 for GPUs.  Above where I mention G3F or G8F, like you, they are 'Base'.  I am really surprised, and disappointed, you are experiecing the same issue as I and even worse.  I was hoping I had something that was able to be fixed.  I still find the massive difference between old and new hard to explain with only morph dials. 

    Oh, and yes they are both using 4.12.  Actually the newer one is using 4.12.1 as I did a fresh install as part of my trouble shooting.

    Do you have a lot of morph packages?  Is the SubD other than 1?  Is it loading with the default MAT?  Sorry for the basics but these are all threads I looked at for an explanation and I keep coming back to morphs. 

    But the two have different content installed?

  • TheKDTheKD Posts: 2,696
    edited May 2020

    Tons of morphs yeah. She loads in with subd 1 and render subd 2, default texture set as far as I can tell, they all have BASE in the texture map name.

    Post edited by TheKD on
  • s_j_gregorys_j_gregory Posts: 104

    Richard,

    Absolutely.  By 'different content' though i mean the new system has much more than the old.  The old one has only one runtime on one drive.  The new one has the app on the system drive along with DIM; and the DAZ content is in two runtimes on two physically different drives.  I could get you some metrics but there is a huge difference in content.  On the old system, as an example, the Genesis 8 folder (in the data/ DAZ 3D/ Genesis 8) is 1.08 GB.  The Genesis 8 folder on the newer system (in the data/ DAZ 3D/ Genesis 8) is 6.2 GB.  I picked this example as it might be telling.  But I would have thought this might suggest loading time rather than RAM consumption.  

    And I acknowledge your point about Undo.  I did a little experiment with G8F on the new system with a skript for clearing the undo stack (I think from sharecg) and it reduced the amount of consumed RAM from 2.6 GB to 0.9 GB after deleting.  Still a bit over the idle for DAZ studio at 0.3 GBs but you nailed that one!!

     

    Thanks again for taking the time to help me with this.   

  • s_j_gregorys_j_gregory Posts: 104
    TheKD said:

    Tons of morphs yeah. She loads in with subd 1 and render subd 2, default texture set as far as I can tell, they all have BASE in the texture map name.

    The KD,

    This sounds the same for me.  All 'default' stuff.  Just wanted to reduce the number of variables. 

    Based on the shape difference I am seeing in particular with G8F (between old and new visual inspection) I am wondering if there are a bunch of 'non-zero default' morphs that are not showing as currently used but are none the less consuming RAM??  But 2 GB worth??

  • TheKDTheKD Posts: 2,696

    A few days ago I scrolled the whole frikken parameter and shaping lists, looking for that, found one morph in the whole list nonzero default, fixed the one I found. I wish there was an easier way, like using a >0 in the filters or something, because scrolling that massive list takes fooooooooooooreeeeeeeevaaaaaaa, like that dude in sandlot says lol. 

  • fixmypcmikefixmypcmike Posts: 19,598
    TheKD said:

    A few days ago I scrolled the whole frikken parameter and shaping lists, looking for that, found one morph in the whole list nonzero default, fixed the one I found. I wish there was an easier way, like using a >0 in the filters or something, because scrolling that massive list takes fooooooooooooreeeeeeeevaaaaaaa, like that dude in sandlot says lol. 

    Click on "Currently Used' in the Parameters Pane

  • TheKDTheKD Posts: 2,696

    That only shows stuff that is not on the default value I thought?

  • s_j_gregorys_j_gregory Posts: 104

    Hear you.  I don't think that is going to get me to where I want to be.  What this might be looking like, depending on what Richard is able to tell me, is that I am going to have to park/disconnect my directory that includes G8 and build a new one with a very conservative collection of morphs.  I am not keen on this because every time I play with Runtimes I end up confusing myself and worry about causing more trouble than good.  But the status quo is not okay.  Everything is too hard.  (AAAAAAND, I have a brand new RTX 2070 with 8 GB of VRAM that more often than not is a paper weight because the CPU does the rendering if I have anything like two characters with clothes and hair.) 

  • Richard HaseltineRichard Haseltine Posts: 102,291
    hzr said:

    Create two versions of your genesis X figure. One for setup of new characters, and one for posing and using it. The creation one can be the default one and stay as it is, and you just setup characters with it. And when you saved a character with it, simply copy the used morphs to your new lighter figures morph data folder.

    The lighter one should only contain pose related morphs and joint controlled fix morphs, and the morphs that you actually use on your custom character(s). You can create such a custom figure by selecting "File > Save As > Support Asset > Figure/Prop Asset.

    Make sure to strip out all the morphs from that figures data folder that you do not need. I would recommend taking out all those hd shapes especially since they take alot of time to load and several can really lead to insane loading times. My default g3 - g8 runtimes are massive, several gigs of morph data from stuff I bought over the years and my own morphs, so I usually look at loading times of several minutes if I would use the default figure for my characters... I could not suffer this anymore, so this is a workaround that indeed works well.

    I still would like to see some system by DAZ so you can manage which morphs the figure should skip loading by default, perhaps on a global level or even per file. That way you could really get away from those insane load times once your runtime starts filling up. It is not just loading times, it also makes pose loading and saving slower, especially when youre working with the excellent puppeteer pane.

    Any selective loadign system for morphs that are isntalled is likely to run into even more overhead than the current system (after all, this has been refined over successive approaches with different figures). There is of course one option,  which is simply to uninstall the unwanted morphs (if the CMS knows about your purchases then DS will offer to download and install any morphs from Daz products that are needed, owned, and not currently installed).

  • Richard HaseltineRichard Haseltine Posts: 102,291
    TheKD said:

    That only shows stuff that is not on the default value I thought?

    It should show non-zero values, this was a change some time back.

  • TheKDTheKD Posts: 2,696
    edited May 2020

    Weird, the one I found was set default 25%, but wasn't showing up under currently used. Yeah, if it's supposed to show all non-zero instead of all morphs not at default, it's not working for me in the beta or general release. Just did a test, set a few morphs to default as non-zero, and none of them showed up as currently used.

    Post edited by TheKD on
  • s_j_gregorys_j_gregory Posts: 104
    TheKD said:

    That only shows stuff that is not on the default value I thought?

    It should show non-zero values, this was a change some time back.

     

    TheKD said:

    Weird, the one I found was set default 25%, but wasn't showing up under currently used.

    Interesting.  But I am also looking for a difference of 2 or 3 GB of RAM consumption loading a G8F base or an out of the box G8F character from one system to another.   So one would have to find a goodly number of nonzero default morphs to account for that.  Or my suggestion is that that alone will not account for the size of the difference here. 

  • TheKD said:

    That's odd, you would think after deleting the figures, it would go back to around the baseline idle, but doesn't

    daz doesn't give up memory once it's taken have to restart daz to clear it's brains. 
    ----

  • s_j_gregorys_j_gregory Posts: 104

    I have not given up but I do have bad news.  I spent the last couple of days doing maintenance on my runtimes.  I have three DAZ Studio content runtimes and two Poser content runtimes spread over multiple drives on my newer system.  Over the years, h/w replacements, various backup strategies, content redistribution to manage load/size, and DIM initiated refreshes to inconsistent destinations and I had multpiple instances of many folders with considerable duplication or updates.  I can confirm, after more than two full days of dedicated effort that the DAZ content is clean of duplication to a reasonably high level of confidence.  So what you ask.  Well I was hoping this was contributing to the size of the G8F RAM consumption.  And the bad news - the effort, though satisfying, made no difference to the amount of RAM being consumed.  Furthermore I tried another experiment.  I loaded G8F Dayton into an empty scene.  As previously reported, RAM consumption jumped from 326 MB to 2639 MB.  I zeroed the figure.  I applied the Dayton head and body morphs and saved as a scene subset.  Restarted DAZ Studio and loaded my scene subset.  And there it was: 2639 MB of RAM being consumed.  My unfortunate conclusion is that the RAM consumption (increase) is the morphs - zero or not. . . . .  I need to either reduce the number of morphs I am carrying or I need a more robust computer.  And the snag is not just the RAM consumption - I have that; it's that other than for rendering I only manage to get one core working.  With the increase in use of dForce clothing and hair, as but one arguement, one nearly must come to grips with timeline simulations.  When saving a keyframe takes 20 minutes, I need to do something . . . .  Still trying. . .

  • TheKDTheKD Posts: 2,696

    I am pretty sure that installed expressions also add to the overhead too. I am going to test that theory when I have time. I am planning on wiping out my G8F runtime entirely and starting again. A lot of characters I installed for the skiin only, but was too lazy to strip everything else out when I installed, bit me in the ass in the long run lol. Was going to start that today, but some other crazy person called me for work, so I go to work.

  • s_j_gregorys_j_gregory Posts: 104

    TheKD,

    I am quite interested in comparing thoughts on this.  If I resign myself to the fact that it is the morphs that are slowing me down, and that (short of a new more powerful machine) my only option is to reduce the morph load, what might be the plan.  I also am wondering if all morphs are created equal.  If character morphs (FBMs) are sets of deltas that are sitting there waiting to be called, that's a lot of overhead.  And I have 150 G8F characters - that might be a lot.  I am thinking that it is character morphs that are having the largest impact (but I don't really know); but there may be multple sets of other morphs (like breast morphs) that are performing very similar functions.  I am perhaps thinking the other way round to you.  In my plan:

    1)  Perhaps the first thing to do is to do away with those characters that I don't like/use.  Too bad I couldn't trade them in for a new model. . . .

    2)  As you have said, find a way to retain the MATs for those characters that I like for only their skin or makeup and blow away the shape presets and character sliders (and killing the data file??)  

    3)  I wonder in cases where there is a separate head morph, if there is a step for me where I retain the head morphs of some characters but not the bodies.  Not sure I know how to do that yet but I don't think it would be that hard.

    4)  Retain completely some significantly smaller number of characters.

    Steps 1 and 2 would directly reduce RAM consumption while 3 and 4 would retain an impact.

    I don't know what the mix of these scenarios might be.  I have no idea what the various load implications of this equation might be.  I find myself wondering.... Are there really clever folks out there, for instance, that have (picking numbers somewhat randomly): ten complete characters that they really like (perhaps some are HD and some not) and ten additional head morphs that they sometimes mix and match.  These are then used as a basis for adjustments.  They may also have some collection of skins, as you are thinking, that they use as start points.  (With no intention to insult, I have noticed some producers have a common look to their families of characters.  Perhaps I only need a couple SASE characters.  (I have twelve x SASE; 4 x 3DUniverse, 6 x Devious Dolls, 7 x Digital Touch. . . . .)   So maybe this is not a bad approach.)

    I picked my numbers specifically so I could say . . . this might reduce the contribution to RAM consumption from character morphs by 90%. 

    This would be a lot of work.  And I am not sure if this is the major source of RAM consumption or not.  Thoughts??

  • s_j_gregorys_j_gregory Posts: 104

    To your point about expressions - I have 44 (G8F) expression packages plus some number that are burried in the poses folder.  I don't know what the RAM consumption impact for them might be or if there is a difference between those that are in the expression folder and those that are included with poses.  And this might highlight my lack of understanding. 

    You might recall early on I was talking about a number of related issues - or issues I thought were realted to this RAM thing and it included saving keyframes and zeroing a pose etc.  I might understand zeroing a figure taking a long time because all the morphs are loaded into RAM and they all have to be checked that they are zero, but zeroing the pose is not going through the pose collection, it is just checking all the bone rotations. . . .

    I always thought posing was rigging/bones and shaping was morphs/mesh.  So as you have said it seems logical that expressions are at least partly (mostly) morphs (I want to say that some also move rigging like the jaw or tongue? Maybe I'm wrong.) and at least some of them (that include dials?) would be loaded into RAM and be contributing to RAM consumption.  Are there expressions that have no dials (only presets?) that behave more like poses and are not retained in RAM?  Or is the 'morph' in RAM anyways?

  • TheKDTheKD Posts: 2,696

    I think if it adds a dial, it adds to the RAM overhead. Genesis figures are a lot mor complex than like victoria 4. In the push to realism, there are a lot of JCM built right into the base. It's why the armpits for example look so much better than V4.

     

    Was able to run some base tests, here is the results


    G8F Starter Essentials                              File loaded in 0 min 29.5 sec            577MB RAM Load
    64 expressions with dials added                 File loaded in 0 min 3.6 sec              633MB RAM Load
    Starter, 41 non-HD Bases                          File loaded in 0 min 27.1 sec.           1,194MB RAM Load
    Starter, 41 Base + HD                               File loaded in 0 min 14.4 sec            1,104MB RAM Load
    Starter, 41 Base + HD + My Essentials      File loaded in 0 min 21.1 sec            1,398 MB RAM Load

    The last test, "my essentials" includes the packs like glute control, breast control, and all my 3rd part naughty geographs lol. It's oddly inconsistant, not sure why. In hindsight I should have did a few loads of each setup, then averaged them. Oh well. The test did convince me that I should do a clean reinstall though, that snappy load time was great. I am planning on doing a test, seeing if regular poses have an effect, and also clothes and hair before I proceed though.

  • s_j_gregorys_j_gregory Posts: 104

    To your first point, clearly there is a trade-off in terms of quality.  And I agree, some of it is quite worth it.  I am still struggling with the large difference in RAM consumption between my old system and my new system.  Both running 4.12 +/- and (scroll up to the table) with G3F and G8F base loads the jump from old to new is roughly 1 GB and 2 GB respectively.  As you will note above I am being convinced the difference is morph loading and I thought character morphs was a good place to start.  Now I am not so sure.  I am more than a bit cautious/conservative so my experiment of the day may be a bit pedestrian to you.  I used my older system and created Characters - Heads and Characters - MATs folders in my Genesis 8 Female folder.  So far so good.  I moved the bits and pieces (.png, .tip and .duf files) about and proved my earlier concept.  I then moved the data files in and out of my runtime, restarting and testing RAM consumption.  I used only one (HD) figure FWSA Sharah which was lucky because it reminded me some characters acttually use morphs from a parent. . . . . But it might not be a large enough sample.  Anyways:

    at idle the older system, with DAZ Studio running, consumes 214 MB, with G8F loaded the RAM consumption goes to 690 MB....Restart

    removing FWSA Sharah (depends on Olympia 8) and loading G8F, the RAM consumption drops to 688 MB....(small cry)....Restart

    adding FWSA Sharah Head morph and loading G8F, the RAM consumption goes to 689 MB

    So.  VERY small test.  But deleting 100 characters and saving 200 MB RAM consumption is not getting me to the 2 GB RAM consumption difference between old and new systems very quickly.  Or said another way, I must have a huge amount of morph sets (as you indicate in your 'My Essentials') contributing to this load.  I am now thinking that is the place to start but I am less certain how to do that.   

  • TheKDTheKD Posts: 2,696
    edited May 2020

    One thing I though of, old system, and new system. Are they both using the same OS? If old system is win7 and new one is win10, that would explain a large portion of the difference. Just about everything I have tested win7 vs win10, win10 uses a big chunk more RAM.

    I think I have my future game plan, at least as far as morph installs go. Going to install a handfull of full morph sets, like one fatty, one model thin, one average, one elf, like that. Have one nice base for everything I can think of, then I can use the essentials to tweak the base I load. Go through the rest, only installing nice looking faces, getting rid of all the body morphs and the load files for them. The rest just install the textures and their load files. That should lighten the load a lot from my bloated version I think lol.

    Post edited by TheKD on
  • s_j_gregorys_j_gregory Posts: 104

    Yes both are Win10. 

    I am not sure my test results support your plan - as that was originally my hope too.  If a FBM is only contributing 1 MB of RAM consumption you might save a couple hundred MB.  (If my one trial is indicative?)  Clearly it is not the only source but I am not even sure now it is the main source of RAM 'bloat'.  I am most interested in what you learn because as I look at my runtime it looks like a significant amount of effort.  I want to be sure it is worth it.   

  • TheKDTheKD Posts: 2,696

    It could just be a few sets are screwed up somehow making most of the ram usage and time to load, but it would take a year to install one thing, test, install one thing, test lol. As it is now, this project will probably take a few weeks, or maybe more, people just keep calling with more work. Making money is nice and all but I would like to have some time to work on my side projects lol.

  • fixmypcmikefixmypcmike Posts: 19,598
    TheKD said:

    It could just be a few sets are screwed up somehow making most of the ram usage and time to load, but it would take a year to install one thing, test, install one thing, test lol. As it is now, this project will probably take a few weeks, or maybe more, people just keep calling with more work. Making money is nice and all but I would like to have some time to work on my side projects lol.

    You could do a bubble sort, try half the items, then the other half, and keep taking half of whichever has the most effect.  If you install with DIM or Connect it should be fairly quick.  You could start with just the top suspects, such as HD morphs.

  • mindsongmindsong Posts: 1,712

    This is interesting - if mostly in the variability being discovered. wow.

    FWIW, *if* you have the GenX2 product from the DAZ store, the 3rd (?) tab can be used to enable/disable some/all of the available figure morphs via checkboxes - (at least with genesis1 figures/morphs). It simply renames the checkboxed morph *.dsf files to *.dsf.OFF in the existing folders in the background, and a reload ignores them.

    For testing and/or general use, this might be a handy tool/feature to have available and know about.

    I haven't tried this w any of the current Genesis figures (2,3,8), but the mechanism is simple enough that it might simply work. The developer died before G8M/F was born, so it may not 'see' those figures at all.

    Also, the same PA (D3D) wrote a highly underrated utility called the Content Gatherer, that will allow you to define and bundle a figure/scene and any/all of its dependencies into a minimized runtime/lib and save it as a zipfile. Maybe that will be useful to some of your purposes. For my projects, I created a complete-but-bloated runtime/lib that contains *all* of my figure's morph resources for my character creation phase.When the figure is 'right', I 'gather' it into a small/tight dedicated runtime/lib (adding expressions, visemes, and other likely-to-be-used morphs) that I use for my active animation projects - with none of the bloat. Godsend tool. YMMV.

    cheers,

    --ms

  • s_j_gregorys_j_gregory Posts: 104
    edited May 2020

    TheKD,

    Loud and clear.  And look on the bright side, lots would like the chance to make some money at this time.  I have said when I look at my runtimes I confuse myself. . . .  In addition to characters, my data folders are also full of clothing.  Lots (most?) of the clothing includes morphs.  But I am thinking only when the clothing is added are the morphs brought into RAM???   Stop and think about that!!  Either my speculation is not true and I have an entirely different/additional collection of items to consider, including things I have converted and/or saved with Fit Control morphs that I need to think about.  Or it is true and there is an experiment to be done that puts character morph data somewhere, other than the data/ DAZ 3D/ Genesis 8/ Female/ Morphs path, and edits the .dufs (head and body morph files for example) to alter the path so that the morph is only loaded when applied - like clothes.  Surely someone has tried this already?  (Edit: this might be harder if the package also includes expressions.  (or is encrypted?)  I have discovered I have about 55 sets of expressions.  Plus, who new Dayton had 20 expressions of her own.  And I agree with you; they are all loaded into RAM because the dials all work for any character.  How many 'afraid' morphs do I need.  That migh be too harsh.  But how many 'afraid' morphs can I afford to carry is a good question I think.  Had I known when I was buying up all this stuff what I know now I might have been more selective.)    

    Mike,

    Like your halfing suggestion.  have to give some thought to how that might work.  Maybe I could just rename the data/ DAZ 3D/ Genesis 8/ Female/ Morphs folder and copy halves over so as to not mess up, risk messing up, the structure too much. 

    Post edited by s_j_gregory on
Sign In or Register to comment.