Is There A Plan To Speed Up G8 (and future generations) Loading Times?

2»

Comments

  • Richard HaseltineRichard Haseltine Posts: 102,252

    TheMysteryIsThePoint said:

    How can you say you can't know the details, then assert that DS is doing something it doesn't need to do - especially here, where the reason for loading all morphs (to get their relationships) has been explained repeatedly?

    Because I made a distinction between things we can observe externally without knowing the internals, and the things we cannot know. Many people have observed it empirically by having their character distorted by an errant morph that the scene does not even explicitly use.

    that happens when the asset is icorrectly saved with a non-zero default value - oe when the relationships on a correction are not set properly

    And just because a particular has been explained, repeatedly as you say, does not mean that the behaviour represents a good design. My point is that a design that requires loading morphs that are not even used, for whatever reason, is not a good design. It may have been an acceptable at some point, but it didn't scale and today it is not. I don't think the people suffering from its artefacts are very much consoled by the "explanation"; my doctor telling me "Well, you see, your head hurts because your hair appears to be on fire" doesn't make it any less painful.

    Let's stop making apologies for a bad design that needs to be re-thought for DS to keep moving forward.

    It isn't bad design, though - it is the latest iteration in a process that enables a figure to be given extra morphsin a flexible manner. It's very good at that, but there is an overhead. Perhaps the next iteration will be able to build further.

  • j cadej cade Posts: 2,310

    ehh were you here for the v4 era? pre genesis morphs didn't load until you told them to aaaannd people hated it. it was an awful design

     

    you had to manually load in any morp sets you planned to use one by one  to load the figure you: loaded the figure scrolled to the head  and body morphs clicked the preset to load them in waited for them to load and set up - oh you have some after market fix morphs for bending? scoll to where they are click the preset let them load. Oh you want to use some of the head or body morph from this character scroll to their folder click the presets let them load and on and on and on.

     

    The loading speed could improve for sure, but I still find it faster to set up a genesis character with that load time than systems where i can just load what I want but I have to manually select what I want, and with the front load time I can just go away and do something for the minute it takes to load - with something more manual, even if it were faster, it would still be active time as opposed to time I can just switch over and browse the interwebs on.

     

    again not saying its pervect but there are very valid advantages to the design

  • onixonix Posts: 282

    In my opinion, MorphSoandso should contain references to JCMSuchandsuch,  and BendThatone.   If not, they should not be loaded.

    But by your explanation, it seems, that's exactly what is happening. This JCM gets loaded then it monitors morph changes and possibly does some corrections by itself without getting permission from the user.

    Since this JCM can't be aware when a certain morph will be activated until it gets activated, Daz creates some kind of complex structure of signals to trigger appropriate CMs which still slows down everything and takes lots of time to load. (which would explain why you don't want post-loading. as that would require a lot of time again to rebuild all that complex structure)

    I would guess that Daz reads all morph files first then it will parse everyone of them and fill the list of controllers and subcomponents for each of them

    Because of that behavior, I potentially could create a morph that turns on morph A from vendor X if Morph B of vendor Y is activated without the user even being aware of what is happening which is a huge security issue. 

    Also, it seems that the loop could be formed when CM's start correcting each other

    This is a bad design in principle which was not that visible in the beginning but now manifesting itself and pretty hard to fix. Probably it would be still the best idea to get rid of all those correctives entirely.  There are better ways to get the same result.

     

    If you hide morphs how do you know you are dealing with all the associated files? "   

    Now I understand what you mean because by the Daz architecture if I load Victoria morph  Daz is supposed to load a Corrective for the smile expression which will change the way how "smile" works specifically when Victoria morph is activated and since this Corrective is not mentioned anywhere in the Victoria morph it cannot be loaded.

    Although when we hide files manually this is not an issue.

    The most obvious solution would be to add an extra file to each morph which will list all dependencies of what else should be loaded when this morph is activated. 

     

    And actually just now, I noticed what kind of mess is happening with parameters and how some vendors could be potentially screwing up things accidentally or on purpose by adding CMs to various base morphs without me being even aware of that. Many Daz-provided base morphs are littered with various irrelevant CMs that are not going to be used ever. Nobody even knows what effect they may have.

    All that stuff could be safely removed without any loss and may even speed up everything.

     

  • onixonix Posts: 282

    marble said:

    I can see why the OP, as a content creator, needs so many characters and their morphs installed. My load times, by comparison, are very short - <10 seconds for a G8 character. I assume that is because I have only around 20 characters in my library. I do have a lot of morph packs though - mosty from Zev0. So maybe the solution is to buy less pre-made stuff and dial-in more shapes using the morph packs? I know that some people have hundreds of characters and that they have excruciatingly long load times.

     

    That would be a good idea, but the problem is that you usually try something and then you cant even get rid of it easily because it is scattered around the content directory everywhere  and all that trash just accumulates forever until you experience HDD crash LOL

    Also, not every character is the same. I just now noticed that some very clean and contain just one of a few morphs while others litter everything with correction morphs for an unknown reason. Probably some creators believe that they are super smart when they inject correction morph into existing morph from another vendor rather than inserting that existing morph of another vendor into their own morph.

     

  • thenoobduckythenoobducky Posts: 68
    edited April 2021

     I think a short tutorial on ERC formula and how Daz store / find / load / use those formula would be beneficial in this discussion. That way people would understand how things work, why it work that way, the benefit and limitation and possible alternatives.

     

    onix said:

    Because of that behavior, I potentially could create a morph that turns on morph A from vendor X if Morph B of vendor Y is activated without the user even being aware of what is happening which is a huge security issue. 

    I can see this be very useful for things like clothing fixers or conversion tool.

    Also, it seems that the loop could be formed when CM's start correcting each other

    This is a bad design in principle which was not that visible in the beginning but now manifesting itself and pretty hard to fix. Probably it would be still the best idea to get rid of all those correctives entirely.  There are better ways to get the same result.

    Daz have logic to detect circula references as can be observed by reading log file.

    If you hide morphs how do you know you are dealing with all the associated files? "   

    Now I understand what you mean because by the Daz architecture if I load Victoria morph  Daz is supposed to load a Corrective for the smile expression which will change the way how "smile" works specifically when Victoria morph is activated and since this Corrective is not mentioned anywhere in the Victoria morph it cannot be loaded.

    Although when we hide files manually this is not an issue.

    The most obvious solution would be to add an extra file to each morph which will list all dependencies of what else should be loaded when this morph is activated. 

    I think Daz already have those infomation stored with the formula, but the question is what would be the trade off. A system that delay load everything would have less consistant performance when applying a pose or expression as daz have to find missing morphs. It could be slower for people with fewer morphs installed as it makes harder to cache things, resulting in more file I/O that would be slower. Should Daz scan all morphs files to create all the sliders for shapes and poses, because this is very useful for novice user to find things they need. But scanning all those files and creating slider might be half of the required work already and might just as well go ahead and finish creating the formulas so it will just work. 

    Hiding stuff from Daz wont fully fix the issue because then you can't use an expression or morph that is not loaded until you reload the character / scene. Some sort of load formulas used by this character / expression / shapes function would be super useful.

    With regard to multithreading I can see how to figure out if a formula referenced in another formula already exist or being made in a thread safe way being a difficult problem to tackle and have potential to accidentally make everything single threaded.

    In short, any way to solve a complex problem will have advantages and disadvantages. I would prefer things to work correctly over being fast, and beginner friendly over complicated to use but faster for advanced user. But there are definitly still rooms for improvement. For example some sort of rescan a selected base directory to add newly installed morphs functionality would be useful.

    Post edited by thenoobducky on
  • PerttiAPerttiA Posts: 10,024

    onix said:

    Also, not every character is the same. I just now noticed that some very clean and contain just one of a few morphs while others litter everything with correction morphs for an unknown reason. Probably some creators believe that they are super smart when they inject correction morph into existing morph from another vendor rather than inserting that existing morph of another vendor into their own morph.

    It's not only JCM's, but also morphs and characters from some other PA, as one cannot copy the work of someone else to one's own product.

    If you make a character that is built on top of Victoria 8, then the dial for your character should activate the Victoria 8 dial as well. The same with products like "Growing Up" or even the DAZ Head and Body Morphs.

  • onixonix Posts: 282

    thenoobducky said:

    I can see this be very useful for things like clothing fixers or conversion tool.

    Those same things still can be done but rather than doing them secretly, vendors should be required to ask users if they want those fixers to be turned on.

    The current situation is very strange to say mildly when things are done backward for an unknown reason.

     

     

    I think Daz already have those infomation stored with the formula, but the question is what would be the trade off. A system that delay load everything would have less consistant performance when applying a pose or expression as daz have to find missing morphs. It could be slower for people with fewer morphs installed as it makes harder to cache things, resulting in more file I/O that would be slower. Should Daz scan all morphs files to create all the sliders for shapes and poses, because this is very useful for novice user to find things they need. But scanning all those files and creating slider might be half of the required work already and might just as well go ahead and finish creating the formulas so it will just work. 

    Richard explained that all this scanning task is not really that time-consuming to care about and it can be multithreaded easily. also it can be safely deferred and just continued after your scene is already loaded.

     

    Hiding stuff from Daz wont fully fix the issue because then you can't use an expression or morph that is not loaded until you reload the character / scene. Some sort of load formulas used by this character / expression / shapes function would be super useful.

    Of course, you cant use what you hide, but the reason to hide things I precisely because they are not required in that scene

    With regard to multithreading I can see how to figure out if a formula referenced in another formula already exist or being made in a thread safe way being a difficult problem to tackle and have potential to accidentally make everything single threaded.

    If you think more actually there is no reason why multithreading cannot be implemented even now.

    My guess in how Daz works is that first, it loads a full list of morphs then it iterates through every one of those morphs and populates controllers and subcomponents lists. so every correction morph that has controllers has to scan at least half of the full list every time to find where it will inject references to itself.

    This process can be easily multithreaded as you can process multiple morphs at the same time 

     

    In short, any way to solve a complex problem will have advantages and disadvantages. I would prefer things to work correctly over being fast, and beginner friendly over complicated to use but faster for advanced user. But there are definitly still rooms for improvement. For example some sort of rescan a selected base directory to add newly installed morphs functionality would be useful.

    In my understanding, the main problem here is with correction morphs as this functionality srews up everything and possibly can be the reason why loading times are so long in the first place (unless Daz actually spends most of the time allocating memory for formulas and inserting them into the table)

    Correction morphs are the reason why you can't know in advance what files you need to load because any of those new files may decide to hook up on something random in your existing system.  And if you are very paranoid about not missing any of those correction morphs you have to rescan everything every time you load a new file as you probably can not even know if some of the correction morphs wanted to reference your new file and did not find it at that time.

     

     

     

     

     

  • onixonix Posts: 282

    PerttiA said:

    It's not only JCM's, but also morphs and characters from some other PA, as one cannot copy the work of someone else to one's own product.

    If you make a character that is built on top of Victoria 8, then the dial for your character should activate the Victoria 8 dial as well. The same with products like "Growing Up" or even the DAZ Head and Body Morphs.

     

    I was talking about different issues here. Building your morph on top of another morphs is normal. 

    Let's say you want to create a smile expression morph for your character which will reference victoria 8 smile.

    normally you would just create an expression folder for your character and populate it with your custom expression morphs that refer to morphs of other vendors.

    but some perverted vendors rather do this:

    They insert extra logic into the genesis 8 smile expression morph which tells it that if character X morph is activated activate Smile correction morph for character X. You will not even be aware that you got a custom expression morph along with that character.

    So if you will ever install this character it will be sucking your CPU power forever because every time you use smile morph that formula for character X (which you maybe never even used) will be executed.

    And how many vendors may have JCMs that are constantly hooked to character joints and every time you change your pose Daz has to process hundreds of those parasitic formulas that do nothing else but just check if character X is loaded? 

    What if the vendor screws up and makes that his JCM gets activated all the time not just when their character is loaded? Now your figure will be screwed up forever because you have no way to even identify buggy JCM

     

     

     

     

     

  • onix said:

    I think Daz already have those infomation stored with the formula, but the question is what would be the trade off. A system that delay load everything would have less consistant performance when applying a pose or expression as daz have to find missing morphs. It could be slower for people with fewer morphs installed as it makes harder to cache things, resulting in more file I/O that would be slower. Should Daz scan all morphs files to create all the sliders for shapes and poses, because this is very useful for novice user to find things they need. But scanning all those files and creating slider might be half of the required work already and might just as well go ahead and finish creating the formulas so it will just work. 

    Richard explained that all this scanning task is not really that time-consuming to care about and it can be multithreaded easily. also it can be safely deferred and just continued after your scene is already loaded.

    I interpret Richard's comment as saying reading the file from disk is not the bottleneck, which I think is correct, it is converting from the text file to objects that Daz can use that is slow. Put it another way, reading file not slow, scanning the content of the file to figure out what to create slow. Daz would need to parse all the formulas to figure out what the figure would finally look like. Because there is no mechanism for Daz to figure out if a formula is active or not (my limited understanding is a formula can either by an input like a dial or applying the dial to something so daz need to figure out the value of all dials and apply all the formulas even if most inputs by default are zero). This is also the reason for morph corrupting the default figure. Now I think about it, this part what is what is causing the load time to be slow cannot be skipped unless there is a way to tell Daz don't load this morph (currently done by hiding it from daz). Why this is not trivial to multithread is addressed below. 

     

    Hiding stuff from Daz wont fully fix the issue because then you can't use an expression or morph that is not loaded until you reload the character / scene. Some sort of load formulas used by this character / expression / shapes function would be super useful.

    Of course, you cant use what you hide, but the reason to hide things I precisely because they are not required in that scene

    No what I mean is if I have a figure loaded and then I unhide a character preset or morph set, I can't use it unless I reload the figure because there is no other way to get new shape dials and pose dials to appear.

    The senario I want to be addressed is this: I have a figure loaded. I install zev0's Shape Shift, how do I use it without reloading. 

    Getting a way to delay load morph / reload morph contained in a base directory would really improv

     

    With regard to multithreading I can see how to figure out if a formula referenced in another formula already exist or being made in a thread safe way being a difficult problem to tackle and have potential to accidentally make everything single threaded.

    If you think more actually there is no reason why multithreading cannot be implemented even now.

    My guess in how Daz works is that first, it loads a full list of morphs then it iterates through every one of those morphs and populates controllers and subcomponents lists. so every correction morph that has controllers has to scan at least half of the full list every time to find where it will inject references to itself.

    This process can be easily multithreaded as you can process multiple morphs at the same time 

    I agree its possible to multithread, just the implementation will be non trivial. The issue to consider is: thread 1 load morph A which reference morph B, at the same time thread 2 is loading morph B. How can thread 1 figure out morph B has been loaded or not. Its solvable but care must be take to ensure the solution dont accidentally end up being single threaded. Another issue: thread 1 create morpha A with name X, thread 2 create morph B also with name Y, how does Daz ensure the names are unique because duplicate formula is an issue that need to be detected.

    In short, any way to solve a complex problem will have advantages and disadvantages. I would prefer things to work correctly over being fast, and beginner friendly over complicated to use but faster for advanced user. But there are definitly still rooms for improvement. For example some sort of rescan a selected base directory to add newly installed morphs functionality would be useful.

    In my understanding, the main problem here is with correction morphs as this functionality srews up everything and possibly can be the reason why loading times are so long in the first place (unless Daz actually spends most of the time allocating memory for formulas and inserting them into the table)

    Correction morphs are the reason why you can't know in advance what files you need to load because any of those new files may decide to hook up on something random in your existing system.  And if you are very paranoid about not missing any of those correction morphs you have to rescan everything every time you load a new file as you probably can not even know if some of the correction morphs wanted to reference your new file and did not find it at that time.

    What do you mean correction morph, I guess you are talking about erc formulas? Can you give some concrete example? I find all the talk so far about high level concept a bit confusing and too much guess work. I think what is slow is loading formulas, but a lot of formulas are actually morphs and expressions. For example, Victoria 8 have something around 60-70 morph file, each file contain around 2 formulas. Whereas eJCMSilly_HD_div2.dsf from Genesis 8 Female Expression pack can have 100+ formulas. 

    Now a formula looks like this:

    {
        "output" : "Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/DAZ%203D/Base%20Pose%20Head/eCTRLNostrilsFlex.dsf#eCTRLNostrilsFlex?value",
        "operations" : [
        { "op" : "push", "url" : "Genesis8Female:#Z%20FO%20Angry?value" },
        { "op" : "push", "val" : 1 },
        { "op" : "mult" }
        ]
    },

    As you can see, Daz know where to find the formula it references. So that is no a problem. One problem would be in a multi thread safe way to figout out if the output it refernces already exist or is being created by another thread or it need to create it. Again now I have think about this a bit more, the issue is Daz dont know if this formula will result in no change at all or cause some morph changes until Daz parsed the formula and applied it, which is the slow process we are trying to avoid here. There is no mechanism to tell Daz to skip formula in this folder until I tell you to load it functionality. I think something like this for advanced user would be useful.

    Another problem, for example I like to use Zev0's shape shift which contains lots of formulas, but I use it often when creating a character so I would like Daz to always load it. How would Daz know which morph need to be loaded? Currently the only way is to selectively add base folders, this is almost workable with manual curation of installed files, or program to automate it, maybe Daz can come up with a better approach (maybe by reworking the way files are installed).

    In short we need a better way to optionally tell Daz skip loading certain folder until it is needed by some file.

  • thenoobduckythenoobducky Posts: 68
    edited April 2021

    onix said:

     

    I was talking about different issues here. Building your morph on top of another morphs is normal. 

    Let's say you want to create a smile expression morph for your character which will reference victoria 8 smile.

    normally you would just create an expression folder for your character and populate it with your custom expression morphs that refer to morphs of other vendors.

    but some perverted vendors rather do this:

    They insert extra logic into the genesis 8 smile expression morph which tells it that if character X morph is activated activate Smile correction morph for character X. You will not even be aware that you got a custom expression morph along with that character.

    So if you will ever install this character it will be sucking your CPU power forever because every time you use smile morph that formula for character X (which you maybe never even used) will be executed.

    And how many vendors may have JCMs that are constantly hooked to character joints and every time you change your pose Daz has to process hundreds of those parasitic formulas that do nothing else but just check if character X is loaded?

    What if the vendor screws up and makes that his JCM gets activated all the time not just when their character is loaded? Now your figure will be screwed up forever because you have no way to even identify buggy JCM

    Can you give an example of this? I would really like to see it.

    Edit: Also I dont get the point of it, For example, anything in /data/Daz 3D/Genesis 8/Female/Morphs get loaded when you load genesis 8 female. Normal convention is morph for a product go to /data/Daz 3D/Genesis 8/Female/Morphs/Vendor/Product. So it would get automatically loaded already, placing things in /data/Daz 3D/Genesis 8/Female/Morphs/Daz 3D/Character would give no benefit and would not be treated any differently by Daz. The buggy JCM would easily be part of the vendor's file.

    Post edited by thenoobducky on
  • onixonix Posts: 282

    thenoobducky said:

    I interpret Richard's comment as saying reading the file from disk is not the bottleneck, which I think is correct, it is converting from the text file to objects that Daz can use that is slow. Put it another way, reading file not slow, scanning the content of the file to figure out what to create slow. Daz would need to parse all the formulas to figure out what the figure would finally look like. Because there is no mechanism for Daz to figure out if a formula is active or not (my limited understanding is a formula can either by an input like a dial or applying the dial to something so daz need to figure out the value of all dials and apply all the formulas even if most inputs by default are zero). This is also the reason for morph corrupting the default figure. Now I think about it, this part what is what is causing the load time to be slow cannot be skipped unless there is a way to tell Daz don't load this morph (currently done by hiding it from daz). Why this is not trivial to multithread is addressed below. 

    If parsing the file would be a bottleneck then multithreading would be an obvious solution  as I understand bottleneck is the function that populates morph dependencies with appropriate formulas

     

    No what I mean is if I have a figure loaded and then I unhide a character preset or morph set, I can't use it unless I reload the figure because there is no other way to get new shape dials and pose dials to appear.

    This is not a problem because Daz should read all names to get a list of sliders and built all that property tree, It should just not go any deeper and don't loom at their dependencies until something from that tree is activated.

    The senario I want to be addressed is this: I have a figure loaded. I install zev0's Shape Shift, how do I use it without reloading. 

    Getting a way to delay load morph / reload morph contained in a base directory would really improve

    This would be pretty easy to do if morphs are straightforward, but unfortunately, the Daz logic is too perverted here. which I will explain later

     

    I agree its possible to multithread, just the implementation will be non trivial. The issue to consider is: thread 1 load morph A which reference morph B, at the same time thread 2 is loading morph B. How can thread 1 figure out morph B has been loaded or not. Its solvable but care must be take to ensure the solution dont accidentally end up being single threaded. Another issue: thread 1 create morpha A with name X, thread 2 create morph B also with name Y, how does Daz ensure the names are unique because duplicate formula is an issue that need to be detected.

    If you look at the morph properties it is not that hard, after all, you can see that every property parameter contains Sub-components and controllers  list which has to be populated, (I think populating that list takes most of the time.)

    I can't see any problems with multithreading because every entry in that list is independent. You load a morph and then add the appropriate formula to the appropriate list the order of how things will be added to the list does not matter.

     

    What do you mean correction morph, I guess you are talking about erc formulas? Can you give some concrete example? I find all the talk so far about high level concept a bit confusing and too much guess work. I think what is slow is loading formulas, but a lot of formulas are actually morphs and expressions. For example, Victoria 8 have something around 60-70 morph file, each file contain around 2 formulas. Whereas eJCMSilly_HD_div2.dsf from Genesis 8 Female Expression pack can have 100+ formulas. 

    Now a formula looks like this:

    {
        "output" : "Genesis8Female:/data/DAZ%203D/Genesis%208/Female/Morphs/DAZ%203D/Base%20Pose%20Head/eCTRLNostrilsFlex.dsf#eCTRLNostrilsFlex?value",
        "operations" : [
        { "op" : "push", "url" : "Genesis8Female:#Z%20FO%20Angry?value" },
        { "op" : "push", "val" : 1 },
        { "op" : "mult" }
        ]
    },

    As you can see, Daz know where to find the formula it references. So that is no a problem. One problem would be in a multi thread safe way to figout out if the output it refernces already exist or is being created by another thread or it need to create it. Again now I have think about this a bit more, the issue is Daz dont know if this formula will result in no change at all or cause some morph changes until Daz parsed the formula and applied it, which is the slow process we are trying to avoid here. There is no mechanism to tell Daz to skip formula in this folder until I tell you to load it functionality. I think something like this for advanced user would be useful.

    Another problem, for example I like to use Zev0's shape shift which contains lots of formulas, but I use it often when creating a character so I would like Daz to always load it. How would Daz know which morph need to be loaded? Currently the only way is to selectively add base folders, this is almost workable with manual curation of installed files, or program to automate it, maybe Daz can come up with a better approach (maybe by reworking the way files are installed).

    In short we need a better way to optionally tell Daz skip loading certain folder until it is needed by some file.

    I can't comment on that because I need to see a complete picture, I will give an example in the next post

     

     

  • onixonix Posts: 282

    thenoobducky said:

    Can you give an example of this? I would really like to see it.

    Edit: Also I dont get the point of it, For example, anything in /data/Daz 3D/Genesis 8/Female/Morphs get loaded when you load genesis 8 female. Normal convention is morph for a product go to /data/Daz 3D/Genesis 8/Female/Morphs/Vendor/Product. So it would get automatically loaded already, placing things in /data/Daz 3D/Genesis 8/Female/Morphs/Daz 3D/Character would give no benefit and would not be treated any differently by Daz. The buggy JCM would easily be part of the vendor's file.

     

    Ok lest look at the specific example  with genesis 3 character "Dandelion"

    this character provides 4 sliders for the user: dandelion body, ears and head then Dandelion overall slider which tuns all of those at once.

    so far so good. 

    But if we now check show hidden properties we can see that 

    there are some strange files like

    MCM-Dandelion-Contempt, MCM-Dandelion-Angry, and many more that are not accessible for user

    If we look at what those files do we can see  this

     

    "formulas" : [

    {

    "output" : "Genesis3Female:#MCM-Dandelion-Angry?value",

    "operations" : [

    { "op" : "push", "url" : "Genesis3Female:/data/DAZ%203D/Genesis%203/Female/Morphs/DAZ%203D/Expressions/eCTRLAngry.dsf#eCTRLAngry?value" },

    { "op" : "push", "val" : 1 },

    { "op" : "mult" }

    ]

    },

    {

    "output" : "Genesis3Female:#MCM-Dandelion-Angry?value",

    "stage" : "mult",

    "operations" : [

    { "op" : "push", "url" : "Genesis3Female:/data/DAZ%203D/Genesis%203/Female/Morphs/MindVision-GDS/Dandelion/FHM-Dandelion.dsf#FHM-Dandelion?value" }

    ]

    (Same can be seen on the parameter settings)

    This formula  injects extra logic that if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated

    I have no clue why this is done, why the vendor of that character could not just provide a list of expressions for his character but decided to go this strange route of modifying default expressions.

     

    Obviously, without loading all morph files it is impossible to know that someone is going to do that kind of stuff and there is no way for the user to know that this is even being done.

    and this must be the main reason why Daz needs to process all files and establish all dependencies.

     

     

  • Richard HaseltineRichard Haseltine Posts: 102,252

    onix said:

    thenoobducky said:

    Can you give an example of this? I would really like to see it.

    Edit: Also I dont get the point of it, For example, anything in /data/Daz 3D/Genesis 8/Female/Morphs get loaded when you load genesis 8 female. Normal convention is morph for a product go to /data/Daz 3D/Genesis 8/Female/Morphs/Vendor/Product. So it would get automatically loaded already, placing things in /data/Daz 3D/Genesis 8/Female/Morphs/Daz 3D/Character would give no benefit and would not be treated any differently by Daz. The buggy JCM would easily be part of the vendor's file.

     

    Ok lest look at the specific example  with genesis 3 character "Dandelion"

    this character provides 4 sliders for the user: dandelion body, ears and head then Dandelion overall slider which tuns all of those at once.

    so far so good. 

    But if we now check show hidden properties we can see that 

    there are some strange files like

    MCM-Dandelion-Contempt, MCM-Dandelion-Angry, and many more that are not accessible for user

    If we look at what those files do we can see  this

     

    "formulas" : [

    {

    "output" : "Genesis3Female:#MCM-Dandelion-Angry?value",

    "operations" : [

    { "op" : "push", "url" : "Genesis3Female:/data/DAZ%203D/Genesis%203/Female/Morphs/DAZ%203D/Expressions/eCTRLAngry.dsf#eCTRLAngry?value" },

    { "op" : "push", "val" : 1 },

    { "op" : "mult" }

    ]

    },

    {

    "output" : "Genesis3Female:#MCM-Dandelion-Angry?value",

    "stage" : "mult",

    "operations" : [

    { "op" : "push", "url" : "Genesis3Female:/data/DAZ%203D/Genesis%203/Female/Morphs/MindVision-GDS/Dandelion/FHM-Dandelion.dsf#FHM-Dandelion?value" }

    ]

    (Same can be seen on the parameter settings)

    This formula  injects extra logic that if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated

    I have no clue why this is done, why the vendor of that character could not just provide a list of expressions for his character but decided to go this strange route of modifying default expressions.

     

    Obviously, without loading all morph files it is impossible to know that someone is going to do that kind of stuff and there is no way for the user to know that this is even being done.

    and this must be the main reason why Daz needs to process all files and establish all dependencies.

    Consider a simple example, a figure with big, round eyes. The morphs for cl;osing eyes are set up to work with the base shape of the eyes, something that chnages the shape or size will not combine well with that - there would probably be a gap with the closing morph at 100%, and there would probably be overlap at the edges if the morph was dialed to a value sufficient to close the eye fully in the centre 9because the rounding will increase the separation there more than at the edges). To make sure that users get the expected result when closing the eyes the creator will add an additional morph that can be applied when both their character and the closure morph are set, so the that eyelid is then properly down, and the ERC makes sure that that additional morph is applied when needed. The Dandelion character https://www.daz3d.com/dandelion-fae makes some marked chnages to the ehad shape, so there are a lot of extra morphs to make the basic expressions look right.

  • cubewotcubewot Posts: 10

    For me the very, very slow loading times with characters are the number one reason to have stopped buying new stuff like I did before.  I really do hope that this will improve in the near future... on the other hand: I am saving a lot of money with the current state of DAZ Studio.

  • onixonix Posts: 282

    Richard Haseltine said:

    Consider a simple example, a figure with big, round eyes. The morphs for cl;osing eyes are set up to work with the base shape of the eyes, something that chnages the shape or size will not combine well with that - there would probably be a gap with the closing morph at 100%, and there would probably be overlap at the edges if the morph was dialed to a value sufficient to close the eye fully in the centre 9because the rounding will increase the separation there more than at the edges). To make sure that users get the expected result when closing the eyes the creator will add an additional morph that can be applied when both their character and the closure morph are set, so the that eyelid is then properly down, and the ERC makes sure that that additional morph is applied when needed. The Dandelion character https://www.daz3d.com/dandelion-fae makes some marked chnages to the ehad shape, so there are a lot of extra morphs to make the basic expressions look right.

    Yes I understand that this may be useful, but there is a better way to do it by adding extra morph which you turn on manually rather than automatically.

    And also if default expressions don't work he can just provide his own list because it is quite possible that I will not even use basic expressions at all but use some other expression pack that is totally unaffected by those corrections. How anyone even can be aware that such corrections are taking place?

    Even If it is required to do  ERC this can be initiated with an extra controller which has to be turned on by the user or can be turned on from another morph.

    rather than injecting itself into the basic expression controller author could just add an extra controller which turns on an extra correction morph if both character and Daz expression controller are turned on.  this way the need to load all files is avoided because turning on dandelion character tells Daz that now it has to load all those correction morphs that are referenced by this character.

    The main idea of this principle is that to implement any kind of correction vendor has to add his own controller which turns that correction on and nobody can inject my corrections into any other vendor content without asking the user to turn those corrections on in some way.

    (I this situation they can be turned on by turning on the character slider which would also tell Daz to load all  those corrective morphs)  

    This looks so obvious and logical that I always thought that it is how things work in the first place and could not understand why all morphs have to be processed

    It was a big surprise that Daz even allows that kind of stuff.

  • Richard HaseltineRichard Haseltine Posts: 102,252

    onix said:

    Richard Haseltine said:

    Consider a simple example, a figure with big, round eyes. The morphs for cl;osing eyes are set up to work with the base shape of the eyes, something that chnages the shape or size will not combine well with that - there would probably be a gap with the closing morph at 100%, and there would probably be overlap at the edges if the morph was dialed to a value sufficient to close the eye fully in the centre 9because the rounding will increase the separation there more than at the edges). To make sure that users get the expected result when closing the eyes the creator will add an additional morph that can be applied when both their character and the closure morph are set, so the that eyelid is then properly down, and the ERC makes sure that that additional morph is applied when needed. The Dandelion character https://www.daz3d.com/dandelion-fae makes some marked chnages to the ehad shape, so there are a lot of extra morphs to make the basic expressions look right.

    Yes I understand that this may be useful, but there is a better way to do it by adding extra morph which you turn on manually rather than automatically.

    Better for whom? Certainly not most users who would be trying to find the right corrective morphs, once they realised that was what was needed. Espeically if they were using expression presets and didn't know immediately which expression they should be correcting. This type of function is a long-standing, and expected, feature.

    And also if default expressions don't work he can just provide his own list because it is quite possible that I will not even use basic expressions at all but use some other expression pack that is totally unaffected by those corrections. How anyone even can be aware that such corrections are taking place?

    People are likely to have and use the expressions (and in fact, many expression packs use the base expressions). People will expect them to work, hence the inclusion of fixes. Adding a slew of fesh expressions for each character would not be a practical solution.

    Even If it is required to do  ERC this can be initiated with an extra controller which has to be turned on by the user or can be turned on from another morph.

    rather than injecting itself into the basic expression controller author could just add an extra controller which turns on an extra correction morph if both character and Daz expression controller are turned on.  this way the need to load all files is avoided because turning on dandelion character tells Daz that now it has to load all those correction morphs that are referenced by this character.

    The main idea of this principle is that to implement any kind of correction vendor has to add his own controller which turns that correction on and nobody can inject my corrections into any other vendor content without asking the user to turn those corrections on in some way.

    (I this situation they can be turned on by turning on the character slider which would also tell Daz to load all  those corrective morphs)  

    This looks so obvious and logical that I always thought that it is how things work in the first place and could not understand why all morphs have to be processed

    It was a big surprise that Daz even allows that kind of stuff.

    But why would you want breakage to be the default behaviour? And how would an extra ERC controller help reduce load times?

    You do seem to be labouring under a slight misapprehension, though - nothing is injected into the default expressions in this case; rather the corrective morph has two links, one to the default expression and one to the head shape, one of which will add its value and the other of which will multiply (so the net value is head shape strength * expression strength, ensuring the correct amount of adjustment is applied).

  • j cadej cade Posts: 2,310

    onix said:

    Richard Haseltine said:

    Consider a simple example, a figure with big, round eyes. The morphs for cl;osing eyes are set up to work with the base shape of the eyes, something that chnages the shape or size will not combine well with that - there would probably be a gap with the closing morph at 100%, and there would probably be overlap at the edges if the morph was dialed to a value sufficient to close the eye fully in the centre 9because the rounding will increase the separation there more than at the edges). To make sure that users get the expected result when closing the eyes the creator will add an additional morph that can be applied when both their character and the closure morph are set, so the that eyelid is then properly down, and the ERC makes sure that that additional morph is applied when needed. The Dandelion character https://www.daz3d.com/dandelion-fae makes some marked chnages to the ehad shape, so there are a lot of extra morphs to make the basic expressions look right.

    Yes I understand that this may be useful, but there is a better way to do it by adding extra morph which you turn on manually rather than automatically.

    And also if default expressions don't work he can just provide his own list because it is quite possible that I will not even use basic expressions at all but use some other expression pack that is totally unaffected by those corrections. How anyone even can be aware that such corrections are taking place?

    Even If it is required to do  ERC this can be initiated with an extra controller which has to be turned on by the user or can be turned on from another morph.

    rather than injecting itself into the basic expression controller author could just add an extra controller which turns on an extra correction morph if both character and Daz expression controller are turned on.  this way the need to load all files is avoided because turning on dandelion character tells Daz that now it has to load all those correction morphs that are referenced by this character.

    The main idea of this principle is that to implement any kind of correction vendor has to add his own controller which turns that correction on and nobody can inject my corrections into any other vendor content without asking the user to turn those corrections on in some way.

    (I this situation they can be turned on by turning on the character slider which would also tell Daz to load all  those corrective morphs)  

    This looks so obvious and logical that I always thought that it is how things work in the first place and could not understand why all morphs have to be processed

    It was a big surprise that Daz even allows that kind of stuff.

    quick q. lets say I dial spin a character made with the Girl, Bette, and Aiko and a couple other head morphs all characters with large eyes that require jcms to close correctly. under your system how do I now close the eyes?

     

    My load time for the genesis figures could be 5 times as long as it is and I'm pretty sure this would still be slower. I only have to load once - having to dial everything specifically for every character expression every time I wanted to change it would add up astronomically

     

  • onixonix Posts: 282

    Richard Haseltine said:

    Better for whom? Certainly not most users who would be trying to find the right corrective morphs, once they realised that was what was needed. Espeically if they were using expression presets and didn't know immediately which expression they should be correcting. This type of function is a long-standing, and expected, feature.

    There is no need to look for specific corrective morphs manually. all current functionality can be fully retained with the only difference that all those morphs are turned on and loaded only when the appropriate character is turned on.

    And if not they are neither loaded in memory neither they can even potentially affect anything at all.

     

    People are likely to have and use the expressions (and in fact, many expression packs use the base expressions). People will expect them to work, hence the inclusion of fixes. Adding a slew of fesh expressions for each character would not be a practical solution.

    There is no need to lose that functionality but I am quite sure many people do not even know that it is being done. so the least that should be done is to mention this in the character description that it contains correction morphs.

    Also, most vendors do not use fixes and release separate expression packs.

     

    But why would you want breakage to be the default behaviour? And how would an extra ERC controller help reduce load times?

    You do seem to be labouring under a slight misapprehension, though - nothing is injected into the default expressions in this case; rather the corrective morph has two links, one to the default expression and one to the head shape, one of which will add its value and the other of which will multiply (so the net value is head shape strength * expression strength, ensuring the correct amount of adjustment is applied).

    Just because it would be much faster and reduce memory usage significantly.  In that situation, all morphs have to be processed just top-down and there is no longer any need to even care or load any dependencies for morphs that are not in use. Broken-buggy content no longer can harm anything and it becomes easy to identify.  This extra ERC controller essentially would act as an index to indicate what else has to be loaded and when it has to be loaded. (Now Daz just loads everything because it can't know what it should load and when.)

    Of course, I do not expect that anyone will go so far as to make changes that will beak half of the existing content so it is just theoretical thinking on how things could be done in the beginning. and I even brought it up because I was not fully aware of how things work. The current system where Daz just loads everything is simpler because vendors do not need to track their morphs either and they can just create some Correction morph and drop it for Daz to process. That alternative system would require them to track everything they do and apply appropriate changes. 

    So looks like we are quite doomed to live with this forever.  The best thing to expect now is maybe some kind of cashing so that rather than processing all this list of morphs every time, daz could just save the entire structure and just look if anything changed in the directory.

    Also, it would be possible to create those extra erc controllers automatically as it is not that hard to look up for correction morphs that are referencing some common property to index them for faster loading (and that index file also becomes extra erc controller which consolidates all those morphs ) but later ask all vendors to do things in another way when they generate those index files themselves. or just give an option to regenerate those index files from time to time when new content is installed.  This way none of the current functionality would be broken. 

    Maybe my terminology was inaccurate because I meant that extra formula is injected into the expression controller making it do some additional calculations every time it is used.

    if such a link is added, extra CPU time will be consumed to perform all those extra checks. 

     

  • onixonix Posts: 282

    j cade said:

    quick q. lets say I dial spin a character made with the Girl, Bette, and Aiko and a couple other head morphs all characters with large eyes that require jcms to close correctly. under your system how do I now close the eyes?

     

    My load time for the genesis figures could be 5 times as long as it is and I'm pretty sure this would still be slower. I only have to load once - having to dial everything specifically for every character expression every time I wanted to change it would add up astronomically

     

    For you, it would all work the same as now. You would turn on those JCMs with Girl or Aiko head sliders. Aiko or Girl vendor would be responsible that the Aiko head slider will turn on appropriate correction morphs. maybe you would have an extra slider along with other Aiko properties which say "Aiko expression correctors" but it would turn on automatically as a subcomponent as "Aiko head". and you would be able to turn it off if you decide that you don't want it.

    From the user side, no functionality is lost at all. and no extra tinkering required but content vendors now have to care that they will activate appropriate JCMs when that is necessary.

     

    Loading, unfortunately, happens more than once every time you add an extra character to your scene it takes the same amount of time. also, it affects your posing performance so you are working slower.

    Just do the test and temporarily move all morphs from the morph directory away then load G8 character and you will see how incredibly fast and responsive it becomes you even can use IK  as it works instantly.

  • j cadej cade Posts: 2,310

    onix said:

    j cade said:

    quick q. lets say I dial spin a character made with the Girl, Bette, and Aiko and a couple other head morphs all characters with large eyes that require jcms to close correctly. under your system how do I now close the eyes?

     

    My load time for the genesis figures could be 5 times as long as it is and I'm pretty sure this would still be slower. I only have to load once - having to dial everything specifically for every character expression every time I wanted to change it would add up astronomically

     

    For you, it would all work the same as now. You would turn on those JCMs with Girl or Aiko head sliders. Aiko or Girl vendor would be responsible that the Aiko head slider will turn on appropriate correction morphs. maybe you would have an extra slider along with other Aiko properties which say "Aiko expression correctors" but it would turn on automatically as a subcomponent as "Aiko head". and you would be able to turn it off if you decide that you don't want it.

    From the user side, no functionality is lost at all. and no extra tinkering required but content vendors now have to care that they will activate appropriate JCMs when that is necessary.

     

    Loading, unfortunately, happens more than once every time you add an extra character to your scene it takes the same amount of time. also, it affects your posing performance so you are working slower.

    Just do the test and temporarily move all morphs from the morph directory away then load G8 character and you will see how incredibly fast and responsive it becomes you even can use IK  as it works instantly.

    then how is what you are describing anything different than how things currently work? unless I am manually turning on those jcms one by one, what you are describing is exactly how things currently work - 'aiko_smilejcm' doesnt get dialed by 'smile' unless aiko is dialed in - otherwise it sits at zero

     

    and while loading happens eveytime you load a character that is still many many many many les interactions than anytime a jcm gets dialed since what you were describing before sounded like you wanted to make those more manual that would be for me hundred to thousands of times per session if that becomes even slightly more manual any loading speed increase would be absolutely irrelevant compared to the vast increase in time i would have to spend adjusting exra dials

     

    im just very confused by what system you are trying to explain. earlier you said

    This formula  injects extra logic that if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated

    but then

    Aiko or Girl vendor would be responsible that the Aiko head slider will turn on appropriate correction morphs. [...] but content vendors now have to care that they will activate appropriate JCMs when that is necessary.

    what is the difference between these two. functionally what difference are you seeing between "if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated"

    and

    "Aiko head slider will turn on appropriate correction morphs" as those will also only need to be on if whatever theyre correction morphs for are turned on.

     

    Both sound like systems where jcms are only turned on if the correct 2 conditions are met.

  • onixonix Posts: 282
    edited April 2021

    j cade said:

    then how is what you are describing anything different than how things currently work? unless I am manually turning on those jcms one by one, what you are describing is exactly how things currently work - 'aiko_smilejcm' doesnt get dialed by 'smile' unless aiko is dialed in - otherwise it sits at zero

     

    and while loading happens eveytime you load a character that is still many many many many les interactions than anytime a jcm gets dialed since what you were describing before sounded like you wanted to make those more manual that would be for me hundred to thousands of times per session if that becomes even slightly more manual any loading speed increase would be absolutely irrelevant compared to the vast increase in time i would have to spend adjusting exra dials

     

    The only difference is that now Daz just loads all files it can find in the morph directory and then spends lot of time processing them and establishing all those dependencies because it is a big mess and nobody knows what controls what

    Under my proposition, every file must contain references to all subcomponents and so if you load any file anytime you will also load all its subcomponents if necessary without any need to search for them. 

    Because there is no such requirement now. Aiko vendor does not bother to "register" all the stuff he uses, he just throws up a bunch of correction morphs into the Aiko directory and expects that Daz will pick them up and establish all dependencies automatically

     

    im just very confused by what system you are trying to explain. earlier you said

    This formula  injects extra logic that if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated

    but then

    Aiko or Girl vendor would be responsible that the Aiko head slider will turn on appropriate correction morphs. [...] but content vendors now have to care that they will activate appropriate JCMs when that is necessary.

    what is the difference between these two. functionally what difference are you seeing between "if you turn on the default Daz provided "angry" expression  AND if you turn on Dandelion slider at the same time the  Dandelion-Angry morphs will be activated"

    and

    "Aiko head slider will turn on appropriate correction morphs" as those will also only need to be on if whatever theyre correction morphs for are turned on.

     

    Both sound like systems where jcms are only turned on if the correct 2 conditions are met.

     

    yes, as you see there is no difference for the user, JCMs get turned on either way under the same conditions. 

    However, now Daz has to scan all files and build those dependencies, and under my idea vendors manually register JCMs and correction morphs telling Daz under what condition they have to be loaded 

     

    In other words, if you look at the parameter settings sub-components and controllers you will see that those are fully populated in advance and you cannot change them without reloading the figure.

    Under my proposition, those lists would be pretty much totally empty when you load base G8 character and they would only get populated on demand when you activate appropriate top morphs. This is why loading time would be near to instant (if you load from SSD).

    Activating morph for the first time would take some extra time, but not that much because only very few files have to be read and parsed in a very straightforward way. 

    This also allows easy installation of new morphs anytime just in the same way as you install new figures, you just refresh the list, new morphs appear and you can apply them.

     

    Post edited by onix on
  • onix said:

    If parsing the file would be a bottleneck then multithreading would be an obvious solution  as I understand bottleneck is the function that populates morph dependencies with appropriate formulas

    We are talking about the same thing, Parsing the data from text file and creating formula

    No what I mean is if I have a figure loaded and then I unhide a character preset or morph set, I can't use it unless I reload the figure because there is no other way to get new shape dials and pose dials to appear.

    This is not a problem because Daz should read all names to get a list of sliders and built all that property tree, It should just not go any deeper and don't loom at their dependencies until something from that tree is activated.

    The problem is Daz don't know if a slider is active until it done the work of building all the formulas associated with the formula.

    Take this example:

    {
        "output" : "neck:/data/DAZ%203D/Genesis%208/Female/Genesis8Female.dsf#neck?center_point/y",
        "operations" : [
            { "op" : "push", "url" : "Genesis8Female:#FHMVictoria8?value" },
            { "op" : "push", "val" : 0.06736755 },
            { "op" : "mult" }
        ]
    },

    This formula is applies a correction to the base genesis 8 when Victoria 8 is used.The formula is "inactive" when victoria 8 is not used, ie when Genesis8Female:#FHMVictoria8?value is 0.But there is no way for Daz to know the formula is "inactive" until it parsed and created the formula. Daz need to know all the inputs value and output value to workout if the formula is going to have an effect on the character. This is the slow part that we want to address, but it is evident that it is nessasary for Daz to fully process all the formulas when loading the character to work out how the character should actually look and behave. 

    I hope it is now clearly established that Daz need to process all morph data to get the correct character look/behaviour because there is no way for Daz to differentiate if a formula is active or not. Just loading the name of the slider is insufficient. Currently we can work around it by hiding morphs we dont need, we need a better process for it but also not too laborious. Like the entire process should only take a few click of button, not a check all the morphs you want to use.  (beside I think scanning all the files looking for name is not an negligble part of the parsing data to create formula work so will still be slow, but not as slow as doing the full work). Like I said, having a way to add load morph package we added after loading a character without having to reload the character would improve this hide morph from Daz process. In a sense it will achieve the delay load morph objective people have been asking.

    In a sense this formula is always "active" but just do nothing unless we use Victoria 8. I think this is part of your complaint, but this appears to be the standard way of doing things. Not sure what alternative there is, but I certainly dont want to manually activate each morph required for an character. Any solution need to maintain the current way of automatically apply when it is needed.

    I agree its possible to multithread, just the implementation will be non trivial. The issue to consider is: thread 1 load morph A which reference morph B, at the same time thread 2 is loading morph B. How can thread 1 figure out morph B has been loaded or not. Its solvable but care must be take to ensure the solution dont accidentally end up being single threaded. Another issue: thread 1 create morpha A with name X, thread 2 create morph B also with name Y, how does Daz ensure the names are unique because duplicate formula is an issue that need to be detected.

    If you look at the morph properties it is not that hard, after all, you can see that every property parameter contains Sub-components and controllers  list which has to be populated, (I think populating that list takes most of the time.)

    Agreed, as explained above, the problem is Daz lack a way to indicate to it this morph is not used. The only way we can do it now is to hide the file from Daz.

    I can't see any problems with multithreading because every entry in that list is independent. You load a morph and then add the appropriate formula to the appropriate list the order of how things will be added to the list does not matter.

    Except the list is stored all over the place and generating the list is the slow formula creating process. Take the above example, neck:/data/DAZ%203D/Genesis%208/Female/Genesis8Female.dsf#neck?center_point/y is in one file, Genesis8Female:#FHMVictoria8?value is in another file, the formula that use it is in another. Daz would only get all the infomation it need after it processed all the formulas in all the files.

    It is the parsing of all the files that is slow, that we want to multithread, but notice how the formulas is not independent, it require an input that can be another formula from somewhere else. This is where multithreading is wanted but difficult to implement.

  • thenoobduckythenoobducky Posts: 68
    edited April 2021

    In other words, if you look at the parameter settings sub-components and controllers you will see that those are fully populated in advance and you cannot change them without reloading the figure.

    Under my proposition, those lists would be pretty much totally empty when you load base G8 character and they would only get populated on demand when you activate appropriate top morphs. This is why loading time would be near to instant (if you load from SSD).

    Activating morph for the first time would take some extra time, but not that much because only very few files have to be read and parsed in a very straightforward way. 

    This also allows easy installation of new morphs anytime just in the same way as you install new figures, you just refresh the list, new morphs appear and you can apply them.

    Ok yeah I kinda agree this should be the approach but not fully. It need to address the following issues:

    1. It should be for advanced user only. Why? Because it will break shaping and posing pane. Being able to find all the poses and shapes applicable for the figure is very useful for a beginner. It is also the quickest way to find things when you want to modify a body part. 

    2. It doesnt work for dial only shapes or expression. For example Zev0's shape shift are a bunch of shape dials. It is not a character so there is no file to apply it. This goes to reason 3

    3. There need to be some way to select contents that should be loaded with base figure. For example, I want the shape shift dial to be populated when I load the base character.

    4. This need to be backward compatable with exisitng content. How do you propose to make it backwards compatable with existing contents? At least work with stuff bought via Daz store, custom work is less of a problem.

     

    What you are propossing can already be achieved currently by going around the daz install process and creating virtual file links, but I dont think it is the solution that Daz dev will want to adopt. 

     

     you just refresh the list, new morphs appear and you can apply them.

    This is what I want implemented first, but in a way that is faster and more convinent than reloading the character. Everything else already be worked around.

    Post edited by thenoobducky on
  • RodrijRodrij Posts: 156
    edited April 2021

    LenioTG said:

    Hi everyone! :D

    As a content creator, I need to keep many morphs installed, and they dramatically lengthen both the loading and the closing time of Daz studio, to the point they sometimes take MOST of the time I need to create content.
    So no, I won't create folders for my morphs and turn them on and off based on what I'm doing, that's not convenient for my workflow. And it's not what this thread is about.

    I've already done everything that's in my power to reduce these loading and closing times (uninstalled what I don't need, upgraded to NVMe SSD only library, upgraded my CPU/RAM), and I've actually made a guide about it a while ago, that helped me reduce the loading times by almost 80% (https://www.daz3d.com/forums/discussion/411446/why-does-it-take-so-long-to-load-figures-solved-with-guide/).
    Although, they're still pretty long.

    So, my question is: "Are you making this a priority, and trying to reduce the lengthening of loading and closing times due to G8 morphs installed?"

    Thanks!

    I've been experimenting to find options to solve this problem with an easy workflow. One solution I found is to use multiple directory sets pictured in attachment which is a feature I discovered recently.  One set has everything which you can use to create characters, the other is a trimmed down directory that only has the things needed for that scene or commonly used characters.

    To get only the files I need for the scene easily without hunting down folders, I use https://www.daz3d.com/content-gatherer.

    Eventually I'm going to segregate Figures with morphs from everything else. So I will have one directory for Figures, one for everything else and one that with only what is needed for a scene.


    If i want to load everything I will have a directory set with the figures directory and the everything else directory. If I'm animating or doing something that doesn't require every morph I load the directory set with the optimized directory  and everything else directory.

    Once you've created your characters that you use often which you only have to do once. The workflow is:

    Open content directory manager > Select the optimized directory >load the scene.


    If you wan to create a new character which you only have to do once per character:

    Open content directory manager > select the full content set > create and save character > drag and drop it in content gatherer and get all the assets to the optimized directory > Then proceed to the steps above for the previous workflow.

    Like stated before in the thread something like Fit control's Remove Unused Morph's would be nice to have but this is the next best thing I've found that you can do now.

    daz directory.png
    473 x 469 - 14K
    Post edited by Rodrij on
  • RuphussRuphuss Posts: 2,631

    would be nice if we could store/save favorites at least

  • Ruphuss said:

    would be nice if we could store/save favorites at least

    Favourite what? Can you elebrate, it might be possible to do it with script.

  • BejaymacBejaymac Posts: 1,897

    https://www.daz3d.com/forums/discussion/comment/6537556/#Comment_6537556

    ^^^ That is a link to a small PoC I did a few weeks back, one of the things this shows is that DS has a built in asset INJection system.

    Basically the devs are going to have to setup DS so it doesn't load every asset with the figure, either that or people are just going to stop buying characters/morph sets, which will hurt DAZ financially.

    Which ever way they do it I suspect there is going to be one hell of a mountain of content that needs reworked, as just stopping assets being loaded with the figure is going to break everything.

  • Bejaymac said:

    https://www.daz3d.com/forums/discussion/comment/6537556/#Comment_6537556

    ^^^ That is a link to a small PoC I did a few weeks back, one of the things this shows is that DS has a built in asset INJection system.

    Basically the devs are going to have to setup DS so it doesn't load every asset with the figure, either that or people are just going to stop buying characters/morph sets, which will hurt DAZ financially.

    Which ever way they do it I suspect there is going to be one hell of a mountain of content that needs reworked, as just stopping assets being loaded with the figure is going to break everything.

    Checked it out, nice find. I think the injection system is actually just the normal character load system, except being redirected to file outside of the /data/ folder. 

    You method also work with hiding/unhiding file method. Basically have each character install into a seperate folder. Add character folder as base directory as needed and load character. This way you dont have to manually edit any files. But getting DIM to install to seperate folder is going to be a pain.

    Do you know of a way to get it working with shape morphs. Otherwise having too many of those can also make loading super slow. That was way I was interested in a system that can handle more than just characters.

     

     

  • onixonix Posts: 282

    thenoobducky said:

    In other words, if you look at the parameter settings sub-components and controllers you will see that those are fully populated in advance and you cannot change them without reloading the figure.

    Under my proposition, those lists would be pretty much totally empty when you load base G8 character and they would only get populated on demand when you activate appropriate top morphs. This is why loading time would be near to instant (if you load from SSD).

    Activating morph for the first time would take some extra time, but not that much because only very few files have to be read and parsed in a very straightforward way. 

    This also allows easy installation of new morphs anytime just in the same way as you install new figures, you just refresh the list, new morphs appear and you can apply them.

    Ok yeah I kinda agree this should be the approach but not fully. It need to address the following issues:

    1. It should be for advanced user only. Why? Because it will break shaping and posing pane. Being able to find all the poses and shapes applicable for the figure is very useful for a beginner. It is also the quickest way to find things when you want to modify a body part. 

    2. It doesnt work for dial only shapes or expression. For example Zev0's shape shift are a bunch of shape dials. It is not a character so there is no file to apply it. This goes to reason 3

    3. There need to be some way to select contents that should be loaded with base figure. For example, I want the shape shift dial to be populated when I load the base character.

    4. This need to be backward compatable with exisitng content. How do you propose to make it backwards compatable with existing contents? At least work with stuff bought via Daz store, custom work is less of a problem.

     

    What you are propossing can already be achieved currently by going around the daz install process and creating virtual file links, but I dont think it is the solution that Daz dev will want to adopt. 

    1 it does not need to be any different experience for the user just for the content vendor

    2 it works fine with all that it would rather have problems to work with nondialable correctors like if you create a custom figure and want to add JCMs for its joints those morphs have no user-initiated sliders and under my system cannot be activated.  to work around this problem a single slider has to be added which will turn on all JCM morphs for that figure and that slider has to be set to 100% when you load the figure.

    3 shape dials will be always populated for all morphs and you can use any of them anytime that have file just their formulas will be empty until you turn them on 

    4 this is a big problem and more or less cant be done effectively. one of the solutions is to create morph relationships files automatically however they need to be regenerated every time new content is installed to rebuild all dependencies. For the same reason, it may be not possible to install new content on demand. Current daz architecture was a mistake from the beginning because nobody imagines that thing can get to this point 

    This is why all this discussion is no more than a theoretical exercise 

    The more reasonable thing to do at this time may be to implement multithreading for different figures because even if G8  takes a long time to load it is not the only thing that has to be loaded scene can contain hundreds or other props textures and morphs which still need to be loaded and it may even happen that G8 is not even main time waster.

     

    This is what I want implemented first, but in a way that is faster and more convinent than reloading the character. Everything else already be worked around.

     

    I think this can be done with a script or plugin because morphs technically can be loaded on demand. Daz developers just don't want to do that because of the fear to screw up something and because this loading may not always work as intended.  maybe there is even some kind of API available to ask Daz to load the morph file?

    I see daz can load OBJ morph  anytime from the script  http://docs.daz3d.com/doku.php/public/software/dazstudio/4/referenceguide/scripting/api_reference/samples/file_io/import_obj_morph_loader_silent/start

    But the possible problems here is how Daz will later save that kind of morph? ideally, it will include it into your scene file or it can create another copy of the existing morph file

Sign In or Register to comment.