Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
New Beta loads all my scenes again, too.
Thank you! Sorry for late reply on my side. I'm going to test it!
Update: Indeed, now it loads my scenes normally! I'm really curious, as fellow former programmer, what was the root cause? :)
Also, loading of the assets seems to be a bit faster, than before - that's nice!
I wanted to mention an issue with PBRLayered shader used in Kennedy's package product (her shoes) from April 2021 (I didn't update it yet, for this reason - testing) - the moment I press render preview and loading and Iray initialization ensues, just before starting rendering, DS crashes, and it seems this crash is being caught by Windows rather, than DS itself.
Is it only me that loading a Genesis 8 figure takes much longer with the August Public Builds ?
I just open DAZ Studio 4.15.0.28 and load a Genesis 8 Female figure into an empty scene and it takes 12 minutes.
With an older Public Build it takes 5.5 minutes (see logs below).
2021-08-28 19:18:00.990 +++++++++++++++ DAZ Studio 4.15.0.2 starting +++++++++++++++++
2021-08-28 19:18:00.990 Performing cleanup...
2021-08-28 19:18:00.990 Release Cycle: Public Build
2021-08-28 19:18:00.990 Platform bits: 64
2021-08-28 19:18:00.990 Qt Version: 4.8.7
2021-08-28 19:18:00.990 OpenSubdiv Version: 3.0.0
2021-08-28 19:18:00.990 Running on Windows 10, Build 9200, No Service Pack Installed
...
2021-08-28 19:18:55.478 *** Scene Cleared ***
2021-08-28 19:24:21.399 Begin Loading Character Addons...
2021-08-28 19:24:21.483 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:21.483 Loaded file: PHMEyelashesLengthUpper.dsf
2021-08-28 19:24:21.483 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:21.483 Loaded file: PHMEyelashesLengthLower.dsf
2021-08-28 19:24:21.483 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:21.483 Loaded file: PHMEyelashesCurl.dsf
2021-08-28 19:24:24.646 File loaded in 0 min 2.9 sec.
2021-08-28 19:24:24.647 Loaded file: Genesis 8 Female Eyelashes.duf
2021-08-28 19:24:24.688 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:24.689 Loaded file: PHMEyelashesLengthUpper.dsf
2021-08-28 19:24:24.711 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:24.717 Loaded file: PHMEyelashesLengthLower.dsf
2021-08-28 19:24:24.729 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:24.734 Loaded file: PHMEyelashesCurl.dsf
2021-08-28 19:24:24.747 Loaded Morph Deltas in 0 min 0.0 sec.
2021-08-28 19:24:24.747 Loaded file: PHMEyelashesExpand.dsf
2021-08-28 19:24:25.342 File loaded in 0 min 0.1 sec.
2021-08-28 19:24:25.342 Loaded file: Base MAT Eyelashes.duf
2021-08-28 19:24:25.344 Finished Loading Character Addons
2021-08-28 19:24:25.667 File loaded in 5 min 30.2 sec.
2021-08-28 19:24:25.680 Loaded file: Genesis 8 Basic Female.duf
2021-08-28 19:24:34.809 Loaded image G8FBaseLegsMapD_1003.jpg
2021-08-28 19:24:34.934 Loaded image G8FBaseEyes01_1007.jpg
2021-08-28 19:24:35.105 Loaded image G8FBaseArmsMapD_1004.jpg
2021-08-28 19:24:35.232 Loaded image G8FBaseMouthMapD_1005.jpg
2021-08-28 19:24:35.409 Loaded image G8FBaseFaceMapD_1001.jpg
2021-08-28 19:24:35.648 Loaded image G8FBaseTorsoMapD_1002.jpg
2021-08-28 19:24:35.761 Loaded image G8FBaseEyelashes_1006.jpg
2021-08-28 19:29:33.326 +++++++++++++++ DAZ Studio 4.15.0.28 starting +++++++++++++++++
2021-08-28 19:29:33.326 Performing cleanup...
2021-08-28 19:29:33.342 Release Cycle: Public Build
2021-08-28 19:29:33.342 Platform bits: 64
2021-08-28 19:29:33.342 Qt Version: 4.8.7
2021-08-28 19:29:33.342 OpenSubdiv Version: 3.0.0
2021-08-28 19:29:33.342 Running on Windows 10, Build 9200, No Service Pack Installed
...
2021-08-28 19:30:37.830 Prepare asset load (open): /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-08-28 19:30:37.830 Locking viewport redraw...
2021-08-28 19:30:37.830 Viewport redraw locked.
2021-08-28 19:30:38.075 Begin asset load (open): /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-08-28 19:30:38.360 *** Scene Cleared ***
2021-08-28 19:30:38.367 Determining missing assets...
2021-08-28 19:30:39.362 Setting textures...
2021-08-28 19:30:40.129 Scanning for addons...
2021-08-28 19:36:39.938 Creating node geometry...
2021-08-28 19:36:39.955 Creating UV sets...
2021-08-28 19:36:39.985 Creating materials...
2021-08-28 19:36:41.129 Resolving legacy figures...
2021-08-28 19:36:41.129 Preparing modifiers...
2021-08-28 19:36:42.660 Creating modifiers...
2021-08-28 19:37:01.692 Creating property aliases...
2021-08-28 19:37:02.359 Creating property links...
2021-08-28 19:37:02.367 Creating sum stage formulas...
2021-08-28 19:42:18.150 Creating multiply stage formulas...
2021-08-28 19:42:18.999 Processing scene data...
2021-08-28 19:42:18.999 Finalizing modifiers...
2021-08-28 19:42:19.007 Finalizing channels...
2021-08-28 19:42:19.008 Finalizing materials...
2021-08-28 19:42:19.008 Sorting property groups...
2021-08-28 19:42:19.035 Finalizing scene data...
2021-08-28 19:42:19.116 Begin Loading Character Addons...
2021-08-28 19:42:19.564 Prepare asset load (merge): /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-08-28 19:42:19.564 Locking viewport redraw...
2021-08-28 19:42:19.564 Viewport redraw locked.
2021-08-28 19:42:19.598 Loaded morph deltas: 0m 0.1s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesLengthUpper.dsf
2021-08-28 19:42:19.600 Loaded morph deltas: 0m 0.1s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesLengthLower.dsf
2021-08-28 19:42:19.601 Loaded morph deltas: 0m 0.1s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesCurl.dsf
2021-08-28 19:42:19.865 Begin asset load (merge): /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-08-28 19:42:19.898 Determining missing assets...
2021-08-28 19:42:20.033 Setting textures...
2021-08-28 19:42:20.317 Scanning for addons...
2021-08-28 19:42:21.616 Creating node geometry...
2021-08-28 19:42:21.620 Creating UV sets...
2021-08-28 19:42:21.621 Creating materials...
2021-08-28 19:42:21.653 Resolving legacy figures...
2021-08-28 19:42:21.653 Preparing modifiers...
2021-08-28 19:42:21.712 Creating modifiers...
2021-08-28 19:42:22.398 Creating property links...
2021-08-28 19:42:22.419 Creating sum stage formulas...
2021-08-28 19:42:22.428 Creating multiply stage formulas...
2021-08-28 19:42:22.435 Finalizing modifiers...
2021-08-28 19:42:22.435 Finalizing channels...
2021-08-28 19:42:22.436 Finalizing materials...
2021-08-28 19:42:22.436 Sorting property groups...
2021-08-28 19:42:22.439 Setting up follow targets...
2021-08-28 19:42:22.439 Start following: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-28 19:42:22.440 Following started: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-28 19:42:22.440 Connect base morphs: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-28 19:42:24.409 Base morphs connected: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-28 19:42:24.411 Creating morph projection map: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-28 19:42:26.345 Finished asset load (merge): 0m 6.479s - /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-08-28 19:42:26.722 Unlocking viewport redraw...
2021-08-28 19:42:26.722 Viewport redraw unlocked.
2021-08-28 19:42:26.749 Loaded morph deltas: 0m 0.25s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesLengthUpper.dsf
2021-08-28 19:42:26.771 Loaded morph deltas: 0m 0.19s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesLengthLower.dsf
2021-08-28 19:42:26.789 Loaded morph deltas: 0m 0.14s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesCurl.dsf
2021-08-28 19:42:26.811 Loaded morph deltas: 0m 0.15s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Base/PHMEyelashesExpand.dsf
2021-08-28 19:42:27.505 Prepare asset load (merge): /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-08-28 19:42:27.505 Locking viewport redraw...
2021-08-28 19:42:27.506 Viewport redraw locked.
2021-08-28 19:42:27.558 Begin asset load (merge): /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-08-28 19:42:27.619 Determining missing assets...
2021-08-28 19:42:27.898 Setting textures...
2021-08-28 19:42:27.899 Creating materials...
2021-08-28 19:42:27.921 Applying animations...
2021-08-28 19:42:28.157 Setting up follow targets...
2021-08-28 19:42:28.157 Finished asset load (merge): 0m 0.598s - /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-08-28 19:42:28.180 Unlocking viewport redraw...
2021-08-28 19:42:28.180 Viewport redraw unlocked.
2021-08-28 19:42:28.183 Finished Loading Character Addons
2021-08-28 19:42:33.930 Finished asset load (open): 11m 55.855s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-08-28 19:44:58.077 Unlocking viewport redraw...
2021-08-28 19:44:58.077 Viewport redraw unlocked.
2021-08-28 19:44:58.738 Loaded image: G8FBaseLegsMapD_1003.jpg
2021-08-28 19:44:58.859 Loaded image: G8FBaseEyes01_1007.jpg
2021-08-28 19:44:59.031 Loaded image: G8FBaseArmsMapD_1004.jpg
2021-08-28 19:44:59.147 Loaded image: G8FBaseMouthMapD_1005.jpg
2021-08-28 19:44:59.321 Loaded image: G8FBaseFaceMapD_1001.jpg
2021-08-28 19:44:59.560 Loaded image: G8FBaseTorsoMapD_1002.jpg
2021-08-28 19:44:59.680 Loaded image: G8FBaseEyelashes_1006.jpg
https://www.daz3d.com/forums/discussion/comment/6987621/#Comment_6987621 has some information on what was done.
Thank you very much!
Could you try this again, this could be taking so long becase it is rebuilding caches. Also if you could check your mapped directories in the beta and make sure they match the ones in the general release it would also help eliminate some potential problems.
Directory mapping is exactly the same and caches are up-to-date.
Test:
Start DAZ Studio 4.15.0.28 Public Build and load G8F into empty scene.
2021-08-31 01:17:35.328 Finished asset load (open): 12m 18.333s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf
Loading a second G8F.
2021-08-31 01:38:02.883 Finished asset load (merge): 12m 19.888s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf
Exit DS and wait until background job finishes (which takes also some minutes).
2021-08-31 01:41:08.827 Clearing Undo Stack...
2021-08-31 01:41:08.954 Deleting Scene...
2021-08-31 01:41:53.417 Stop following: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 01:41:53.417 Following stopped: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 01:45:31.884 Stop following: Genesis 8 Female (2) << Genesis 8 Female Eyelashes (2)
2021-08-31 01:45:31.884 Following stopped: Genesis 8 Female (2) << Genesis 8 Female Eyelashes (2)
2021-08-31 01:48:21.189 *** Scene Cleared ***
2021-08-31 01:48:21.198 Deleting Undo Stack...
2021-08-31 01:48:21.198 Deleting Authentication Manager...
2021-08-31 01:48:21.198 Deleting Main Window...
2021-08-31 01:48:30.650 Deleting Viewport Manager...
2021-08-31 01:48:30.850 Deleting Action Manager...
2021-08-31 01:48:30.873 Hexagon Bridge : End logging.
2021-08-31 01:48:30.935 Deleting Pane Manager...
2021-08-31 01:48:31.843 Performing cleanup...
2021-08-31 01:48:31.845 Deleting Plugin Manager...
2021-08-31 01:48:31.845 Deleting Application Settings Manager...
2021-08-31 01:48:37.006 --------------- DAZ Studio 4.15.0.28 exited ------------------
2021-08-31 01:48:37.006 ~
Start DS again and load G8F into empty scene.
2021-08-31 02:07:03.940 Finished asset load (open): 12m 7.723s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf
Remark: I have a test scene with one G8F and one G8M and simple environment to check for G8F/G8M warnings.
To load this scene the first time with the August Public Build took about 45 Minutes.
With previous Public Builds it took < 10 minutes.
Edit:
Just checked with my test scene and it still needs 36 minutes to load :-(
2021-08-31 02:49:36.079 Begin asset load (open): /Scenes/G8FM.base.duf
2021-08-31 02:49:36.999 *** Scene Cleared ***
2021-08-31 02:49:37.011 Determining missing assets...
2021-08-31 02:49:40.081 Setting textures...
2021-08-31 02:49:42.259 Scanning for addons...
2021-08-31 03:14:52.195 Creating node geometry...
2021-08-31 03:14:52.245 Creating UV sets...
2021-08-31 03:14:52.299 Creating materials...
2021-08-31 03:14:54.474 Resolving legacy figures...
2021-08-31 03:14:54.474 Preparing modifiers...
2021-08-31 03:14:58.153 Creating modifiers...
2021-08-31 03:15:36.076 Creating property aliases...
2021-08-31 03:15:37.690 Creating property links...
2021-08-31 03:15:37.707 Creating sum stage formulas...
2021-08-31 03:25:57.658 Creating multiply stage formulas...
2021-08-31 03:25:59.635 Processing scene data...
2021-08-31 03:25:59.652 WARNING: QColor::setRgb: RGB parameters out of range
2021-08-31 03:25:59.657 Finalizing modifiers...
2021-08-31 03:25:59.673 Finalizing channels...
2021-08-31 03:25:59.675 Finalizing materials...
2021-08-31 03:25:59.675 Sorting property groups...
2021-08-31 03:25:59.741 Setting up follow targets...
2021-08-31 03:25:59.741 Start following: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 03:25:59.742 Following started: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 03:25:59.742 Connect base morphs: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 03:26:02.133 Base morphs connected: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 03:26:02.143 Creating morph projection map: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-08-31 03:26:04.384 Start following: Genesis 8 Male << Genesis 8 Male Eyelashes
2021-08-31 03:26:04.385 Following started: Genesis 8 Male << Genesis 8 Male Eyelashes
2021-08-31 03:26:04.386 Connect base morphs: Genesis 8 Male << Genesis 8 Male Eyelashes
2021-08-31 03:26:04.981 Base morphs connected: Genesis 8 Male << Genesis 8 Male Eyelashes
2021-08-31 03:26:04.991 Creating morph projection map: Genesis 8 Male << Genesis 8 Male Eyelashes
2021-08-31 03:26:05.380 Setting current camera...
2021-08-31 03:26:05.381 Finalizing scene data...
2021-08-31 03:26:15.461 Finished asset load (open): 36m 39.380s - /Scenes/G8FM.base.duf
2021-08-31 03:30:45.103 Unlocking viewport redraw...
2021-08-31 03:30:45.103 Viewport redraw unlocked.
@DAZ_Rawb: I tested G8F load time with a subset of the methodology I've described in other threads. There's no significant difference between 14.15.0 versions .2, .14 and .28 without or without the cache. The cache gave me a 10% speed-up and I was able to demonstrate that actual load times with the cache are quite dependent on the cache disk access - I could demonstrate a 5% or so speed difference by swapping the cache for the Beta with the one for the General Release! (I.e. without changing anything in either configuration I justed mv'ed the 'data' directories around, using OpenSUSE to be sure no read/write of the contents happened.)
The raw methodology is to execute the following script from the IDE:
The IDE gives me the time to execute, so that includes the load for the draw of the figure (not included, I think, in @CrimsonMagic's figures from the log file.) The method is somewhat subject to variation with system load but apart from one outlyer figure which I could not reproduce my timings were in the range 99.164s to 117.349s. The time with the DSON cache deleted was (apart from the outlyer) between 115.129s and 117.349s.
With the cache times varied from 99.164s to 110.300s with 4.15.0.2 giving a with-cache time range of 99.164s-107.233s and 4.15.0.28 giving times of 103.825s-110.300s. The 110s figure came after I had left the machine idle for a while - I suspect the cache files had been swapped out of memory.
Details: I selected my "Everything" CMS, which I used before while testing and contains all my content. I selected this under DIM, just reselecting my normal CMS to install .28, so the figures reflect a single unmodified CMS setup. I also ran Postgre from DIM, so even though I started and exited DAZStudio multiple times the database was the same when I swapped between release and beta. (I had to restart if after the .28 install so I repeated the general tests after installing .28).
I copied *.dsx from "%AppData%/DAZ 3D/Studio4 Public Build" to "%AppData%/DAZ 3D/Studio4" to ensure the CMS directory sets were identical along with things in the UI which might, conceivably, cause speed changes.
I alternatived general/beta first with the relevant cache directory deleted (and the trash emptied) then did the same swap after it was populated.
Detailed figures:
@jbowler: Seems that you are testing with not much G8F figures installed. I have a massive amount of G8F's and that maybe makes the difference.
How many G8F's do you have? Certainly the conclusion from the discussions and measurements in March, when I measured 4.15.0.14 is that the characters are the biggest source of the problem.
Pretty much it is the number of morphs that can be applied to the base character. The way it works a new figure tends to contribute a small set of shape morphs and I stopped buying new figures for that reason but other morphs, in particular anatomical morphs and expressions make a difference. It's a known issue, extensively discussed, but that issue is the same in 4.15.0.2 as in the betas. There were some logging changes in the betas and I think there might have been a slight speed up of caching (I can't remember for sure) but no solution has been provided as yet. The close time problem is separate and has two easy solutions; either delete the scene before you hit the close button or just kill DAZStudio. .28 has an interesting fix - it sometimes crashes inside Qt on close ;-)
The main thread on the G8F load topic is:
https://www.daz3d.com/forums/discussion/477596/how-long-does-it-take-you-to-load-g8f#latest
But there is also a lot of discussion in the general release thread. My detailed timings showing the breakdown by package types are in that thread:
https://www.daz3d.com/forums/discussion/comment/6541996/#Comment_6541996
My current figure for 4.15.0.2 is consistent with the one I got then - that was 92s for "everything" with the cache and I've purchased more stuff since then. My total package count is just over 5000, my tests were done with at least 350 G8F figures in the CMS. My G8F figure count breaks down thus:
G8Figures-Core: 60 (figures I always have accessible.)
G8Figures-New: 29 (figures I bought recently buthaven't decided if they are core or not.)
G8Figures-Unused: 260+ (figures I eliminated to save load time because I hadn't used them in scenes.)
Plus a few figures from Renderosity
Plus an unknown number extra figures which appear in sub-directories of the "Characters" directory rather than the "Characters" directory itself.
I just ran the script in my normal work CMS directory configuration and the load time is 33.678s; slow but workable.
EDIT: That 33.678s number is wrong, it's 8.148 in .2 and 8.602s in .28; I got the 33.678s number because I changed from my "everything" CMS directory set to the default before doing the timing. So my reduced configuration is giving me a better than 10x speed-up for character load.
Can't tell how many G8F's are "active" (hundreds I guess).
I've moved the "...\data\DAZ 3D" folder to SSD and removed a lot of characters (morphs) from it to get acceptable G8 load times.
File sizes may indicate the amount of data:
"...\data\DAZ 3D\Genesis 8\Female\Morphs" 21.7 GB 33606 Files 1517 Folders
"...\AppData\Roaming\DAZ 3D\dson\cache\data\DAZ 3D\Genesis 8\Female.cache.dsf" 677 MB
I tried again loading my test scene (1 G8F + 1 G8M + simple environment) with DS 4.15.0.2 Public Build (that's the last Public Build I had a backup)
2021-08-31 22:56:17.148 File loaded in 7 min 25.9 sec.
2021-08-31 22:56:17.178 Loaded file: G8FM.base.duf
As stated in previous post, with DS 4.15.0.28 Public Build it takes
Finished asset load (open): 36m 39.380s - /Scenes/G8FM.base.duf
So on my side the August Public Builds are much slower loading G8 figures.
As can be seen in the logs most of the time is used for
2021-08-28 19:30:40.129 Scanning for addons...
2021-08-28 19:36:39.938 Creating node geometry...
2021-08-28 19:37:02.367 Creating sum stage formulas...
2021-08-28 19:42:18.150 Creating multiply stage formulas...
To me it seems that August Public Builds introduced something that makes processing of large amounts of .dsf files very slow.
I'm at about 60% of your morph count; my "/Genesis 8/Female/Morphs" files are split across multiple CMS directories but I got this (ooh, how I love OpenSUSE ;-)
20772
I suspect this is the most useful measure because it represents the actual stuff DAZ Studio resolves while loading G8F base. Indeed in my 4.15.0.2 DSON cache, which I haven't used since doing those timings, I only have 2 cache files! G8F herself and her eyelashes:
121878664915716489 100 -rwxrwxrwx 1 jbowler users 101108 Aug 31 02:32 ./data/Daz\ 3D/Genesis\ 8/Female\ Eyelashes.cache.dsf
332140472518681884 68244 -rwxrwxrwx 1 jbowler users 69879844 Aug 31 02:32 ./data/Daz\ 3D/Genesis\ 8/Female.cache.dsf
jbowler@Jule:/mnt/c/DAZTemp/release> du -s -BM ./data/Daz\ 3D/Genesis\ 8/Female\ Eyelashes.cache.dsf ./data/Daz\ 3D/Genesis\ 8/Female.cache.dsf
1M ./data/Daz 3D/Genesis 8/Female Eyelashes.cache.dsf
67M ./data/Daz 3D/Genesis 8/Female.cache.dsf
(Times are in UCT; I never bothered to set up a timezone, output is as in "ls -dils"; inode/block-size/permissions/ref-count/user/group/byte-size/date/file-name.) That's with my "Default" CMS set, where the G8F load time is 10x better than my "Everything" CMS set. With my "Everything" set:
1M ./data/Daz 3D/Genesis 8/Female Eyelashes.cache.dsf
357M ./data/Daz 3D/Genesis 8/Female.cache.dsf
So that is 5x my "default" size yet it results in a 10x speed cost for me. More points on the graph would clarify what is happening yet I suspect it is a simple power law, let's hope it isn't exponential. Our timing figures are not comparable because we have different systems with different speed limits.
There was a very real problem before. I can't say that the post .14 releases haven't made it worse but I'm certain they haven't made it better.
using 4.15.0.29 (not 4.15.0.28)
time taken to load "Genesis 8.1 Female Dev Load"
4.15.0.29 (public beta) 46.526s
4.15.0.2 (normal version) 36.1s
my size for ...\data\DAZ 3D\Genesis 8\Female\Morphs is 5.20GB and my library is mostly installed on a spindle drive (I ran out of space on my SSD)
What is the size of your DSON G8F cache file, i.e. "%AppData%\DAZ 3D\dson\cache\data\DAZ 3D\Genesis 8\Female.cache.dsf" or the equivalent (if you've moved it elsewhere)?
171MB, on SSD
My guess is that it's meant for the 3dl render engine...
EDIT: Again I forgot that the Daz forums don't place you at the beginning or end of the thread when you click on the thread title, but in the bloody middle, as your post was most likely already answered...
It starts you from where you last left off
About half my size. Your load time is faster but we know from previous discussions that the G8F load is CPU limited; so long as the content is on a local drive the actual disk read time didn't seem to be a factor. (Writing the DSON cache, which happens when G8F assets are purchased, may affect the load time.) I can slow DAZ Studio down by heavy disk access from other programs but that would be expected too. So CPU speed or CPU load is the other critical part of the load time equation.
I'll see what .29 produces; the changes were in 4.15.0.25, everything since has been urgent but apparently unrelated bug fixes and logging:
4.15.0.28:
Added logging to various points in the asset loading process
4.15.0.29:
Extended asset load logging; DSON cache update required/started/finished, per unique object
Made various updates to asset load logging; accuracy, consistency
One of the things that I have seen before when attempting to debug speed problems like this is that small code tweaks can expose large speed changes on some machines as a result of hitting the limits of one or other of the highest level CPU caches. Maybe that is why different people see different results with the .25+ releases.
A thought comes to mind... Have they increased the logging (in the betas) to track down what kinds of assets have the biggest impact on loading times, increased loading time being just an unavoidable side effect.
It's possible. These days Qt provides extensive logging support but I don't think that was in Qt4.8 so maybe DAZ are using QDebug. That said, logging at the assert level seems unlikely to perturb the steps taken when handling an individual asset; my script only produces 271 lines in the log file with .28:
* * *
Line 271: 2021-09-01 09:55:44.509 DEBUG: Script executed in 124 secs 977 msecs.
A lot of those are WARNING lines which are produced anyway:
2021-09-01 09:55:40.728 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(5957): Could not find parent for modifier: PBMDDStilettoNails.
2021-09-01 09:55:40.737 WARNING: ..\..\..\..\..\src\sdksource\fileinput\dzassetdaz.cpp(1727): Failed to create modifier: PBMDDStilettoNails!
If I delete those lines only 81 lines remain, I've posted these below with the big jumps in bold:
2021-09-01 09:53:39.532 DEBUG: Executing Script...
2021-09-01 09:53:39.534 Prepare asset load (open): /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-09-01 09:53:39.534 Locking viewport redraw...
2021-09-01 09:53:39.534 Viewport redraw locked.
2021-09-01 09:53:39.588 Begin asset load (open): /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-09-01 09:53:39.624 *** Scene Cleared ***
2021-09-01 09:53:39.624 Determining missing assets...
2021-09-01 09:53:39.701 Setting textures...
2021-09-01 09:53:39.830 Scanning for addons...
2021-09-01 09:53:56.261 JSON interpreter error: Too many values for color.
2021-09-01 09:53:56.372 G:/DAZ 3D/Studio/Genesis 8/G8Figures-Unused/data/Daz 3D/Genesis 8/Female/Morphs/Hinkypunk/Kinsley/KinsleyBody.dsf (24,0): Illegal new-line in string
2021-09-01 09:54:21.331 Creating node geometry...
2021-09-01 09:54:21.337 Creating UV sets...
2021-09-01 09:54:21.339 Creating materials...
2021-09-01 09:54:21.426 Resolving legacy figures...
2021-09-01 09:54:21.426 Preparing modifiers...
2021-09-01 09:54:21.685 Creating modifiers...
2021-09-01 09:54:24.043 Creating property aliases...
2021-09-01 09:54:24.072 Creating property links...
2021-09-01 09:54:24.073 Creating sum stage formulas...
2021-09-01 09:55:39.374 Creating multiply stage formulas...
2021-09-01 09:55:39.686 Processing scene data...
2021-09-01 09:55:39.686 Finalizing modifiers...
2021-09-01 09:55:39.688 Finalizing channels...
2021-09-01 09:55:39.688 Finalizing materials...
2021-09-01 09:55:39.689 Sorting property groups...
2021-09-01 09:55:39.697 Finalizing scene data...
2021-09-01 09:55:39.772 Begin Loading Character Addons...
2021-09-01 09:55:39.793 Prepare asset load (merge): /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-09-01 09:55:39.793 Locking viewport redraw...
2021-09-01 09:55:39.793 Viewport redraw locked.
2021-09-01 09:55:39.802 Begin asset load (merge): /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-09-01 09:55:39.804 Determining missing assets...
2021-09-01 09:55:39.815 Setting textures...
2021-09-01 09:55:39.875 Scanning for addons...
2021-09-01 09:55:40.671 Creating node geometry...
2021-09-01 09:55:40.672 Creating UV sets...
2021-09-01 09:55:40.672 Creating materials...
2021-09-01 09:55:40.675 Resolving legacy figures...
2021-09-01 09:55:40.675 Preparing modifiers...
2021-09-01 09:55:40.731 Creating modifiers...
2021-09-01 09:55:40.748 Creating sum stage formulas...
2021-09-01 09:55:40.758 Creating multiply stage formulas...
2021-09-01 09:55:40.758 Finalizing modifiers...
2021-09-01 09:55:40.758 Finalizing channels...
2021-09-01 09:55:40.758 Finalizing materials...
2021-09-01 09:55:40.758 Sorting property groups...
2021-09-01 09:55:40.759 Setting up follow targets...
2021-09-01 09:55:40.759 Start following: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-09-01 09:55:40.761 Following started: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-09-01 09:55:40.761 Connect base morphs: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-09-01 09:55:40.919 Base morphs connected: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-09-01 09:55:40.925 Creating morph projection map: Genesis 8 Female << Genesis 8 Female Eyelashes
2021-09-01 09:55:41.006 Finished asset load (merge): 0m 1.205s - /data/DAZ 3D/Genesis 8/Female Eyelashes/Tools/Script Loads/Genesis 8 Female Eyelashes.duf
2021-09-01 09:55:41.027 Unlocking viewport redraw...
2021-09-01 09:55:41.027 Viewport redraw unlocked.
2021-09-01 09:55:41.029 Loaded morph deltas: 0m 0.0s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesLengthUpper.dsf
2021-09-01 09:55:41.030 Loaded morph deltas: 0m 0.0s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesLengthLower.dsf
2021-09-01 09:55:41.035 Loaded morph deltas: 0m 0.0s - /data/DAZ 3D/Genesis 8/Female/Morphs/DAZ 3D/Head/PHMEyelashesCurl.dsf
2021-09-01 09:55:41.092 Loaded morph deltas: 0m 0.0s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/D.Master/Dolly/PHMEyelashesExpand.dsf
2021-09-01 09:55:41.095 Loaded morph deltas: 0m 0.1s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesLengthUpper.dsf
2021-09-01 09:55:41.096 Loaded morph deltas: 0m 0.0s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesLengthLower.dsf
2021-09-01 09:55:41.098 Loaded morph deltas: 0m 0.1s - /data/Daz 3D/Genesis 8/Female Eyelashes/Morphs/DAZ 3D/Head/PHMEyelashesCurl.dsf
2021-09-01 09:55:41.267 Prepare asset load (merge): /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-09-01 09:55:41.267 Locking viewport redraw...
2021-09-01 09:55:41.267 Viewport redraw locked.
2021-09-01 09:55:41.275 Begin asset load (merge): /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-09-01 09:55:41.281 Determining missing assets...
2021-09-01 09:55:41.292 Setting textures...
2021-09-01 09:55:41.292 Creating materials...
2021-09-01 09:55:41.294 Applying animations...
2021-09-01 09:55:41.305 Setting up follow targets...
2021-09-01 09:55:41.305 Finished asset load (merge): 0m 0.30s - /People/Genesis 8 Female/Materials/Base Materials/Iray/Base MAT Eyelashes.duf
2021-09-01 09:55:41.306 Unlocking viewport redraw...
2021-09-01 09:55:41.306 Viewport redraw unlocked.
2021-09-01 09:55:41.308 Finished Loading Character Addons
2021-09-01 09:55:41.458 Finished asset load (open): 2m 1.870s - /People/Genesis 8 Female/Genesis 8 Basic Female.duf
2021-09-01 09:55:44.508 Unlocking viewport redraw...
2021-09-01 09:55:44.508 Viewport redraw unlocked.
2021-09-01 09:55:44.509 DEBUG: Result:
2021-09-01 09:55:44.509 DEBUG: Script executed in 124 secs 977 msecs.
Extracting the big gaps:
16.791s doing that..., then two errors:
2021-09-01 09:53:56.261 JSON interpreter error: Too many values for color.
2021-09-01 09:53:56.372 G:/DAZ 3D/Studio/Genesis 8/G8Figures-Unused/data/Daz 3D/Genesis 8/Female/Morphs/Hinkypunk/Kinsley/KinsleyBody.dsf (24,0): Illegal new-line in string
24.959s doing that...
2021-09-01 09:54:21.331 Creating node geometry...
75.301s doing that...
2021-09-01 09:55:39.374 Creating multiply stage formulas...
So of the total 124.977s to execute the script:
Of course I can't get the corresponding times for 4.15.0.2, but if would be interesting if other contributors could do the same analsys on their log files for .28 to see if those logs show different breakdowns.
@CrimsonMagic's numbers show the same pattern, this is with a populated DSON cache, my numbers above are without one:
Those figures have a third large peak on "Creating Modifiers" but it is small compared to the two other peaks. The bottom row of the spreadsheet is just the total time, as a check.
Using 4.15.0.29, on MacOS 10.15.7 (19H1323)
I see this error (I looked and the shaders directory is missing):
2021-09-01 15:26:43.654 WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(480): Could not add path: "/Users/<redacted>/Library/Application Support/DAZ 3D/Studio4 Public Build/shaders/iray". Due to unknown error -2
I am unable to run dForce simulation on GPU or CPU
2021-09-01 15:36:20.517 Unlocking viewport redraw...
2021-09-01 15:36:20.517 Viewport redraw unlocked.
2021-09-01 15:36:20.739 Loaded image: 3duTileable_GreenFlowers_DF.jpg
2021-09-01 15:36:36.993 WARNING: /src/dzdynamicsengine.cpp(426): Using device: ATI Radeon Pro W5700X Compute Engine
2021-09-01 15:36:37.235 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: Build Program driver returned (519)
2021-09-01 15:36:37.235 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0x1021e00 (ATI Radeon Pro W5700X Compute Engine) (err:-2)
2021-09-01 15:36:37.235 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: Build Program driver returned (519)
2021-09-01 15:36:37.235 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0x1021e00 (ATI Radeon Pro W5700X Compute Engine) (err:-2)
2021-09-01 15:36:37.235 WARNING: /src/dzopenclkernelfactory.cpp(499): ERROR: clBuildProgram (-11)
2021-09-01 15:36:37.235 WARNING: Source built for simpleAdd. Build log:
2021-09-01 15:36:37.237 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: Build Program driver returned (519)
2021-09-01 15:36:37.237 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0x1021e00 (ATI Radeon Pro W5700X Compute Engine) (err:-2)
2021-09-01 15:36:37.237 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: Build Program driver returned (519)
2021-09-01 15:36:37.237 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0x1021e00 (ATI Radeon Pro W5700X Compute Engine) (err:-2)
2021-09-01 15:36:37.237 WARNING: /src/dzopenclkernelfactory.cpp(499): ERROR: clBuildProgram (-11)
2021-09-01 15:36:37.237 WARNING: Source built for setPast. Build log:
and
2021-09-01 15:36:37.288 WARNING: /src/dzopenclkernelfactory.cpp(351): m_isValid = false @ dzopenclkernelfactory.h, line 118 (after loading kernel members)
2021-09-01 15:36:37.585 WARNING: /src/dzdynamicsengine.cpp(3625): Error initializing OpenCL!
2021-09-01 15:36:37.586 Total Simulation Time: 2.51 seconds
2021-09-01 15:36:47.243 WARNING: /src/dzdynamicsengine.cpp(426): Using device: Intel(R) Xeon(R) W-3265M CPU @ 2.70GHz
2021-09-01 15:36:47.520 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: build program driver returned (-1)
2021-09-01 15:36:47.520 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0xffffffff (Intel(R) Xeon(R) W-3265M CPU @ 2.70GHz) (err:-1)
2021-09-01 15:36:47.520 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: [CL_DEVICE_NOT_AVAILABLE] : OpenCL Error : Error: build program driver returned (-1)
2021-09-01 15:36:47.520 WARNING: /src/dzopenclkernelfactory.cpp(32): Open CL notify: OpenCL Warning : clBuildProgram failed: could not build program for 0xffffffff (Intel(R) Xeon(R) W-3265M CPU @ 2.70GHz) (err:-1)
2021-09-01 15:36:47.520 WARNING: /src/dzopenclkernelfactory.cpp(499): ERROR: clBuildProgram (-11)
2021-09-01 15:36:47.520 WARNING: Source built for simpleAdd. Build log:
The same error happens on Windows, it doesn't stop anything working:
2021-09-01 14:21:34.277 Iray [INFO] - API:DATABASE :: 0.0 API db info : Loaded "C:\Program Files\DAZ 3D\DAZStudio4 Public Build\libs\iray\libneuray.dll"
2021-09-01 14:21:34.277 Iray [INFO] - API:MISC :: 0.0 API misc info : Iray RTX 2020.1.6, build 334300.9558, 27 Mar 2021, nt-x86-64
2021-09-01 14:21:34.277 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(480): Could not add path: "C:/Users/jbowl/AppData/Roaming/DAZ 3D/Studio4 Public Build/shaders/iray". Due to unknown error -2
2021-09-01 14:21:34.277 WARNING: ..\..\..\..\..\src\pluginsource\DzIrayRender\dzneuraymgr.cpp(480): Could not add path: "G:/Production/Daz3D/beta-temp/shaders/iray". Due to unknown error -2
If you want the error to go away just create the directory yourself; I suspect the error is something like "path not found". The weird thing is that I've created the directories in the past but somehow they are getting deleted, maybe installing a new version of DAZ Studio does it.
I double checked; launched .29, inserted Keicy, dForced, Iray previewed and Iray rendered. Everything worked and the error message is still in the log file. This is Windows of course with a different GPU and CPU. The problem you have is due to the lack of OpenCL (used for dForce IRC and, I suspect, the rendering).
Thank you, at least on MacOS the whole directory structure is removed/replaced
Thanks, OpenCL is present / available but I suspect a deprecated library is invoved. It works on a Mac that I have that is on same OS version but older hardware.
I suggest a DAZ Productions "Request" (click on "Help" above and eventually you will find the page). I had assumed that this was old hardware (it's years since I had a running ATX Radeon), but I just googled it and I suspect Daz will want to support it ;-)
Timing numbers from the log files for 4.15.0.30, the left column is the log message, the columns to the right are the percentage of the total time:
What does the + mean in your 2nd and 4th column titles?
Yeah... It happens because Excel won't allow me to make a table with the two columns with the same name and it also won't allow me to make a table with two rows as the header. So it insisted on "General2" and "Public Build3" (I don't know why '3', or, for that matter '2').
The first of each pair of columns is that release (final or beta) with the DSON cache cleared - Absent - the second column has the DSON cache (generated by the first set of timing figures) Present, hence the somewhat lame '+'; the previous column with the DSON cache 'added' (populated). At least I can understand Excel most of the time, unlike PhotoShop.
Incidentally; I'm happy posting the full set of numbers as a .xlsx (i.e. XML; it's fairly simple so the Open stuff should read it too, it does have conditional formatting but I think that is ignored), but there is a lot of data in there and not much information. I tried to extract it in a meaningful form. The absolute times for the act of "Creating sum stage formulas..." are in the range 60-70s, but there is no obvious correlation with the cache so that's probably just me do stuff while the test was running.
jbowler
Thanks for you work in this, I hope Daz sees it and can turn the battleship to help with load times and remove the penalty for those of us who buy lots of characters/morphs (I've stopped buying due to the load time issues).
I really have my fingers crossed for Daz5, in hopes that they address this issue.
hello community
I'm trying to find the notes or items that have changed in this version, but I'm not getting it, my English doesn't help much, can someone link to me?