SSD Issues
raylikesvideos
Posts: 9
Hello, currently for Daz I use an HDD disk and as you can imagine the loading times are very long (5-10 mins)
But I had an idea, if I load my entire library to an M.2 SSD disk, do you think the loading times would reduce a lot?
Comments
No, because the speed of the drive is not the reason for long loading times.
well I can confirm just having a SSD for your Cdrive definitely improves DAZ loading times
my library isn't on it but obviously the temporary stuff that clogs it up deleting between scenes etc uses it
my Win7 has one my Win10 doesn't and while I don't see much difference with other software it makes a huge difference with DAZ studio scene clearance
I don't get this sitting there for an hour on my Win7
SSD didn't improve it that much. General system improvement improved it a little (now on m.2 with a faster processor). But that's not the first order reason for the slowdown so don't blame your computer. The problem is under the hood. Some (I think rather impractical) approaches are available to speed things up, like making a separate library with just the content you intend to use but otherwise you'll have to wait for the devs at Daz to do something about it. Hopefully given the number of threads about this it's in somebody's inbox.
Given that this has been an issue going back at least to 4.11 (I still have 4.11 and 4.12 installed on my secondary computer and load times are just as bad in both), don't hold your breath waiting for Daz dev to get around to fixing it.
I've never really studied this, mostly because I've never had much issue with it (my G3-era stuff loads fairly quickly), but if you look at how much system RAM is used when loading a scene, and compare that with typical SSD (or HDD) read times, if it was relying on the SSD alone most stuff would load in a few seconds. For example, of your scene takes say 2GB of system RAM, and a typical SSD reads at a rate of 300MB/sec (it can vary substantiatlly), you'd expect it to load in a few seconds.
Which implies that, as others have said, the SSD isn't the bottleneck. And if you look at your CPU cores activity during loading, I'm guessing there's only one core that does most of the work, implying that it's a serial process. In other words, you can't do multiple things at the same time, or in parallel. Which means you just have to wait for DAZ to load it and arrange it in RAM.
So if you can't use all those nice CPU cores (I'm assuming...), then there may be nothing the developers can do other than just wait while they quickly grab the scene data off the SSD, then slowly, step by step, decode and arrange all that data in the proper way in system RAM. There is, after all, an absolute TON of stuff that has to be done to set up the scene and objects and morphs and lights and character meshes with zillions of faces and vertices and on and on...
Yeah, maybe there's some untapped optimization they can do, but as characters and textures and morphs get more and more dense I wouldn't be surprised if that's just the way it is.
It's not about version of DS since all the versions are basically handling loading the same way, I don't know at which point they introduced the cache file, but it doesn't seem to have any positive effect on loading times.
Even though the problem was there already with the G3, with G8 the number and complexity of the morphs (JCM+HD) installed with the products seemed to increase by tenfold which meant the problem became apparent with fewer products in the library.
Unless the loading process is changed to multi-threaded and/or some sort of selective loading is introduced, the only way to speed up the loading is to control (=limit) the amount of morphs/characters that are loaded onto the figure.
Here's the thing: I know as a dev that a lot of "problems" like this can be solved with hash tables/dictionaries (in other words, a relatively simple cache). It doesn't make any sense to me that this problem isn't tractable using such a simple technique and I'm definitely not going to assert the devs are dumbos. So I have to conclude whatever's preventing this being fixed is deeper. I've always suspected the UI framework as being responsible in some fashion, so I conclude that perhaps it'll be fixed when a new version of the UI framework is implemented (and they're prepping to do that, as noted in the last release notes).
Of course it's impossible to say without the source code... a dev blog about it would be nice for those of us interesting in these things.
I was thinking of installing DazStudio on a 500GB SSD (disk C), but installing the entire library on a 1TB M.2 SSD disk (disk D) to load scenes faster. But according to what I'm reading, wouldn't that be a big change? : /
I don't think this problem can be fixed at all. Daz developers seriously screwed up with the architecture itself never expecting to have thousands of morphs and cross-references to each other that they will need to load every time. Caching is not an option because it will mess up with content updates and other changes, it will still require checking all cross-references and cache files will be pretty huge.
One thing which comes to mind why everything is so slow is that many files are compressed and Daz has to unzip several gigabytes worth of content before it can process it. so maybe unzipping them would imp[rove things a little (but since those are text files they also require quite some work to process)
Daz developers definitely could implement multithreading for all that uncompressing and text file processing task to speed up things but why they just don't do it until now?
Y'know that's a good point. If the slow loading is partly a result of just trying to decompress and manage tons and tons of individual morphs and expressions and stuff, maybe some of that could be assigned separate threads. Hmmm....
But from what I've seen of DAZ Studio, I'm extremely impressed with all the development efforts. The UI is amazingly well done, the feature set is incredible (I'm still discovering stuff after many years), and overall I'd tend to assume the best from them. And if folks are complaining about 10 minute loading times, and there's ways to improve that, I wouldn't be suprised if we'll see something in the near future.
The loading, including decpmpressing, is not the main bottleneck (and remember that loading a compressed file is going to be faster than loading an uncompressed file, so depending on disc interface speed loading compressed and unpacking might actually be the faster option).
Daz is aware of the potential complexity - we have seen successive iterations of their attempts to find an efficient way of handling the issue, injection then ExP in Poser format, then the move to more-or-less the current structure with the Genesis line. It is a hard problem if they want to allow people the ability to expand the pool of shapes, rather than being limited to a set repertoire with perhaps a free slot of some kind for a custom character shape. Genesis 8 has had a longer run than previous characters, at least of the Genesis line, which has made the balance harder to maintain.
If you are using magnetic disks still, yes it would be much faster. My old laptop went from like 5 minutes to boot to 30 seconds or less.
Assuming you have a PCIe gen 3 motherboard, you'd be better off with a nvME M.2 of 2 TB and read speed in excess of 3000, eg PNY has one on Amazon US for about $250 USD for the system disk. The OS and programs use the faster read speeds better. If DAZ is the only program you use the 1TB SSD is enough and 2TB SSD for the DAZ 3D Content Library to allow for growth. You can move your entire Windows 10 Documents Library heirarchy to the 2TB SSD you have DAZ on as the biggest files on those are videos which get streamed and buffered anyway so the slower read speeds are not a problem.
With DAZ Studio while reading those gigabytes of files is helped by a faster SSD the difference in speed because the sata SSDs and the NVME SSDs is held back by the DAZ Studio's long times processing text and image files in RAM & CPU not by the the reading of the files into RAM from SSD. You want enough RAM to keep the files in RAM though once opened by a program and not use SSD as swap so 16GB, better 32GB or more.
If you have a new motherboard that supports 4th gen PCIe then buy 4th Gen PCIe cards and SSDs and such if you can afford it and really like being always cutting edge as the prices are astromonically more expensive.
Personally, my current desktop has a 2TB Sata SSD which has a slow read times of like 600 mb/s (I think) which is fine given the slowness in DS processing the files as you create scenes and render and not so much reading off the SSD.
Also note, SSDs slow down and fail faster when they exceed 70% capacity if you believe what others have said in their reviews and tests and I generally do. eg, my 2TB system & DAZ disk SSD is over 70% I am now going to buy a $250 2TB 3000+ mb/s PNY nvme M.2 SSD and move my Windows and applications to that SSD. I want to keep both about about 60% or less full.
I can get my machine to run out of memory by increasing smoothing on characters that have fitted items. Up to the point where I get the "You are out of memory" popup, the computer runs very slowly. I just ramp down the smoothing.
I have one of those nvME M.2 drives mentioned above but it is 1TB and not my system disk (my C: system drive is an older type SSD). I use the M.2 drive to store my DAZ library (at least the DIM folders - the old Poser style folders are still in a Legacy Library on my HDD). However, I did not see a marked difference when I switched my DIM content to the M.2 drive. I have to say that my load times are quite good compared to some numbers I've seen mentioned on this forum - a base G8F loads in under 10 seconds and any character not much longer.
Still, I only have around 25 G8 characters wheras other people have hundreds so I'm sure that is the main reason my load times are quite low. I tend to buy characters for the skins, not the shapes. So I have several morph packs to make it possible to dial-in the shape I want.
Maybe I'm missing something. My DAZ Studio loads almost instantly. My boot drive is an SSD. My DAZ content is on an internal 1TB hard drive and an external 3TB Hard drive. I also have 32GB of RAM.
It is not the load of DAZ Studio that is the problem, it is the time to load G8 figures/characters. The more you have, the slower they load.
That said, there is a delay for DAZ Studio (the program) shutting down. On my PC, and depending on the scene, it can take from 1 to 3 minutes (according to Task Manager) and my scenes are not really that heavy on content.
My understanding has been that there is a speed improvement if the OS is on one drive and your files that are in use are on another.
But, with that said, the real issue is the bottleneck/log jam. So for example, if you do not have enough ram and thus you need a hard drive swap file that will slow things down a lot. There are many other examples like not enough CPU cores.
My suggestion is to use one of the built in "system resource monitors" and see what is maxing out - thus, bogging you down.
To see what is maxing out and when - render vs load vs etc...
I think Mac, Windows and Ubuntu all have a system resource monitor.
This might help you make the best investment.
No doubt a faster drive will be faster in you have a drive interface that can support it - yet there might be a better investment.
Consider posting some screen captures of what the system resource monitors show.
My Experience: My DAZ Library is about 1,5 TB. Test with Load of G8F Base Female:
SSD (SATA) - 25% faster than HDD
SSD M.2 - 10% faster than SSD (SATA) - so overall apporximately 35% faster than HDD