DS and virtual memory / page file

Taoz

I've never really looked at how much DS is using virtual memory / page file, since I currently only have 8 GB RAM in my render PC I know it is swapping when I load too much into the scene, but I didn't expect it to do so as long as there's still available RAM.  But maybe it's Windows memory management that controls that, though it usually only throw data into virtual memory when they haven't been accessed for a while.  But this is what I discovered when checking memory use with AnVir Task Manager (can be downloaded for free at anvir.com). 

I've tested on two PCs which hardware wise are 100% identical, except for GPU and HDDs.  PC1 is running Windows 8.1 and has a 1 GB Asus card, PC2 is running Windows 10 and has an 8 GB GTX1070 - I don't think this matters in this context though as no rendering is taking place when testing this.

When testing, I start DS, load one character, take a screenshot of memory use, then load another character and take another screenshot.  Same characters for both tests.

The first test is done on PC1, with DS 4.12 and 14 G8F characters installed. Total load time for the two characters is about 40 seconds. 

The second test is on PC2 with DS 4.15 and 755 GF8 characters plus some G8F morph packages installed.  Total load time for both characters is 10+ minutes.

I'm really surprised about the difference in both the amount of RAM as well as virtual memory that is being used on these two setups.  On PC2 DS uses about 5 times as much memory in total when loading these two characters which must mean an incredible overhead.  Writing 8 GB page file to an HDD also takes a good while which probably is responsible for a lot of the load time.

So I wonder how it behaves on a system with a lot of RAM.  Since it apparently uses the page file even if there's a good amount of free RAM available it could be interesting too see how much it's being used on a system with lots of RAM, like 32 or 64 GB, under the same circumstances.  Have anyone tested that? 

 

 

 

TheRetiredSailor

Short answer, I have been able to perform iRay CPU renders without using the page file if the scene fits into physical memory.  Of course, if your doing a GPU iRay render then the memory utilized is on the graphics card and computer physical memory is a mute point (aside from normal system overhead).

While Windows seems to use some amount of virtual memory, no matter what, I have found that having 16GB of physical memory allows heavier scenes to render using the CPU (iRay) without shuffling a bunch of stuff out to virtual memory (page file). In my mind this allows for faster CPU renders as physical memory responds much quicker than hard drives and even SSDs.

I use Task Manager, Performance tab to monitor memory usage during renders.  The Memory tab (inside the Performance tab) shows how much physical memory is in use as well as Paged pool and other memory statistics.  More in depth monitoring can be obtained inside Resource Monitor, but Task Manager provides a quick and easy look into resource utilization.

PerttiA

I did some testing on W7/RTX 2070 Super (8GB)/64GB RAM to see how RAM and VRAM were used while rendering.

Taoz

TheRetiredSailor said:

Short answer, I have been able to perform iRay CPU renders without using the page file if the scene fits into physical memory.  Of course, if your doing a GPU iRay render then the memory utilized is on the graphics card and computer physical memory is a mute point (aside from normal system overhead).

While Windows seems to use some amount of virtual memory, no matter what, I have found that having 16GB of physical memory allows heavier scenes to render using the CPU (iRay) without shuffling a bunch of stuff out to virtual memory (page file). In my mind this allows for faster CPU renders as physical memory responds much quicker than hard drives and even SSDs.

I use Task Manager, Performance tab to monitor memory usage during renders.  The Memory tab (inside the Performance tab) shows how much physical memory is in use as well as Paged pool and other memory statistics.  More in depth monitoring can be obtained inside Resource Monitor, but Task Manager provides a quick and easy look into resource utilization.

Thanks.  Yes, I expected it would use the page file less if there is enough RAM.  What surprised me is the increase of general memory use when loading just a single character when you have a lot of characters installed, compared to when you only have a few installed. 

It is well known that having a lot of G8 characters installed will slow down character load considerably, but I hadn't noticed that memory overhead when loading characters also would increase proportionally (exponentially?) to the number of characters installed also, other things (like number of characters loaded into the scene) being equal, because on my system with only 8 GB RAM it is "secretly" using the page file to handle most of the used memory, even when there still is a good deal of free RAM available. 

So when I load say two characters and RAM use increases with say 4 GB, it's not the loaded characters that use those all those 4 GB, it's for a great part the overhead generated by having 775 characters installed.  If I uninstall these it's not just character load time that decreases considerably, but also general memory use. 

Taoz

PerttiA said:

I did some testing on W7/RTX 2070 Super (8GB)/64GB RAM to see how RAM and VRAM were used while rendering.

Thanks, yes it looks like it is using a lot of RAM here because it's available (assuming the page file is being used proportionally less).