Memory usage by IRAY

cheznous2029_28ab1adedc

I have a pretty good machine now, I am using Windows 7 with an I5 processor, 32 GB Ram, and I have a EVGA GTX970 video card with 4GB of VRAM as well as an older GT620 video card with 2GB of VRAM.  So that I can track what is happening, I downloaded a nice program from Microsoft call Process Explorer that give a lot of good information about live memory usage, both on board RAM and Video RAM.  I have observed that when I load a scene and Process Explorer shows that DAZ studio is using 3GB of RAM things change a lot when I do the Ctrl R and start the render.  One reason DAZ stays so long at 0 iterations is that before it starts rendering the memory usage by DAZ Studio climbs from 3GB to 9-10 GB!  Once the render starts the GRAM used goes from about 1.5GB of VRAM to about 5GB!

What is happening here?  Why is there a more than 3 fold increase in system RAM usage when you start a render?  Why does the GRAM usage only increase once the render actually starts?

Richard Haseltine

DS is preparing data for Iray, then it passes that to the GPU and - assuming it fits - the render starts. I suspect you are seeing such high vlaues due to a lot of models being sub divided, possibly to quite a high level - have you changed the defaults?

mjc1016

Also, there are texture conversions going on...image files being compressed, resized, etc...and for each one being done there are two copies of it...the original and 'prepared' one.  You don't see that in 3Delight because tdlmake runs in the background...and then caches them in the temp folder on disk, Iray keeps them in RAM before passing them to the video card.

Iray uses about 3 bytes per pixel...per image....

AndyS

I too observed that iRay is very memory consuming.
Even for such a simple scene it exceeds 8GB.

And referring to Richard's statement "assuming it fits", which confirms the info out of other discussion round here, if it needs more memory your GPU-cards can support, iRay falls back to the very very slow CPU-only mode.

Richard Haseltine

I suspect your sand plane is heavily sub-divided to give the ridges.

AndyS

Hi Richard,

 

iRay claims to be photorealistic. But the other part of realism are the 3-dimensional surfaces. For 3Delight: no problem.
But iRay ?

One solution suggested: The creator of the prop has to construct these surfacial structures. But the result is a high amount of polygons.
Alternatively: Set a suitable value for "SubD Displacement Level".
Both results in the same gigantic size of needed memory for iRay renders.

You would like to call a value of 5 as "heavily" ?

Realism's got its price (since iRay).

Please don't get me wrong: I like the outcome of the light. And an adequate mode for 3Delight needs longer render times.

What we really need are GPU cards with way more memory - or better: an DAZ-Iray render cloud for the community.

Richard Haseltine

SubD of five increases the polygon count by 4^5, 1,024, times. Yes, this is a serious drawback to using Iray - the expectation seems to be that normal maps will be used (bumps in Iray have their own quirks). However this is a pretty typical rendering issue, not just in Iray - LW used to work this way, Carrara too, I believe several other amajor applications do not have sub-polygon displacement. One option it to bump the mesh up just enough to give coarse relief and then use normals to smooth it, another is to make sure that areas that are to be displaced start with a higher resolution so that applying SubD does not push other areas - that are already higher resolution - to to ridiculous levels, but of course a lot of items are made for 3Delight where this simply wasn't an issue and so the products weren't modelled and textured that way.

cheznous2029_28ab1adedc

The newest release version of DAZ studio.. 4.9.1.30 has some memory management problems also.  The first one I've noticed is that if you render a scene, then make a small change the memory usage increases each time you do this.  Second, now and then after do several successful renders, IRAY will just quit after doing just a few iterations.

Richard Haseltine

Are you closing the render windows as you go? if not they will sit there with their scene data and consume memory.

mjc1016

Richard Haseltine said:

Are you closing the render windows as you go? if not they will sit there with their scene data and consume memory.

And that is not a flaw...it is to allow the render to be resumed.

AndyS

Sorry Richard,

Richard Haseltine said:

SubD of five increases the polygon count by 4^5, 1,024, times. Yes, this is a serious drawback to using Iray - the expectation seems to be that normal maps will be used (bumps in Iray have their own quirks). However this is a pretty typical rendering issue, not just in Iray - LW used to work this way, Carrara too, I believe several other amajor applications do not have sub-polygon displacement. One option it to bump the mesh up just enough to give coarse relief and then use normals to smooth it, another is to make sure that areas that are to be displaced start with a higher resolution so that applying SubD does not push other areas - that are already higher resolution - to to ridiculous levels, but of course a lot of items are made for 3Delight where this simply wasn't an issue and so the products weren't modelled and textured that way.

but normal maps are not producing a "real displacement" effect. This is something totally different.

In the attached file you see the difference at the border between the red and blue area.
The left part shows real displacement - on the right is the outcome of bump/normal. Only an optical illusion.

In general you're right, and I previously stated it too, the mesh has to have the structure by itself. But for most of the available products the authors only used simple surfaces and added bump/displ maps.
And as already showed: To construct those small surface details, the object must have round the same dense amount of polygons/triangles, as attained by Disp-SubD.

So for the render engine the amount is the same.

AndyS

mjc1016 said:

Richard Haseltine said:

Are you closing the render windows as you go? if not they will sit there with their scene data and consume memory.

And that is not a flaw...it is to allow the render to be resumed.

And even when you do (I normally save the render result, which automatically closes the window), you see in the task manager, that DAZ occupies still more memory compared to the situation before starting the render.

So DAZ still has a serious problem with memory management.

mjc1016

AndyS said:

mjc1016 said:

Richard Haseltine said:

Are you closing the render windows as you go? if not they will sit there with their scene data and consume memory.

And that is not a flaw...it is to allow the render to be resumed.

And even when you do (I normally save the render result, which automatically closes the window), you see in the task manager, that DAZ occupies still more memory compared to the situation before starting the render.

So DAZ still has a serious problem with memory management.

Maybe the Linux/WINE memory manager is just that much quicker than Windows, because I'm not seeing that.  When I close out a window I get an immediate release of the memory...back to the pre-render level.

cheznous2029_28ab1adedc

Richard Haseltine said:

Are you closing the render windows as you go? if not they will sit there with their scene data and consume memory.

Yes, I am closing the render after it get to where I like it.

Richard Haseltine

AndyS said:

Sorry Richard,

Richard Haseltine said:

SubD of five increases the polygon count by 4^5, 1,024, times. Yes, this is a serious drawback to using Iray - the expectation seems to be that normal maps will be used (bumps in Iray have their own quirks). However this is a pretty typical rendering issue, not just in Iray - LW used to work this way, Carrara too, I believe several other amajor applications do not have sub-polygon displacement. One option it to bump the mesh up just enough to give coarse relief and then use normals to smooth it, another is to make sure that areas that are to be displaced start with a higher resolution so that applying SubD does not push other areas - that are already higher resolution - to to ridiculous levels, but of course a lot of items are made for 3Delight where this simply wasn't an issue and so the products weren't modelled and textured that way.

but normal maps are not producing a "real displacement" effect. This is something totally different.

In the attached file you see the difference at the border between the red and blue area.
The left part shows real displacement - on the right is the outcome of bump/normal. Only an optical illusion.

In general you're right, and I previously stated it too, the mesh has to have the structure by itself. But for most of the available products the authors only used simple surfaces and added bump/displ maps.
And as already showed: To construct those small surface details, the object must have round the same dense amount of polygons/triangles, as attained by Disp-SubD.

So for the render engine the amount is the same.

Yes, for something like the beach sand there is no simple solution - but in general I was suggesting using displaceemnt to get a saw-tooth effect and then using normals to soften the appearance of that might be a workable compromise. For the sand I think the nearest thing to a fix might be to have a simple mesh with normals only and patches of higher resolution mesh with displacement that could be layered on top where the ridges were needed to have volume.