Renders goes to cpu after some frames
![abunezek](https://secure.gravatar.com/avatar/a8773001f52a12f92232187b4b37be7b?&r=pg&s=100&d=https%3A%2F%2Fvanillicon.com%2Fa8773001f52a12f92232187b4b37be7b_100.png)
When rendering multiple frames, doesnt matre if its a video or sequence, after several frams (depend on scene size) it stops render on GPU and goes to CPU which is much slower.
after DAZ3d restart it works again. (1x1080 card)
How do i solve this?
Post edited by abunezek on
Comments
I've seen some discussions about Iray holding on to some of the memory it uses during a render which (over time) can make it so a scene doesn't fit into your VRAM anymore. A couple of ideas come to mind.
First you could use a product like V3Digitime's Scene Optimizer to further reduce the memory your scene needs to render.
Second would be to divide the animation into multiple scene files and use something like ManFriday's Render Queue product. It allows you to queue up several scenes to be rendered and even restarts Studio between each to ensure that memory is being released between each render.
This has been discussed in the two 4.12 threads and is well known. Whether DAZ are going to concede there is a problem and do anything about it is another matter. It happens with small scenes, well within my 8GB VRAM limits, when rendering an image series. Not only that but I'm also getting renders fall back to CPU if I experiment with materials and shaders in a scene - eventually the VRAM becomes overloaded with all the shaders going in and out that it cries "enough". Lastly, rendering a scene and then moving the characters around and rendering it as a subsequent scene is something I have been doing since I started using DAZ Studio. This is not an animation image series, just re-rendering slightly different images from the same basic scene. This too falls back to CPU after a few renders. VRAM is not being managed efficiently and Optix Prime seems like a major culprit too. They seem to have concentrated on the new RTX GPU and left the rest of us to struggle with the problems.
Just FYI, I use Scene Optimizer religiously. The image series problem has nothing to do with that - it fails on the second frame consistently with scenes of any size.
so what could be a slotion? even if i had 50GB of vram it will eventualy run out.
Just to let you know that this specific issue doesn't seem to happen in my pc. I did a simple test and the vram seems to be handled fine, apart the extra buffer needed for optix in 4.12. From gpu-z the used vram in my card is about 2.7 gb and it doesn't seem to grow up in time. I use the iray preview, so there's about 2gb for the working buffer that's loaded at startup, plus about 0.5 gb for the scene itself. My system specs are in the signature. I included the scene duf file for testing.
It would be nice if you can include a test file yourself where this issue does happen. Possibly as simple as possible and including only the free standard content so anyone can test it. This way we can compare and test to see if this issue happens to everyone. This may also help the daz developers to better understand when this issue rises up.
my simple test:
1) load the g8f
2) apply the idle dance animation
3) render
DAZ Studio 4.12.0.86, nVidia Driver 431.86 studio, Windows 10 Pro 1809
Ryzen 3 2200G, Geforce GTX 1060 6G, 16G RAM, 480G SSD
Your test shows VRAM usage at a very low starting point. My renders were working renders, not minimal tests, and VRAM was already approaching 7GB (which, to me, seems strange seeing as the scene itself contained only two characters, two room walls and a little furniture). Nevertheless, the VRAM soon spiked over the 8Gb limit and fell back to CPU. The memory allocation error was recorded in the log file too. As I said previously, I always reduce texture sizes with Scene Optimizer.
The point I am making is that while it might work for tiny scenes, that is of no benefit to me. I think you have said previously that you do all your serious rendering in Blender and I believe that is the way I need to go too but I'm finding it a long haul to teach myself how to get scenes into Blender and then figure out how to set up materials. Neither of the two available free bridging scripts have proved successful for me thus far. That's probably down to my lack of Blender skills.
What I can see is that 4.12 needs an extra 1gb for optix. That's the same if you enable optix prime in 4.11, it takes an extra 1gb for its own working buffer. I can't see any memory leak growing up in time. The fact that I can't reproduce the issue doesn't mean it doesn't exist though. That's why we could share test scenes to check it out.
Of course if the scene was barely fitting 4.11 without optix, then it will not fit 4.12. But this is not a bug.
I use blender because it gets far better animation and rendering options than daz studio. Nevertheless I believe daz studio is growing up fine and if we can help tracking down bugs I believe it's fair since daz studio is free. For example I would never waste my time to help with max or maya since people already pay for them.
EDIT. I tried a bigger scene with an environment and some dressed characters and it seems to work fine, I mean no memory leaks growing up in time. To take more resources I didn't resize textures so they are at 4K. Scene file attached.
Ummm--I think it may be the nvidia driver and how it reacts with Studio (4.12)? I was using 430.86 (431.86 must be a little bug fix) and had no problems. I'm currently on 436.48 and have the problem of dropping to CPU if I render even a tiny scene more than twice in a row.
And I don't have Optix Prime checked. The other difference on my PC is that 431.86 driver is "game ready" whereas the 430.86 was the "studio" nsd version.
4.12 always uses optix, it does not matter if the option is checked or not. If you check it you get a warning in the log that the option is not functional. Indeed it has been removed in the latest beta. As for 431.86 there should be no difference between the game and studio versions. That is, game drivers become studio drivers if they pass the quality check.
I made the change and am now back to 431.86
There's more to the drivers than just game ready or studio because I don''t have that problem anymore (so far anyway). I created a scene, not a huge one, and rendered several times, checked the log and nothing changed re memory consumption at all. Just fiddled lights, positioning in between.
Under 436.48 the scene that blew up (which I had rendered several times under the 430.86 driver with no problems) was small and the changes I made between renders was only positioning (and the ground plane). That scene blew up on the third render and dropped to CPU.
Of course, I have more testing to do...
This is a really huge issue and I wonder why is it not addressed yet? A lot of people render animations and having to split an animation that is 30 frames into 30 different scenes is outlandish. Please, Daz Studio developers, fix this because it really sucks.
A lot of it may well be on the Iray side. The current build cycle (4.12.1.x) has introduced some chnages in the order in which things are done intended to ameliorate the issue wth sequential renders, have you tried that? https://www.daz3d.com/daz-studio-beta
My own tests seem to suggest shader memory seems to be the biggest culprit; On my 6Gb after I hit over 1.1 Gb in texture memory , something will fail, usually the noise optimizer. After a couple of times of just pushing the render button again, it will drop to CPU. Its like clearing the memory is likely an issue. Mesh sizes vary in this equation, so thats interesting, but 1.1 seems to be a hard limit for my setup. Doing more tests this week probably.
Hmm. I am using the 4.12.0 version. I just downloaded the beta build and I will be testing it and let you know if it's any better!
Update: Seems to be working so far! But I encountered a number of issues with the beta build. At first I disabled the "rely on CPU" option and the noise filter kept messing up and not showing properly, then the textures on the character started showing weird stuff like artifacts, then when I tried to start to render the scene again, it would loop infinitely and not use any cpu or gpu power and just stay in that state trying to load forever. After I turned the option back on I so far don't have issues, but I haven't really tested much since then.
I have a GTX 1070 and 8Gram on the card. I noted that whenever I do data intensive render - it automatically goes to the CPU and stalls my i7 32gigram machine.
Even when I have only ticked GPU. Why can't Daz program intentionally only use the GPU for this job? Slightly slower is ok so long as the whole machine does not stall. All the CPU and Ram gets consumed in doing the duty tasked to it. There seems no way of telling it to use only a portion of the CPU and Ram. Systems crash bec of this. If I canged and increased my ram to 64Gig - will there be a difference?
I wonder.
Super RTX 2060 8Gig ram tends to crash alot in Daz renders... is this true?
Please let me know if this has been resolved now and whether I should invest in RTX GPUs.
Aograi : you RAM makes little difference in this case, as in the Iray render is send to your gfx card (gpu) and it doesn't fit, it falls back to cpu. So all the textures and geometry need to fit on your card. Check the render log (expand the little iteration progress window when you render to see when your render throttles back to cpu)
Thanks... I will do that.
As for CPU using all cores, maybe you could try to run DS with a lower priority ... https://www.daz3d.com/forums/discussion/comment/7218/#Comment_7218
It worked on XP, on Windows 7 - I don't know about Windows 10 ...
I had been having this issue a lot recently, even on simpler scenes but of many frames. It would render on GPU and then CPU and back again, one frame gpu, one frame cpu, one frame gpu etc.
Causes issues when using denoising. However, using the BETA fixed this, purely for the new option of being able to completly disable cpu fallback. Rendering without issue now :) (2080rtx)
That is good to hear. Coz everytime it falls back to CPU it affects the rest of the Running PC causing to to stagger or cursor point getting stuck while process is using all the ram for the render of Daz... and sometimes crashing as well.
I have also been having this problem and can confirm that 4.12 Beta Pro does indeed resolve this issue. I did not have to remove the force to not drop to CPU option. I believe the update in the previous version addressed the issue of making the IRAY memory call too quickly between frames resulting in GPU drop. Has not happened since switching to BETA yet. Will advise if it creeps back up again.