Iray Rendering GPU Usage Bug
kickflipsonly
Posts: 29
I'm pretty sure this bug has been around off and on for a while, trying to see if anyone's figured out a solution/fix to it.
When I've set up a scene the way I like it, I'll hit render, and the iteration count/speed of the render are quite slow.
I'll cancel the render, and in advanced render settings I'll un-check then re-check the graphics card I'm using for rendering (as in, de-activating it then re-activating it for use in rendering), and then when I restart the render, the iteration count/speed of the render are incredibly fast. Sometimes un-checking/re-enabling my graphics card doesn't work and I'll have to keep disabling and re-enabling it, aka doing the same exact thing to fix the issue, but with enough patience I can get my render to go "fast".
Keep in mind that other than the disabling then re-enabling of my graphics card in the advanced render settings, no other settings or objects are changed. Not one single thing.
Anyone know why this is happening? I'm just tired of having to cancel a render 2-3 times to get the bug I'm experiencing to not happen.
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
I don't recall ever hearing about an issue like this in my years using DAZ Studio (or any other software for that matter). Which leads me to wonder if it's not a bug, but rather a problem with your computer. Maybe a BIOS problem, driver problem, bad installation of Studio, or any number of things.
You might want to search online for what is probably a long list of resources for figuring out why a GPU isn't being recognized. Personally, I'd first re-install Studio and make sure all my drivers and BIOS are up to date. And also see if different software has problems recognizing the GPU.
Thank you for the reply. My issue isn't that Daz isn't recognizing my graphics card half the time -- it's always being recognized. When I hit render, there's like a (roughly) 50% chance my render goes slowly - the telltale sign of this is the Iray iteration count... it will go up very slowly, and the render will take a long time.
Then, when I cancel that render, I'll go into my Advanced Render settings, and uncheck my GPU to disable it from being used to render, but then immediately re-enable/re-check it. This, in its entirety, takes like 3 seconds.
I'll start up the exact same render immediately after doing this. Nothing else changes, I don't move anything, I don't save anything, I don't restart or reset anything outside of Daz, literally nothing changes. And yet, often times when I restart the render, it goes much, much quicker. The Iray iteration count increases at a much faster pace, and my render finishes quicker as a result.
To reiterate, in both the slow/fast versions of the exact same render, my GPU is being used. I verify this in MSI Afterburner/Task manager (cuda core usage). This is why this problem is so annoying, it feels so unintuitive... it genuinely makes no sense.
So the act of re-selecting the GPU might be unrelated and irrelevant to the issue, since it's somewhat random, right? Maybe it's just the fact that you're re-rendering the scene after some amount of time. Maybe it's a high-VRAM scene that the GPU can't quite handle or something like that. Or maybe a bad installation of DAZ. Or a whole list of other things.
My first question is What version of DS are you currently using?
Second, What GPU?
Next, what's the actual render time difference between just letting it go, and doing the stop/start?
I ran a couple quick tests in 4.12 with a P106-100, and wasn't able to achieve any, significant, difference in render times.
The scene rendered out in ~24s either way.
Scene: G8F(Base), 80's sport swimsuit, bayla hair, default pose and render settings, Pixel size 1000x1000.
Using 4.15.0.13 64 Bit Pro Public Build.
RTX 3080 is my gfx card.
I generally render slightly more complicated scenes, quite a few various ones, so it's hard to give an exact difference in time. One render might run 2 to 2 and a half hrs when it's bugged out slow .. then when I uncheck-recheck it the graphics card as previously mentioned, it'll be done in 45 minutes, give or take?
@ebergerly - You could very well be right, and the process of unchecking/rechecking it is completely irrelevant. I don't use my CPU to render at all, and I regularly check how much each of my renders/scenes are using in terms of VRAM by way of Daz's log file, so I don't think it has anything to do with high VRAM usage, though maybe there's some bug happening with it? I don't know. I also doubt it's a bad install of Daz, given that this issue has persisted through multiple versions, and also because I keep it updated.
Go into Task Manager/Details, then right click on any of the column headers, and select anything that says "GPU". One of those will be Dedicated GPU memory, and will tell you exactly how much GPU VRAM DAZ Studio is using. It will also tell you how much all the other processes are using. I'm guessing that if your renders are in the 1-2 hour range with an RTX-3080 then it's probably a big VRAM scene. Maybe when you cancel the render, then re-start, there's not enough time to release the memory and it overloads or something? Also, on the bottom of the Render Settings/Advanced are the three options checked (including allow CPU fallback)?
I do not allow CPU fallback, the other 2 are checked. And yes, my scenes can be somewhat complicated, but never in the 10GB range (3080 VRAM size). I think this issue I'm having might be hard to detect for the average user, as most don't use Daz for complicated/very high quality renders. I only noticed it when I made the switch to the RTX 3080. Given it's formidable power, I was noticing slow renders and very confused. Then by closely monitoring everything during a render, I discovered the issue we're talking about now.
The logfile doesn't tell you how much VRAM is/will be used while rendering - Difference between reality and what's been told in the log has been about 20% and that's on top of what the OS, DS and the scene are using already before start of rendering.
Made some testing yesterday;
First case was with one G8F figure loaded to an empty scene with clothes and hair, lit with three point lights. (log said it would use ~2GiB:s of VRAM)
Second case was four G8 figures with clothing and hair, architecture and lit with three point lights. (log said it would use ~4.7GiB:s of VRAM)
Checking and unchecking the GPU may reset the cards VRAM.
If it started after you installed the RTX 3080, shouldn't you be assuming that the problem lies with the RTX-3080? Drivers, BIOS, firmware?
By the way, you're not doing any overclocking or anything that might make your GPU thermally throttle are you?
Happens to me all the time in 4.10, 4.11. (Haven't installed .12, .14, or .15 but it sounds like it's not been fixed.) Typically see it on the second or subsequent render or if I've run an iray preview, even if the vp is subsequently closed. My hypothesis is that it's either an iray memory leak or an intended behavior due to how NVIDIA implemnts caching of previously compiled resources for future use), as I can see available vram drop on each subsequent render until the GPU goes TITSUP. Usually, unchecking the GPU, running a quick iray preview (5-10 iterations at most), quitting the preview, and re-checking the GPU fixes the issue, but the only 100% reliable solution I've found is to quit and restart DS between renders.
Everything is updated on my end. If I were to make an assumption, it wouldn't be that it's a driver or BIOS thing, it would be that Daz has issues with the new RTX cards. But I can't even confirm this is exclusive to the new RTX cards. All I know is something weird is going on, and I feel like hyper-focusing on a super simplified explanation of the issue I'm having really illuminates how weird this whole problem is.
I run a render, iteration count/render completion time are slow. I cancel it, uncheck-recheck the GPU in the advanced render settings, that exact same render improves in speed in every measurable way. I change nothing in and outside of Daz besides what I just said.
Again, go into Task Manager and see what VRAM DAZ Studio is using, and see what else is going on. Only the facts will solve the problem.
So in a scene I just rendered, it used 9.5GB of VRAM. This render was fairly simple, one character, not-too-complicated background, it took 17 minutes and had 10.5k Iray iterations - it was a fast one. One thing I should note is that this bug I've talked about doesn't always happen. I don't always have to cancel a render and uncheck/recheck my gpu. More than that, sometimes I'll have to cancel a render (and uncheck/recheck the gpu) multiple times before I get a "fast" render.
Now that the render has finished, it says Daz is using 6.7GB of VRAM in the viewport... the scene is still loaded, but nothing Iray related is happening. Perhaps this is the issue?
So a one character scene took 9.5GB of VRAM on a 10GB VRAM GPU?
Indeed, and as I said, right now, with nothing rendering (and the scene still being open) I'm looking at Daz using 6.7GB VRAM.
Using that much GPU VRAM for what sounds like a simple, 1 character scene doesn't sound right at all. And if Task Manager is showing that DAZ Studio is the process using all of that 9.5GB, there's a problem. You're operating at the point where the GPU will be unable to fit the scene into VRAM and render it. Which means it would fall back to CPU rendering, which sounds like what's going on in your case.
I have CPU fallback disabled. Also, I'd notice if my CPU were being used in a render. I used to use my CPU in rendering, and I'd watch its usage fly up to 100% when I used it. It would tax my entire system a lot more; webpage navigation would become laggy, for example. None of this happens when I render now.
Right now I'm running that exact same render - 30% CPU usage from Daz.
Then perhaps if the GPU fails due to insufficient VRAM, and CPU rendering is disabled, rendering proceeds via the GPU BUT it grabs scene data from main memory or something, which might explain the slow renders, since you now have to access system RAM.
But still I don't see why a one character scene requires almost all of the GPU's 10GB? I load a G8 and render and all it uses is like 2.5GB, nowhere near 9.5GB.
Anyway, DAZ/Iray doesn't release all the VRAM after a render, presumably because it makes no sense to erase it and then have to reload it from the CPU and system RAM when you hit Render again.
Does this happen with a scene that only takes, say, 3GB of GPU VRAM? You might try it by just loading a simple G8 or whatever and hit render.
Just loaded a default G8F with Basic Wear, render took like 15 seconds with about 500 iterations, used 4GB VRAM. Was a completely new scene.
Without the rendering running, and just the "scene" loaded (g8f in basic wear), 3.5GB VRAM is being used.
Sure seems like you're hitting the VRAM limit of your GPU and suffering the consequences. Clearly, if DAZ/Iray is using 9.5GB of 10GB, there's a problem, and your renders will suffer. You can do some tests to see if your problem happens with smaller scenes that use less GPU VRAM. But at this point it seems like it's hitting the VRAM limit and as a result you're getting slower renders, maybe due to having to grab scene data from system RAM, not onboard GPU VRAM. Not a bug, just a scene that's too big.
But why am I at 4GB VRAM for a g8f in basic wear render?
I feel like there's something wrong with that, and therein lies the issue. The render is super fast of course, but I don't know why the VRAM usage is so high.
Where did you measure the 4GB of VRAM? Is it just the DAZ Studio process, as measured in Task Manager/Details like I mentioned?
Yes, that's how I'm measuring it.
Seems a bit high, but maybe if you have some fancy textures or HDR's or something.
Since rendering a scene that is inside an enclosed room/box tends to take a long time to render, you might want to try that. And if it's a simple scene inside a building/room whatever with no characters it will take a long time but not use much VRAM. You can see if your problem still exists when you're nowhere near the GPU VRAM limit.
Just put a camera and a light inside a building or whatever and see what happens.
Yeah, I'll keep tinkering with it. Thanks for your help, if I figure out anything meaningful I'll update this.
I was looking for reasons Daz would not render on my GPU, and I ran across this thread. My scene is relatively simple and the log shows it was no where near the 11 GB VRAM limit of my card (1080TI). I tried closing Daz and re-opening it to no avail. Mind you I have had no problems in the past with far more complex scenes using the GPU for rendering.
I tried the process you outlined in your original post of deselecting and then immediatly re-selecting my GPU in the render advanced properties, and tried rendering again. The scene that just failed, rendered on the GPU. I think you are onto something here, and I will experiment with this more. I had assumed I had exceeded VRAM limit, but apparently not.
I would note that I also have have Octane installed. Do you? I was wondering if Octane grabs the VRAM memory on startup, and deselecting and reselecting my GPU in IRAY settings might re re-asserting the cards allocation. I need to experiment more.
Sorry, I don't know if this will be helpful, but I've noticed DAZ does not release VRAM/memory when the render is still open (if you render to a new window). Closing or saving the render output frees up the memory. This may cause fallback to CPU or perhaps just tied up resources if you leave multiple renders open. Again, not sure if this is relevant, just thouught I'd share my experience.