Iray/CUDA problem - anyone else?

jstevewhite

I have a Mac pro with an ATI 7970 as my primary display and an NVidia GTX 780 just for the CUDA processing (no monitors attached). DS 4.8, GF3. In iRay 'Advanced' I have CPU unchecked and the NVidia CUDA entry checked. When I do an iRay render, it works fine and engages the CUDA system properly - It's pretty clear that it does because a render takes 8 minutes that took almost forty minutes with just CPU. So far so good. But at some point as I work - I run renders to check lighting, say, and cancel them after just a few iterations - it will stop using the CUDA cores and use only CPU, even though CPU is unchecked. I can tell because the render suddenly takes a long time, and all 12 cores are maxed, while DS only uses about 200% when CUDA is working properly. 

Anyone else have this issue, or a suggestion on mitigating it?

Totte

What does the DAZ Studio log say? You find it under Help>Troublwshooting->View Log file. It's pretty detailed when it comes to IRAY reports. 

All I know is that there are issues with "non suported" nVidia cards on OSX, which really is every cards except the 2GB 680 and the Quadro cards (K5000, K6000, M6000), but they play in another vallet-legue.

But the log might give you hints, like "timeout cancelling render" or such things.

 

 

jstevewhite

Thanks for the tip on the log. I'll check it in a bit. (when this render is complete :D )

The CUDA drivers install cleanly and the system preferences panel shows active CUDA. I'm not actually using it as a display card - it's not supported in that role, as you pointed out. Even the NVidia drivers are incredibly finicky - they only work on one specific build. It would show video if I plugged in a monitor, but it doesn't get acceleration for display.

jstevewhite

Thanks. That helped. Here's what I found:

Iray INFO - module:category(IRAY:RENDER):   1.11  IRAY   rend info : CUDA device 0 (GeForce GTX 780): Scene processed in 33.575s
Iray INFO - module:category(IRAY:RENDER):   1.11  IRAY   rend info : CUDA device 0 (GeForce GTX 780): Allocated 19 MB for frame buffer
Iray INFO - module:category(IRAY:RENDER):   1.11  IRAY   rend info : CUDA device 0 (GeForce GTX 780): Allocated 804 MB of work space (1024k active samples)
WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.12  IRAY   rend error: CUDA device 0 (GeForce GTX 780): out of memory (while launching CUDA renderer)
WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.12  IRAY   rend error: CUDA device 0 (GeForce GTX 780): Failed to launch renderer
WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.11  IRAY   rend error: CUDA device 0 (GeForce GTX 780): Device failed while rendering
WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.11  IRAY   rend error: All workers failed: aborting render
WARNING: /src/pluginsource/DzIrayRender/dzneuraymgr.cpp(261): Iray WARNING - module:category(IRAY:RENDER):   1.11  IRAY   rend warn : All available GPUs failed.

 

I'm a little puzzled though. It's a 3GB card, and this looks like it only allocated 1 GB or so. It must not be clearing the GPU memory each run because if I restart Daz Studio, it will work fine on this render. I need to find some way to monitor GPU memory. 

 

Totte

I think memory cleanup sometimes is a little fishy, which version of CUDA are you useing and which driver (I think you need a decent Driver for it, check for nVidia WebDriver (it's called so as it gets downloaded from the web). Cuda 7.0.xx is the latest. I had CUDA 5 and an old driver and I also had lots of problems, mostly solved wirth CUDA 7

jstevewhite

Yeah, I'm running CUDA 7. The Web drivers don't do me any good - they won't install on this machine because of a build number mismatch. Thanks for your help. 

I need a realtime memory usage monitor for CUDA now. LOL

jstevewhite

If you're interested, iStat Menus has a realtime GPU monitoring widget!