IRay Error, falling back to CPU

I have a GTX 1070 and a 970 and am having problems rendering a scene. Sometimes it will render but then I change a pose and it fails. Reload and sometimes it works and sometimes it doesn't. It is not very VRAM hungry so when it works, both cards are used. When it fails, neither card is used - only CPU. I don't know if it is scene specific but I've only expereinced it with this scene so far.

This is from the log:

2017-01-31 12:08:53.689 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.3   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while initializing memory buffer)2017-01-31 12:08:53.689 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.6   IRAY   rend error: CUDA device 1 (GeForce GTX 970): Device failed while rendering2017-01-31 12:08:53.689 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.3   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while de-allocating memory)2017-01-31 12:08:53.689 WARNING: dzneuraymgr.cpp(307): Iray WARNING - module:category(IRAY:RENDER):   1.6   IRAY   rend warn : All available GPUs failed.2017-01-31 12:08:53.689 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.3   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while de-allocating memory)2017-01-31 12:08:53.690 Iray INFO - module:category(IRAY:RENDER):   1.6   IRAY   rend info : Falling back to CPU rendering.2017-01-31 12:08:53.690 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.3   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): unspecified launch failure (while de-allocating memory)

 

Comments

  • marblemarble Posts: 7,500

    This is the log when it (the same scene) works:

    2017-01-31 12:20:30.435 Iray VERBOSE - module:category(IRAY:RENDER):   1.4   IRAY   rend stat : Materials memory consumption: 481.598 KiB (GPU)2017-01-31 12:20:31.416 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Scene processed in 51.206s2017-01-31 12:20:31.420 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Allocated 46.8754 MiB for frame buffer2017-01-31 12:20:31.475 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Allocated 1.65625 GiB of work space (2048k active samples in 0.054s)2017-01-31 12:20:31.475 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Used for display, optimizing for interactive usage (performance could be sacrificed)2017-01-31 12:20:31.554 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : CUDA device 1 (GeForce GTX 970): Scene processed in 51.344s2017-01-31 12:20:31.559 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : CUDA device 1 (GeForce GTX 970): Allocated 46.8754 MiB for frame buffer2017-01-31 12:20:31.618 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : CUDA device 1 (GeForce GTX 970): Allocated 848 MiB of work space (1024k active samples in 0.058s)2017-01-31 12:20:32.295 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : Allocating 1 layer frame buffer2017-01-31 12:20:32.345 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00001 iterations after 52.135s.2017-01-31 12:20:32.655 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00002 iterations after 52.444s.2017-01-31 12:20:32.884 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : CUDA device 1 (GeForce GTX 970): Allocated 251.336 MiB of work space (303k active samples in 0.001s)2017-01-31 12:20:32.919 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00003 iterations after 52.709s.2017-01-31 12:20:33.528 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00004 iterations after 53.317s.

     

  • marblemarble Posts: 7,500

    Seems I was wrong about VRAM. A few more tests and it is looking like the fall-back is due to the VRAM exceeding the capacity of the 970 card. I was under the impression that it should fall back to single card mode but it looks like I was wrong about that too.

    On the plus side, the 1070 on its own it almost as quick as the two cards together. I just hate wasting the resources of what was a pretty expensive card less than a year ago.

     

  • marblemarble Posts: 7,500

    Actually not really almost as quick - the same scene ran for:

    1070 + 970 = 310 secs (5 min 10 secs)

    1070 alone = 478 secs (7 min 58 secs)

  • BeeMKayBeeMKay Posts: 7,019

    It should fall back to one card, at least that's what I gathered from previous versions.

    Perhaps it's a bug? Have you checked the official thread? http://www.daz3d.com/forums/discussion/138871/daz-studio-4-9-3-pro-general-release#latest

  • marblemarble Posts: 7,500
    BeeMKay said:

    It should fall back to one card, at least that's what I gathered from previous versions.

    Perhaps it's a bug? Have you checked the official thread? http://www.daz3d.com/forums/discussion/138871/daz-studio-4-9-3-pro-general-release#latest

    That's what I thought too. In fact I've seen it fall back to the 1070 only although, as usual, I begin to doubt my powers of recall. It is so inconsistent that I couldn't begin to describe it to tech support - they always demand reproducibility and I can't say for sure when it will fall back. As I say, I've rendered a scene on both cards, closed DAZ Studio, opened the same scene only to have it fall back to CPU. And it did seem quite stable until the last couple of days and I have not installed any new updates.

  • BeeMKayBeeMKay Posts: 7,019

    You can always provide them with the scene file you made. Unless it contains pletny of self-made stuff, it should help them, given that they have as similar setup.

    Though, you could also purposefully create a scene that you know is more than your 970 can handle, but still doable for the 1070, and see how that flies.

  • oomuoomu Posts: 175

    Don't forget, it may possible for others softwares running in your computer to also push stuff in your GPU. Are you sure you don't also sometimes use a software like Photoshop beside DAZ ? It may explain why sometimes your GPU can eat the whole scene, sometimes not.

    Some utilities like gpu-z can tell you how much gpu ram is available. Check it before to launch your test.

    And of course bugs: leaks, memory never freed after use and so on. Don't hesitate to do a cold reboot to clear ANY doubt.

  • marblemarble Posts: 7,500
    oomu said:

    Don't forget, it may possible for others softwares running in your computer to also push stuff in your GPU. Are you sure you don't also sometimes use a software like Photoshop beside DAZ ? It may explain why sometimes your GPU can eat the whole scene, sometimes not.

    Some utilities like gpu-z can tell you how much gpu ram is available. Check it before to launch your test.

    And of course bugs: leaks, memory never freed after use and so on. Don't hesitate to do a cold reboot to clear ANY doubt.

    Yep - been through all that. I have also tried with a big scene I know would not fit on a 970. It went straight to  CPU with both cards enabled but rendered on the 1070 with the 970 disabled. So I'm pretty convinced that the fallback is inconsistent at best. However, I'm also suspicious of the 970. I have come across reports from gamers that it becomes unstable with high memory load and, significantly, this is my second card. I had the original replaced by the supplier because it failed (inconsistently) when the 4GB limit was in sight. 

    You are probably right about a bug where the memory is not freed after a render. I have made it a habit to exit DAZ Studio before trying a re-render of a scene because of that and I know I'm not alone in this. Others here have complained about it.

Sign In or Register to comment.