IRAY sporadically crashes to CPU

I run a GTX 970 and a GTX 780 ti.  I have noticed after multiple renders, it can crash to the CPU for reasons that are unclear.  Can work great for 2 renders, crashes to CPU, I restart DAZ or restart the system and it works fine.  Here's the log file:

 

2016-28-23 13:28:32.649 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.8   IRAY   rend error: Kernel [18] failed after 0.000s
2016-28-23 13:28:32.652 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.8   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while launching CUDA renderer in core_renderer_wf.cpp:821)
2016-28-23 13:28:32.654 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.8   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): Failed to launch renderer
2016-28-23 13:28:32.654 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.654 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.655 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.656 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.656 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.657 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): Device failed while rendering
2016-28-23 13:28:32.657 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while initializing memory buffer)
2016-28-23 13:28:32.657 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: All workers failed: aborting render
2016-28-23 13:28:32.658 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.658 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.658 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.659 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.659 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.659 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.660 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.660 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.660 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.660 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.661 WARNING: dzneuraymgr.cpp(261): Iray WARNING - module:category(IRAY:RENDER):   1.5   IRAY   rend warn : All available GPUs failed.
2016-28-23 13:28:32.661 Iray INFO - module:category(IRAY:RENDER):   1.5   IRAY   rend info : Falling back to CPU rendering.
2016-28-23 13:28:32.663 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Retry rendering with other devices.
2016-28-23 13:28:32.664 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.0   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.664 WARNING: dzneuraymgr.cpp(261): Iray ERROR - module:category(IRAY:RENDER):   1.0   IRAY   rend error: CUDA device 1 (GeForce GTX 780 Ti): misaligned address (while de-allocating memory)
2016-28-23 13:28:32.664 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : CPU: using 8 cores for rendering
2016-28-23 13:28:32.665 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Rendering with 1 device(s):

 

I don't know if it happens because I'm running different cards or not.  This one happened when I was just running the 780.  Has anyone else had this happen?  I make sure that the scene size is acceptable for the GPU ram.

Comments

  • mjc1016mjc1016 Posts: 15,001
    edited November 2016

    Looks like you may be running out of memory on the cards...what is the amount of memory they have?

    Also, are you closing out and saving each completed render?

    Post edited by mjc1016 on
  • areg5areg5 Posts: 617
    edited November 2016

    The 780 has 3 gig, the 970 has 4.  I make sure all scenes fit in the smaller memory.

    No, I'm not closing out.  I actually like to use this Batch Rendering plug in:

    https://www.daz3d.com/batch-render-for-daz-studio-4-and-rib

    It might work well for 4-5 scenes, then I have to close Daz and restart it, and it works again for a while.  All of the scenes fit in the GPU RAM as per the Iray memory assistant:

    https://www.daz3d.com/iray-memory-assistant

    It seems like after a render, the RAM sometimes doesn't completely clear.  What I'm looking for is consistency.  For an example, if I was to do an animation of say 150 images, it might have to run a long time.  Is there any way around this?  Very frustrating.

     

     

    Post edited by areg5 on
  • mjc1016mjc1016 Posts: 15,001

    If you are not saving each one and closing out the render after it's finished, it's still being held in memory.  Iray allows you to resume a render (make minor changes like max time and then continue), but for that to work, it needs to be held in memory and eventually, a few renders or scenes, later, you'll fill up all the video card's memory with 'finished' but held renders.

    One way around it, but you lose the resume ability...render direct to file.  For animation...an image sequence direct to file would be the best way.  Direct to file also works well for batch rendering, too.

  • areg5areg5 Posts: 617

    Well, the batch renderer renders direct to file by default.

Sign In or Register to comment.