[Solved] Daz iRay dropping from GPU to CPU during render

squared002squared002 Posts: 3

EDIT SOLUTION: I solved my case by lowering the stock overclock on my graphics card. Hopefully this helps someone else having the same issue.

Hi.

I've made sure to uncheck CPU from Advanced Settings in render, so the only thing checked is my 1070. Even when doing a simple scene with a model, a couple lights and some clothing, the render will speed through until about 80% and then crash to CPU. I specifically bought an 8GB card to be able to render scenes with it, but this is getting frustrating now.

System:
AMD FX-8350
16GB RAM
EVGA 1070 GTX 8GB - Newest drivers, literally got and installed the card today.
600w PSU (card is 500w recommended)
Windows 7 Ult x64
My Daz library is located on an SSD to help load speeds

Log file from a larger scene I tried. I reduced the texture sizes and deleted some lighting and this was the result:

2018-07-19 03:59:43.734 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating geometry.2018-07-19 03:59:43.845 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Using built-in ray tracing (1.0.0).2018-07-19 03:59:43.845 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Importing scene graph.2018-07-19 03:59:43.845 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Importing geometry.2018-07-19 03:59:44.307 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Geometry import (1 object with 1070k triangles, 1 instance yielding 1070k triangles) took 0.4614622018-07-19 03:59:44.307 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating materials.2018-07-19 03:59:44.356 Iray INFO - module:category(MATCNV:RENDER):   1.0   MATCNV rend info : found 235 textures, 0 lambdas (0 unique)2018-07-19 03:59:44.366 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Emitter geometry import (12 light sources with 12152 triangles, 1 instance) took 0.00s2018-07-19 03:59:44.366 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating environment.2018-07-19 03:59:44.369 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating lens.2018-07-19 03:59:44.369 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating lights.2018-07-19 03:59:44.370 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating object flags.2018-07-19 03:59:44.370 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating caustic portals.2018-07-19 03:59:44.370 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Updating decals.2018-07-19 03:59:44.390 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Allocating 1 layer frame buffer2018-07-19 03:59:44.397 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Using batch scheduling, architectural sampler disabled, caustic sampler disabled2018-07-19 03:59:44.398 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Initializing local rendering.2018-07-19 03:59:44.560 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Rendering with 1 device(s):2018-07-19 03:59:44.561 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : 	CUDA device 0 (GeForce GTX 1070)2018-07-19 03:59:44.561 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Rendering...2018-07-19 03:59:44.562 Iray VERBOSE - module:category(IRAY:RENDER):   1.3   IRAY   rend progr: CUDA device 0 (GeForce GTX 1070): Processing scene...2018-07-19 03:59:46.392 Iray VERBOSE - module:category(IRAY:RENDER):   1.2   IRAY   rend stat : Geometry memory consumption: 35.4182 MiB (device 0), 0 B (host)2018-07-19 04:00:22.651 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : Initializing light hierarchy.2018-07-19 04:00:22.651 Iray VERBOSE - module:category(IRAY:RENDER):   1.2   IRAY   rend stat : Texture memory consumption: 1.71384 GiB (device 0)2018-07-19 04:00:22.944 Iray INFO - module:category(IRAY:RENDER):   1.2   IRAY   rend info : Light hierarchy initialization took 0.29s2018-07-19 04:00:22.947 Iray VERBOSE - module:category(IRAY:RENDER):   1.2   IRAY   rend stat : Lights memory consumption: 2.04511 MiB (device 0)2018-07-19 04:00:23.207 Iray VERBOSE - module:category(IRAY:RENDER):   1.2   IRAY   rend stat : Material measurement memory consumption: 0 B (GPU)2018-07-19 04:00:23.209 Iray VERBOSE - module:category(IRAY:RENDER):   1.2   IRAY   rend stat : Materials memory consumption: 384.539 KiB (GPU)2018-07-19 04:00:23.526 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Scene processed in 38.967s2018-07-19 04:00:23.531 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Allocated 21.0941 MiB for frame buffer2018-07-19 04:00:23.704 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Allocated 1.65625 GiB of work space (2048k active samples in 0.171s)2018-07-19 04:00:23.705 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : CUDA device 0 (GeForce GTX 1070): Used for display, optimizing for interactive usage (performance could be sacrificed)2018-07-19 04:00:24.527 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : Allocating 1 layer frame buffer2018-07-19 04:00:31.368 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00010 iterations after 46.809s.2018-07-19 04:00:38.657 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00020 iterations after 54.098s.2018-07-19 04:00:46.522 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00031 iterations after 61.964s.2018-07-19 04:00:53.990 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00042 iterations after 69.432s.2018-07-19 04:01:04.175 Iray INFO - module:category(IRAY:RENDER):   1.0   IRAY   rend info : Received update to 00057 iterations after 79.618s.2018-07-19 04:01:09.610 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): Kernel [18] failed after 0.039s2018-07-19 04:01:09.610 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): an illegal memory access was encountered (while launching CUDA renderer in core_renderer_wf.cpp:807)2018-07-19 04:01:09.610 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.5   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): Failed to launch renderer2018-07-19 04:01:09.610 WARNING: dzneuraymgr.cpp(307): Iray ERROR - module:category(IRAY:RENDER):   1.3   IRAY   rend error: CUDA device 0 (GeForce GTX 1070): Device failed while rendering2018-07-19 04:01:09.610 WARNING: dzneuraymgr.cpp(307): Iray WARNING - module:category(IRAY:RENDER):   1.3   IRAY   rend warn : All available GPUs failed.2018-07-19 04:01:09.611 Iray INFO - module:category(IRAY:RENDER):   1.3   IRAY   rend info : Falling back to CPU rendering.

Also want to note, while rendering, I had CPUID open and my memory peaked at 87-90% on the GPU. The temp peaked at 59c.

Thank you.

Post edited by squared002 on

Comments

  • ebergerlyebergerly Posts: 3,255
    Is it just me, or are we starting to see a ton of "illegal memory access" errors lately? And are many/most involving Windows 7?
  • Just going to add this in. After walking away from the frustration and relaxing for a little while, I did some reflecting on my setup. Here is my theorycraft based on my setup.

    My motherboard is a 990FXA for my AM3+ socket FX-8350. This motherboard does not have a PCIE 3.0 slot, just 2.0 x16 slots. The graphics card I picked up is an EVGA 1070 GTX SC GAMING ACX 3.0 Black Edition. This card comes overclocked out of the box. I was observing GPU Clock speeds hitting 2000mhz and memory clock speeds of close to 5000mhz (might have been higher, wasn't paying attention to this until afterwards). 

    With the OC out of the box in mind, and the fact I'm funneling through 2.0 and not 3.0, I theorycrafted that perhaps I am getting memory errors because the slot can't keep up with the speed. So I downloaded Precision XOC from EVGA and cranked my GPU and MEM clock speed offsets all the way down. My GPU clock is running at 1733mhz and the mem clock is at 3305mhz now. I haven't finished a render 100% yet, but it has been holding steady on the GPU for 30 minutes now with a larger scene. I'll test this some more when it finishes and post back my findings. But, so far, reducing the out-of-the-box-OC back to something more standard is working.

  • I just went through this same issue, it sounds identical (on the surface at least) and so far your theory and my solution are about the same. The short version is my 1080ti SC's over clocking setting was messing with DS for some unknown reason (my system was up and running for about 4 months before the issue, so I dont know if the PCI reason is correct or not? Someone with more tech experience may be able to answer that). There is a setting in the Nvidia control panel (help drop down, "Debug") that apparently resets the OC setting to the factory specs for other 10 series cards. I changed that and have run a hand full of renders without being able to repeat the "illegal memory access was encountered" error. I know you already foud the solution, but its always nice to confirm with someone else that experienced the same thing that youre on the right track. Good luck! 

  • Just out of curiosity, are you also getting the "Unsupported Hardware" pop up from windows when it scans for upgrades? I'm just curious if that had something to do with the OC issue? It seems that while there wasnt a lot of info on the web about the error in regards to 3D work, a lot of the bitcoin miners are running into lately as well...

  • squared002squared002 Posts: 3
    edited July 2018

    Considering this solved. I ran a 720p render in a large scene for 3 hours and it held in GPU the entire time. I am currently running the same scene in 1080p with some higher render and quality settings for 16 hours now and it is holding the render in GPU. Edited the initial post to reflect solved and the solution that worked for me.

    @allfunandgames3D: No, I was/am not getting an "Unsupported Hardware" message from Windows. Windows saw and worked with the card just fine. And yeah, I hear you, I found like 3 links in Google search for this issue and none of them solved my issue. As soon as I left it more open ended in the search, I found tons of things for mining. Glad to hear you got your issue figured out as well. Hopefully this thread will help those with similiar situations. And you're right about it possibly not being a PCIE slot issue. If I may ask, do you know if your card is sitting in a 2.0 or 3.0 slot? While I believe the issue lies in the overclock itself, I'm curious if the PCIE slot is playing a role in it as well. It really shouldn't as the bandwith between 2.0 and 3.0 is not that great when it comes to actual usage (not theoretical), but computers can be very finicky.

    Post edited by squared002 on
  • fastbike1fastbike1 Posts: 4,078

    What kind of scene keeps a 1070 rendering for 16 hours?

  • ebergerlyebergerly Posts: 3,255
    fastbike1 said:

    What kind of scene keeps a 1070 rendering for 16 hours?

    Perhaps one that is rendering a lot of special effects that might be done far faster in a post production application?

     

  • ebergerlyebergerly Posts: 3,255
    edited July 2018

    But, so far, reducing the out-of-the-box-OC back to something more standard is working.

    Interesting. I have a fairly strict rule that intentional overclocking is not allowed on any of my computers...ever...but I never thought it could cause memory errors. On the other hand, I think it's a sufficiently un-provable theory that only time will tell if it's the actual cause. And turning off the o'clocking, IMO, is always a good idea. Hope it works for you.  

     

    Post edited by ebergerly on
  • fastbike1 said:

    What kind of scene keeps a 1070 rendering for 16 hours?

    I intentionally loaded a large scene with 4k textures, indoors, particle effects, and cranked the render quality from 1.0 to 10.0, convergence to 100% and filtering cranked up high. The object of the render was not to actually finish the render, but to extend the render time as long as possible to make sure it holds in GPU and not drop into CPU. I had things to do in real life, so I wasn't going to be using my computer anyway. I shut it down after 27 hours in render (which it held in GPU the entire time), so I'm calling it a success.

  • fastbike1fastbike1 Posts: 4,078

    @squared002

    Gotcha. Thanks for the info. Definite success. I suspect you are right about funneling through 2.0 although the new 10XX cards are ultra fast. I suspect that may be a bigger contributor. I have a factory OC 980ti thats doesn’t through any errors. 

Sign In or Register to comment.