CPU fallback (Update and partial workaround)

13»

Comments

  • PerttiAPerttiA Posts: 10,024

    reserv888 said:

    I have run stress tests with OCCT and Furmark. Maximum load on the GPU doesn't cause the temperature to rise. Maximum load (11GB) on the GPU memory doesn't cause black screens.

     

    An idea that came to me while monitoring GPU-Z during a render... Have you stress tested your RAM?

    In my case with the scene loaded, 14.5GB's of RAM was used, after starting the rendering RAM usage went to 16.7GB's but shortly after peaked at 17.2GB's until it came down to 16.9GB's for the rest of the rendering. 

  • robertswwwrobertswww Posts: 790

    @reserv888 I sure hope the Daz community can get to the bottom of this problem for you, because I know how frustrating these type of issues can be. 

    Since this problem started for you in January 2021, have you done any Windows PC updates (or BIOS updates) around that time or any updates since then that may have triggered the instability? (if so, you could try a rollback to a time before the problems started, or see if there are any new updates available that could potentially help)

    Do you happen to have another GPU available (even a weak one) to test with your simple basic scene file to see it it can render without problems?

    Have you tried to render with CPU unchecked in both Photoreal Mode and Interactive Mode, and Allow CPU Fallback unchecked (so it won't try and use the CPU)? If you make any changes, restart and try the same render and then check the log afterwards to compare results.

    Alternately, uncheck the GPU and only render your simple basic scene via CPU to see it everything works fine while bypassing the GPU.

    Also, check your "CPU Load Limit" at the bottom of Render Settings-->Advance tab I believe it is usually set to your number of cores of your PC (8 for your i7 11700)... if not try setting it to that, and if it's already set to match your PC, you can try and reduce it to see if it has any effect on your heat spike at time of CPU usage.

    What is causing the GPU to fallback to CPU has still not been determined.  There was an issue awhile back with some GTX 2080 Ti cards that had Micron M6 and M7 GDDR6 modules that would overheat (Samsung GDDR6 modules were OK).  Some users were able to fix GTX 2080 Ti issues by under-clocking their GPUs.
    https://www.extremetech.com/gaming/280654-nvidia-confirms-some-rtx-2080-ti-gpus-are-defective-promises-remedy
    https://www.gamersnexus.net/guides/3394-rtx-2080-ti-artifacting-failure-analysis-crashing-black-screens

    It could also be a Power Options issue... try turning off Power Saver in: Control Panel-->Hardware and Sound-->Power Options

    From the graphs, your GPU temps look stable (and your GPU passed the stress tests), but your GPU fails and your CPU temps spike when the render hits CPU fallback. That could either be a cooling issue, or a faulty PSU, or a RAM overload issue. Let's try and tackle these 1-at-a-time:

    1. Cooling:
    If your GPU is not overheating, it may be some other component in your system that is overheating and causing a voltage spike that trips your GPU to fallback to CPU. Heat can cause VRAM instability and this could be due to an overheating PSU...

    2. PSU:
    Check if your PSU that has a fault.  Run your OCTT software again for a few hours, but this time focus on the PSU instead of the GPU, and then check the generated voltage and temperature values, and make sure there are no major voltage spikes or drops.

    3. RAM:
    This may happen when the scene is dumped back to the system RAM, as it takes a lot more RAM than VRAM to render a scene. You mentioned that your GPU has 11GB VRAM.  Ideally, your PC should then have about 4x that in system RAM = 48 GB (or 64 GB)... check your Memory usage in the Performance tab of your Task Manager as your scene renders.

    Also, while your in the Task Manager, you should also customize it to monitor your GPU:
    1. In Task manager go to the "Performance" tab to see graphs, select the GPU in the list on the left, and then click the triangle icon at top-left of one of the graphs and change it to CUDA.
    2: In Task Manager go to the "Details" tab, Right-Click on the column headers (ex. Name) and choose "Select Column", then enable the one(s) you want by checking the box next to: GPU, GPU Engine, Dedicated GPU Memory, Shared GPU Memory - and you will get application-specific values.

    I found another thread here in the forums that has several suggestions to try and fix the problem you are experiencing... 
    Intermittent drop from GPU to CPU rendering in Iray
    https://www.daz3d.com/forums/discussion/425046/intermittent-drop-from-gpu-to-cpu-rendering-in-iray

    One of the fixes suggested was: To allow access to GPU performance counters to all users
    https://www.nvidia.com/content/Control-Panel-Help/vLatest/en-us/mergedProjects/nvdev/To_enable_access_to_GPU_performance_counters_for_developrs.htm

  • PerttiAPerttiA Posts: 10,024

    Any warnings or errors in Windows logs that would coincide with the black screens?
    Computer Management->Event Viewer->Windows Logs->System

    The GPU/CPU temperatures could also be an issue as both your temperatures are about 20C higher than mine while rendering (I'm running i7-5820K and RTX3060)

  • AgitatedRiotAgitatedRiot Posts: 4,437

    PerttiA said:

    Any warnings or errors in Windows logs that would coincide with the black screens?
    Computer Management->Event Viewer->Windows Logs->System

    The GPU/CPU temperatures could also be an issue as both your temperatures are about 20C higher than mine while rendering (I'm running i7-5820K and RTX3060)

    I had to underclock my CPU, it would thermal throttle while rendering. Look at my boot clock and what my system proposed.   

    CPU.jpg
    390 x 764 - 28K
  • reserv888reserv888 Posts: 1,143

    I will be travelling for a couple of days and wont have access to my computer. Therefore I can't test your suggestions until next week.

    Thanks!

  • reserv888reserv888 Posts: 1,143
    edited May 2022

    I tested to make a pure CPU render. The CPU temperature went up to 100 degrees C in a matter of seconds.
    I canceled the render.

    Post edited by reserv888 on
  • AgitatedRiotAgitatedRiot Posts: 4,437
    edited May 2022

    barbult said:

    If your CPU is going up to 100 degrees C, you don't have adequate cooling in your computer.

    I second, What are you using for CPU cooling? Is this an old build like more than 2 years old? If it's more than 2 years old, a new layer of thermal paste might help.

    poke around here also Re: i7 11700 high temperature - Intel Communities

    Post edited by AgitatedRiot on
  • AgitatedRiotAgitatedRiot Posts: 4,437

    reserv888 said:

    I tested to make a pure CPU render. The CPU temperature went up to 100 degrees C in a matter of seconds.
    I canceled the render.

    Take a look at this video I made for you.

    https://1drv.ms/u/s!AsLRJ6DzP9KIgbELdvhdOnJdPsZ_IQ?e=WaHKFV ;

  • robertswwwrobertswww Posts: 790

    @reserv888 With your pure CPU-only render overheating... I think you can probably rule out the GPU having a problem.

    @AgitatedRiot Thanks for sharing your video... the Intel Extreme Tuning Utility (i.e. Intel XTU) is a cool piece of software.
    https://www.intel.com/content/www/us/en/support/articles/000006636/processors/processor-utilities-and-programs.html

    List of Intel XTU Supported Processors:
    https://www.intel.com/content/www/us/en/download/17881/intel-extreme-tuning-utility-intel-xtu.html?

    Unfortunately, it looks like the XTU Utility doesn't support the Intel i7-11700 processor (and only works with the i7-11700K and i7-11700KF variants of this particular processor series).

    However, there is a less sophisticated alternative to XTU... with Daz Studio running, launch the Task Manager go to the "Details" tab, and Right-Click on DAZStudio.exe and select "Set Affinity" and you can uncheck CPU cores, so Daz Studio is not taking up all of your CPU power and it should run cooler until you find out the exact cause of the overheating issue.  Until the exact cause is determined, you may want to take the case off your PC and set an external fan blowing over it to keep it cool while using it.

  • AgitatedRiotAgitatedRiot Posts: 4,437
    edited May 2022

    robertswww said:

    @reserv888 With your pure CPU-only render overheating... I think you can probably rule out the GPU having a problem.

    @AgitatedRiot Thanks for sharing your video... the Intel Extreme Tuning Utility (i.e. Intel XTU) is a cool piece of software.

    However, there is a less sophisticated alternative to XTU... with Daz Studio running, launch the Task Manager go to the "Details" tab, and Right-Click on DAZStudio.exe and select "Set Affinity

    You can do that in Daz>render Setting>cpu load limit. 

    Cpu Diagnostic Tool Intel® Processor Diagnostic Tool

    Threads.jpg
    533 x 279 - 26K
    Post edited by AgitatedRiot on
  • Norman_RNorman_R Posts: 95
    edited May 2022

    My graphics driver always crashed while rendering. I turned off CPU fallback and render in memory mode. Since then no more problems. (RTX 3090)

    Post edited by Richard Haseltine on
  • reserv888reserv888 Posts: 1,143

    UPDATE 

    Thanks a alot to everyone who have provided help and ideas. Especially PerttiA, AgitatedRiot (thanks for the video!), and robertswww. You have been super!

    While the render problem persists, with your help I have found a workaround:

    Underclock 7%, AND press WIN+Ctrl+Shift+B

    I have not receieved any black screens if I do booth of them. I also have to check the amount of VRAM used, to stop the render, since the processor temperature goes way above 70 degrees C if the CPU fall back kicks in.
     

    I still have major problems with dForce simulations, but at least I can render now.

    Thanks again everybody!
     

  • robertswwwrobertswww Posts: 790

    @reserv888 So happy to hear that you can render again! The work-arounds can be a bit inconvenient, but at least you have no more black screens.

  • I found something that solves the problem for me. However, I cannot say whether this is a problem with the driver or the Dazstudio: If I limit the target temperature of the graphics card so that it remains below 80 ° C, there is no fallback. This limits the performance to about 80%, but 80% rendering with the GPU is significantly more than 100% with the CPU.

    My guess is that either Dazstudio is watching the temp and kicking out, or the driver is sending a warning that Studio is misinterpreting. That's your part or that of the Dazstudio. Found a solution for me.

  • Richard HaseltineRichard Haseltine Posts: 100,948

    agelmarjargad said:

    I found something that solves the problem for me. However, I cannot say whether this is a problem with the driver or the Dazstudio: If I limit the target temperature of the graphics card so that it remains below 80 ° C, there is no fallback. This limits the performance to about 80%, but 80% rendering with the GPU is significantly more than 100% with the CPU.

    My guess is that either Dazstudio is watching the temp and kicking out, or the driver is sending a warning that Studio is misinterpreting. That's your part or that of the Dazstudio. Found a solution for me.

    80C should be well within the safe range for most GPUs, as far as I ama ware. If, for soem reason, it was causing the device to stop rendering then Iray would drop to CPU/stop (deepnding on setting) - certaibnly others have reproted rendering with higher temps so it isn't a general issue at that threshold.

Sign In or Register to comment.