24 Cores Of Rendering Goodness
Garstor
Posts: 1,411
I finally stopped procrastinating and built my "big server." :coolsmirk:
Ultimately, this fire-breathing dragon is going to be running a simulated SQL Server 2012 failover cluster as the lab machine that I desperately need for work. But I wasn't about to let Carrara go without some loving!
I used 2 AMD Opteron 12-core CPUs and loaded up with 64 GB of RAM. Tomorrow I start building this machine's "little sister" (just one 12-core CPU and 32 GB RAM).
Here's the Render Room screaming along with (nearly) all the dials turned to 11...
Garstor is happy tonight!
Daz 3D is part of 
Connect
DAZ Productions, Inc.
7533 S Center View Ct #4664
West Jordan, UT 84084
Licensing Agreement | Terms of Service | Privacy Policy | EULA
© 2024 Daz Productions Inc. All Rights Reserved.
Comments
Ya got me drooling :bug:
nice to have big bucks
Actually, I cashed in the remainder of my Microsoft shares. Most of that is for my move to Seattle next year, but I do need these machines for work (and I do miss the Microsoft labs) to build a mini-network of virtual machines.
I dropped about $6000 on a system last Christmas from http://www.digitalstormonline.com/ I've been more than pleased with the system and the service and actually thought I was rockin Daz Studio and Carrara pretty hard till I seen this!
Ah the vagaries of Moore's Law!
It was just over $6000 for both of my servers - nearly everything bought through New Egg; the hard drives I had to get through Amazon.
For the record, the GI sample living room scene is providing quite the workout.
Garstor---
Wow ---sounds like a great machine-----if you could you post the specs ie. ram (company/ type) Mobo etc ---I am sure others might be interested as well. Always wondered about those opterons with all those cores. Its so helpful to have specs of a working machine to copy when making a new rig or buying components.
And of course --the egg is just awesome.
rich
Hello Garstor!
Nice to have so much buckets but its not nice to have a very big bug:
Why are the shadows of "DAZ3D" incorrect ??? You can see it especially at the last "D"-letter...
Greetings
tezen
SPECS - msinfo32.exe
OS Name Microsoft Windows Server 2008 R2 Enterprise
Version 6.1.7601 Service Pack 1 Build 7601
System Manufacturer ASUSTeK Computer INC.
System Model KGP(M)E-D16
System Type x64-based PC
Processor AMD Opteron(TM) Processor 6238, 2600 Mhz, 6 Core(s), 12 Logical Processor(s)
Processor AMD Opteron(TM) Processor 6238, 2600 Mhz, 6 Core(s), 12 Logical Processor(s)
Total Physical Memory 64.0 GB
PARTS
ASUS KGPE-D16 SSI EEB 3.61 Server Motherboard Dual Socket G34 AMD SR5690 DDR3 1600/1333/1066/800
AMD Opteron 6238 Interlagos 2.6GHz 16MB L3 Cache Socket G34 115W 12-Core Server Processor (x2)
ASUS ENGTX550 Ti/DI/1GD5 GeForce GTX 550 Ti (Fermi) 1GB 192-bit GDDR5 PCI Express 2.0 x16
* not a bleeding edge video card because the primary purpose of the machine is to run virtual servers - but I did need something beefier than an on-board video card
Western Digitial Caviar Black 1TB SATA-3 hard drives (x5)
Athena Power CA-SWH01B90 Black Steel Pedestal Server Case
* this was the only case I could find to fit the over-sized motherboard - only caveat here is that it comes with a single fan, so I may need to add more in the future
* also, I am not sure about blaming the case or the video card, but the when the card is seated in the mobo, it is still too tall to be secured to the case with screws
BE AWARE: I am still investigating this and it is probably AMD's chip design fault...but I noticed in PerfMon that occasionally the total CPU processing would drop to near zero. It quickly shot back up to over 80% but would eventually repeat this. I suspect that this is caused by the on-chip L1 cache becoming invalid and needing to be flushed. Certainly a major performance if you are doing genetic research into cancer or something...but probably not serious for Carrara - just annoying for an uber-geek.
Honestly; I would have preferred buying Intel but I wanted tons of cores and the price couldn't be beat. I was feeling better about the purchased when I learned that 3DAGE has used Carrara exclusively on AMD chips. :-) But for long-term, "real" server-grade work, Intel is probably the way to go.
I hear you brah! Fortunately, I didn't make the scene! :coolhmm:
That's a nice system Garstor! Hopefully it'll be able to up your Carrara dosage and take the edge off your current environs. :cheese:
Work??!!? What's that? SQL-what? SharePoint-huh? :-D
Oh if I could only win a lottery...
Work??!!? What's that? SQL-what? SharePoint-huh? :-D
Oh if I could only win a lottery...
That must be Windows code. Please don't port it to my system. ;-)
I don't think that's a bug so much as a consequence of the scene set-up and render settings.
If you take a look, the light through transparency is disabled for the main setting but enabled for the indirect light.
The caustics are also on. We are actually seeing shadows that are too dark rather than missing shadows and this shows on every letter.
This may be a stylistic choice by the scene designer.
I've never seen a 24 core render before. Looks like fun.
Why do the tiles only number to 8?
That surprised me too. That might be considered a bug in Carrara display of the rendering tiles; however, I am more inclined to point at the MSINFO32 data...the AMD chips are actually 6-cores with Hyper-Threading / Hyper-Transport (or whatever) technology enabled.
I bet that Carrara is actually counting cores via socket-to-logical processor ratios rather than summing up the grand total of all available logical processors.
On that note, I'd love to see Carrara implement an affinity mask feature for CPUs. That way you could "give" X-number of cores to the Carrara process and reserve Y-number of cores for the rest of the system.
Garstor,
Not really hyperthreading. They have 12 integer cores and share a floating point unit between each pair of integer cores. Much better design than Intels hokum with using wait states and claiming twice the power.
Joe
A setting in Carrara would be nice. I set affinity through the task manager.
Apologies if you already know this but;
right click on the Carrara process and select set affinity.
Just uncheck which core(s) you want to release.
You can do this mid render if you want and then re-check it when you want to give it full bore.
A good thing is that Carrara will still use the cores when they are available but will release them when other processes want them.
Personally I prefer to leave my render machine to render alone as much as possible but there are a few occasions when I need to use software that is not on my laptop or on my second string PC. If you are rendering on a server clearly that is not an option.
The obvious question...how much speed improvement compared to a "normal" PC with 4 cores/8threads; say i7 2600 3.4GHz for duplicate render?
My guess is 24 vs 8 cores is a little under 3x faster to account for overhead...and optimum size of render square tile size (assuming CPU speed is same and RAM sufficient)
Do you have any data?
I must confess my deepest shame for not having thought of this myself! :red:
I'll get this remedied later today...find an older scene on my other machine and use the statistics tab from the Render Room to compare both boxes results.
More to follow after the experiment!
True! That is an easy option to forget about. It is also controlled by the OS rather than the process; so it doesn't persist between process lifetimes. I'll try to remember it though when I get a long-running physics animation calculating...
VERY surprising results in my "render race" between the two machines.
I posted the MSInfo32 stats for the new AMD server earlier in this thread. Here are the stats for my older Intel machine:
OS Name Microsoft Windows 7 Ultimate
Version 6.1.7601 Service Pack 1 Build 7601
System Type x64-based PC
Processor Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz, 2934 Mhz, 4 Core(s), 8 Logical Processor(s)
Total Physical Memory 7.99 GB
Here is the scene I was rendering: http://www.flickr.com/photos/garstor/6981911339/in/photostream
Scene Render Settings
Width 1680
Height 1050
AntiAliasing Best
Object Accuracy 0.5
Shadow Accuracy 0.5
Sky Light True
Indirect Light True
Light Thru Transp. True
Lighting Quality Excellent
Accuracy 1
Irradiance One for all frames
Tile Size 16
Caveat: on the new machine, Carrara could not find the Lana Elite texture or bump maps for Victoria (so she had a zombie/undead pall to her...poor thing!) or the texture for Michael's jeans. I am not sure if this affected the render times.
Old Intel Machine
Start Time: 8:15:49 AM
End Time: 12:24:36 PM
Elapsed: 4:08:38
New AMD Server
Start Time: 10:14:21 AM
End Time: 1:49:40 PM
Elapsed: 3:35:10
Surprisingly close! I think that this shows Intel chips have better overall computational strength. However, bang-for-buck, AMD gets my vote...I simply wasn't willing to spend the $$$ needed for a 2-socket hex-core Intel machine.
If anyone thinks I did something wrong here, I'm happy to re-test with further suggestions (at least until I pave over the Intel box...I'm thinking of getting rid of its SSDs as it keeps blue-screening on me).
so you gain 30 mins. - boy I don't know - don't seem right to me
Yeah, not intuitive...I am guessing something is amiss. Perhaps some power setting (I did set Windows to High Performance though)...maybe something I missed in the BIOS. This is the first AMD machine I've ever owned.
Hi, I'm not sure about the total render time but try measuring from when Carrara actually starts dsplaying the image, it may be interesting.
I think some things are still single threaded sometimes. (Could be wrong).
And did you have any verification that all 24 cores were being used to render? I could be wrong but thought I read somewhere a couple years ago that Carrara was limited to perhaps 8 or 12 cores (on one machine, not across multiple PCs as a render farm). This may need some investigation. May be why no more than 8 tile squares of render show at a time?
I seem to remember reading about an 8 thread limit somewhere as well, but I've been looking unsuccessfully for a reference before posting.
Kendall
I was watching the render progress frequently. There was most certainly 24 distinct "render tiles" at work (remember I used a size of 16x16...and Carrara only puts numbers in them when they are 48x48)..
Does task manager show all 24 cores running at 100%.??
I have noticed on some scenes with IDL / GI enabled Carrara will crawl along and use only about 25-35% of your cores, yet if you disable IDL /GI on the same scene, it shoots back up to 100%.
I have two Intel 3930K's networked and over clocked to 4.2ghz, and there is nothing more frutstrating than seeing Carrara not using all of my rendering cores.
Peace,
SK.
Honestly, I didn't watch Task Manager. I am quite sure that I do not have IDL / GI in that scene though.
In your scene settings you state.
Sky light - true
IDL - true
Lighting Quality - excellent..
Those settings would indicate that you DO have IDL / GI enabled for your scene..unless I am mis-undertsanding your nomenclature.
Peace,
SK.