My Renderfarm v4.0

Analog-X64

Warning this is a long Post :)

I have finally finished setting up my renderfarm, re-built and ready for some testing. In this initial thread I'm going to list everything about the setup and will follow with further info as I do some testing.

There are some things I'm not going to cover since there is an excellent thread regarding renderfarm advice
started by boojumbunn here: http://www.daz3d.com/forums/discussion/40152/

Working in the I.T. industry I have the perk of getting my hands on recycled computers every few years. My original render farm consisted of Pentium Computers, Followed by Pentium II, Pentium 4 and now Core 2 duo machines.

Machines:
The first thing I did was to refurbish the machines, making sure to remove all dust from CPU, Heatsink, Fans, video cards etc, making sure all fans were in working order and replacing any bad hard drives. I could go further and apply new thermal paste to heatsink/CPU but at this time the machines are running at a good temperature so I wont bother unless it becomes necessary to do so.

Power:
I have a total of 8 machines and although I did the math, I wasn't 100% sure the amount of power needed, I added a 20 AMP circuit breaker and special 20 AMP outlet, to my electrical panel in the basement. I have a server grade power bar that plugs into the special 20 AMP outlet and has a nice display that shows me the amount of current being pulled.

With all 8 Machines on at full power, the unit is drawing 7.8 AMPS, just under 1 AMP per machine, so on the power side of things I'm covered, interesting as well with the machines powered off, I'm drawing 0.5 Amps, I'm guessing that comes from the Wake On Lan and NIC cards.

PC management:
Each machine has Windows 7 64-Bit installed with TightVNC, Carrara Render Node and Bryce Lightning installed.

TightVNC allows me to run these machines headless i.e. no Keyboard, Mouse, Monitor.

The BIOS on each machine has been configured to ignore keyboard errors on startup as well Wake On Lan is enabled, which will allow me to power on these machines remotely. There are various ways to utilize WOL, I went with Wake On Lan (http://www.nirsoft.net/utils/wake_on_lan.html) very easy to use, on install required.

For each machine I initially, let DHCP provide the IP address that way I can easily capture the machine in Wake On Lan software, once I changed the IP to static, it was easy to edit the entry in WOL.

I also configured BG Info (http://technet.microsoft.com/en-ca/sysinternals/bb897557.aspx) to run on startup on each machine, I'm going to attach a screenshot so you can see the kind of info that can be displayed. I kept mine simple, but certainly you can show more info.

This is optional and not necessary.

Firewall is turned off, you can certainly leave firewall on, but need to add extra rules to allow the render node to work properly, thats just a personal choice. I'm not going to use these machines on the internet they are on their own network for rendering.

For making things simple, I plan on adding Carrara Render Node and Bryce Lighting to the startup of each machine that way I just power each machine on and wont have to manually start those programs. One should not interfere with the other, I'll have to test to make sure.

I still have to sort out shutting down the systems with a script. I have some permission issues to resolve.

Network:
In my case I have a recycled CISCO Gigabit switch with 48 ports, configured in simple switch mode, again this is overkill and a you can get a $60 NetGear GS-108 (8-port gigabit switch) that will do the job.

I'm also using CAT6 cables to connect everything to the switch.

One of the test that I will eventually do is, connect everything to a 100 Mbit switch and see if it will make a difference in render time.

All machines are set with Static IP ending with a number matching Computer name so Render04 is 192.168.0.44 Render05 is 192.168.0.45 etc..

Initial Render Test:
I wanted to test functionality and not necessarily look at speeds, I did multiple renders and noticed that although all 8 Machines were available at the most only 4-6 Machines were actually working and rendering on the network.

This is the scene I rendered https://www.youtube.com/watch?v=xL0r22SmTyk which is particle based, oh and at 00:16 you will see a cube doing a jitter. Still I haven't figured out how to get rid of the jitter. I did open a separate thread on that a few weeks ago but no response from anyone.

I also played with Tile Size settings but didn't seem to make a difference on how many machines were being utilized.

Machine Specs: All - 64-Bit
Main Render Machine: Intel Core 2 Quad CPU Q8300 @ 2.50 GHz - 8 GB of RAM
Secondary Main Machine: AMD A8-3870 @ 3 GHz - 8 GB of RAM

6 x Intel Core 2 Duo CPU E6750 @ 2.66 GHz - 2 GB of RAM
2 x Intel Core 2 Duo CPU E6300 @ 1.86 GHz - 2 GB of RAM

Dartanbeck

Wow. Very cool... Very interesting!

Dream Cutter

Nice configuration. If you find it lacking , a place too look maybe the network, as even at 100mbs it may be a bottleneck and all your cows may not be getting fed. I render animation heavily but use E-Ons Vue mostly so its a bit different. In Vue in rendering 100+M poly scenes consuming 128mb ram on Dual 16 Core Opteron cows. I found having dual nics and a managed gigabit switch is essential. I monitored the traffic before and after and there results were impressive. Windows Pro will attempt to load share bandwidth across all nics on a net. The other factor is RAM for speed and of course tons is better. 2GB seems a bit lean. Remember before going whole hog, Windows Pro is needed for 32GB+ The other thing is the limits of the render cow app for Carrara. How does it have a maximum number of cows and/or cores? Vue's hyperview render manager does although I found you CAN run multiple cow clients on a PC if you change ports for the second instance and also it does work in virtualized instances if you have a mega server or want to use Amazon S3.

EDIT: You may find this PC pricing & configuration table helpful, its amazing http://www.logicalincrements.com/