Twenty core cluster from Xeon X3210's ... need advice!

[WillyWonka]

I'm getting ready to build a 20 core compute cluster from 5 kentsfields and I would appreciate XS input and advice ...

Application:
Design of a new device to assist failing hearts (#1 killer in the West)
Scientific computing i.e. FP intensive.
24x7x365 @ 100% load
Each of the 5 nodes will be headless: No GFX card, no hard drive. Just case PS, mobo, CPU, RAM, cooler etc.
Nodes will be linked with gigabit ethernet.

Goals:
Highest possible performance on a limited budget.
Overclocking is necessarry to improve the price-performance ratio but stability is more important than extreme overclocks.
I need "guaranteed" results so experimenting with untested configurations is not on the cards, I need something as close as possible to a sure thing.

Proposed configuration:
I have searched far and wide for reported quad core OC results but there isnt a hell of a lot. To date, this is what I've narrowed things down too but things are obviously still fluid!

OS: 64 bit linux
Mobo: P5B Deluxe £120 / Commando £140 / Abit AB9 Quad GT £115
I'm leaning towards the P5B because of its wide use.
CPU: Xeon X3210 8x multiplier £450
Ram: Corsair TWIN2X1024-6400C4 DDR2-800 i.e. 2x 512 MB sticks per box £100
PS: Silverstone ST-40F 400 W £40
Cooling: Air cooling. Tuniq tower 120 +- £35

Target performance:
FSB 375-400 = 3.0-3.2 GHZ
RAM 1:1 @ 400 MHZ.
Memory timings: Loose is fine e.g. 5-5-5-15
Voltages: As low as possible for stability and power consumption reasons.

Comments:
My biggest issue is memory ... I really don't know what to buy. For budget reasons I need to keep the memory cost low but obviously I need to hit my performance targets. Comments? Suggestions?

As regards the mobo: Does the vanilla P5B OC like the P5B deluxe? I could save quite a bit by going for the vanilla P5B but again I dont want to compromise my performance targets. Comments? Suggestions?

Thanks for the advice! One day your heart might need this product

[Hassan]

Why would you oc on something so serious... for a job like this I would be looking at Woodcrests or Clovers, server grade boards, ecc memory

[WillyWonka]

Why would you oc on something so serious... for a job like this I would be looking at Woodcrests or Clovers, server grade boards, ecc memory

Good question regarding doing any OCing at all. Answer: Limited budget + the code is inherently fault tolerant.

Due to the nature of the code parrallalisation, server class mobos arnt required (They're nice to have but not required). This class of codes is routinely run on multiple gigabit linked whitebox desktop PCs.

[Hassan]

I would swap the Silverstone PSUs for Seasonics, more efficient and run cooler, and swap the Corsair for Crucial dimms. For true stability, I would run intel or asus boards, crucial mem, and seasonic psus

[WillyWonka]

I would swap the Silverstone PSUs for Seasonics, more efficient and run cooler, and swap the Corsair for Crucial dimms. For true stability, I would run intel or asus boards, crucial mem, and seasonic psus

I've always found Silverstone to be good quality PSs but I'll look at the Seasonics.

As regards RAM any particular reason for preferring Crucial over Corsair? I'm having a look at these: http://www.dabs.com/productview.aspx...090,4294955840

[Hassan]

I prefer micron ic's to promos, with Crucial you will always get Micron, with Corsair who knows... I usually go with the 3-3-3 stuff with Crucial, and Seasonics S12's are very nice, check them out.

[DDTUNG]

You should be able to achieve your target FSB range with just about any PC6400 ram. Two things to consider:
1. Will the 2x512 ram be sufficient for your app?
2. OCed quadcores put a heavy drain on the 12V line so go with a PSU with the highest current rating on a single 12V line that you can afford.
3. The main advantage of the P5B Deluxe over the P5B is the 8 phase power design.

I'd be very interested in how your project turns out, as I run a 24/7/365 DC farm of close to 100 rigs.

Good luck.

DDTUNG

PS I can refer you to a reliable source who can probably get you X3210s for much cheaper than $900 each. PM me if interested.

[WillyWonka]

I prefer micron ic's to promos, with Crucial you will always get Micron, with Corsair who knows...

Good point. A good selection available I see: http://www.crucial.com/store/listpar...del=P5B+Deluxe

The 430 W Seasonic looks reasonable too: http://www.seasonic.com/product/pc_retail.jsp

[K404]

For such an application, ask for company support?

Theres cheap generic PC6400 out these days..timings and OC ability will suck but they`re guaranteed to do 400MHz so...

Good luck with the build

Oh...Tagan 480W PSUs have a single 12V rail of 28A...they might be cheap these days?

[WillyWonka]

You should be able to achieve your target FSB range with just about any PC6400 ram.

Thats a relief

1. Will the 2x512 ram be sufficient for your app?

I should have been more specific on the OS: Its 64 bit linux with no windowing environment ... just a plain shell. The code I use spreads the simulation seamlessly over the available resources in the cluster. So from the codes point of view it sees a 5 GB pool of RAM. For interest, the code is a CFD code which calculates the behaviour of fluids e.g.

2. OCed quadcores put a heavy drain on the 12V line so go with a PSU with the highest current rating on a single 12V line that you can afford.

How many amps do you think I would need?

3. The main advantage of the P5B Deluxe over the P5B is the 8 phase power design.

Does this have a real impact on overclockability?

I'd be very interested in how your project turns out, as I run a 24/7/365 DC farm of close to 100 rigs.

Interesting. I'll definately post on XS once the rig is up etc.

[WillyWonka]

Oh...Tagan 480W PSUs have a single 12V rail of 28A...they might be cheap these days?

Hmmm ... I see the Seasonic s12 430 W only does +- 15 A

Your Tagan looks good @ £57! http://www.hexus.net/content/item.php?item=694&page=6

Forgot to add: Thanks to everyone for all the input so far!

[DDTUNG]

Since I don't have a P5B I couldn't tell you the real impact of the 8 phase power on quadcore OCs. However I will venture to say that it probably gives better power regulation which will mean better stability for any system, OCed or otherwise.

How many As you need depends on your OC and Vcore. The Tagan looks good though.

DDTUNG

[dave_graham]

honestly, save yourself some more coin and get Kingston memory.

i'd heartily recommend Clovertown 5345s due to the:
a.) increased memory bandwidth
b.) ability to reduce server count to 2
c.) intel's legendary support for enterprise platforms

dave

[WillyWonka]

honestly, save yourself some more coin and get Kingston memory.

i'd heartily recommend Clovertown 5345s due to the:
a.) increased memory bandwidth
b.) ability to reduce server count to 2
c.) intel's legendary support for enterprise platforms

dave

Hmmm I had sort of premptively written off Clovertown rigs due to cost. But after some Goolging there is some method to your madness:

The E5345's are an extra £100 a CPU. Clovertown mobos are about £300 which is about equivalent to two 775 mobos.
Ram is pretty much the same: £100 / GB e.g. http://uk.insight.com/apps/productpr...050I0&src=FRO1
I'd loose about 20% speedwise due to the Clovers 2.33 GHZ clock and OC limitations.

So its an extra £100 per socket with a 20% speed penalty but no OCing and server level stability.

Hmmm ... I will have to consider this seriously. Of course, the April 22 40% Intel price cut would swing using the 775s ... assuming Intel doesnt chop the server line too. Considering the K8L is aimed squarely at servers an Intel server price chop seems reasonable. Too many factors to consider


PS: My code isnt very dependant on memory bandwidth.

[DDTUNG]

Hmmm I had sort of premptively written off Clovertown rigs due to cost. But after some Goolging there is some method to your madness:

The E5345's are an extra £100 a CPU. Clovertown mobos are about £300 which is about equivalent to two 775 mobos.
Ram is pretty much the same: £100 / GB e.g. http://uk.insight.com/apps/productpr...050I0&src=FRO1
I'd loose about 20% speedwise due to the Clovers 2.33 GHZ clock and OC limitations.

So its an extra £100 per socket with a 20% speed penalty but no OCing and server level stability.

Hmmm ... I will have to consider this seriously.


PS: My code isnt very dependant on memory bandwidth.

A few of our WCG team members are running dual Cloverton rigs OCed through a BSEL pin mod which changes the default FSB. Hassan, Movieman, and I forget who else.

DDTUNG

[WillyWonka]

A few of our WCG team members are running dual Cloverton rigs OCed through a BSEL pin mod which changes the default FSB. Hassan, Movieman, and I forget who else.

Yeah I'm aware of the (limited) OCs possible. That eliminates the speed penalty.

Too many choices ... damn ... It was easier when server and desktop computing were separated by a cavernous price gap

[Hassan]

Scott

[WillyWonka]

Hahaha ... just realised something. The Q2 K8Ls are also an option!

Choices => <= Choices

[Order]

Thats a hell of a project. I'll be looking forward to updates.

[dinos22]

also i would invest in a small aircon and partition the cluster in its own room with good air cooling and aircon controlled temps

use infinity coolers as the fins on them are a lot wider to other HSs and you will find that no matter what you do with small fins you will always have a dust buildup and temp deterioration. Good quality fans.

Mild OCs are the way to go for tight budgets and any half decent RAM will do 400MHz do don't be alarms but investigate whether RAM timings/latency affect performance of the cluster...if so go for Micron RAM that can run tight timings at 400MHz

i would even get a 975X board like Badaxe 2 to be honest and you can pickup some awesome deals on ECC RAM on ebay with PC4300 RAM and D9DCN ICs which will run 400MHz without any issue at all

Johnny Guru hold Seasonic in high esteem so i would go that way myself

i would even look at PSUs with redundant modules like some Zippys or Deltas....i use one for my benching computer with 3x350W modules....those PSUs are designed to run 24/7 at high load unlike the consumer based ones

[s7e9h3n]

Hahaha ... just realised something. The Q2 K8Ls are also an option!

Not an option......THE ONLY CHOICE BTW, they go retail channels in Q3...

[[XC] hipno650]

i would go for the kents and a low oc to keep things totally stable with a rock solid psu. so maybe only oc (200-600mhz) but wait for the price cits and you'll save allot of money. and good luck with the project!!!

[WillyWonka]

Thats a hell of a project. I'll be looking forward to updates.

This is just the trial run cluster. Once our VC funding gets finalised we have to go to 128-256 cores. Even on 20 cores the simulation will run for 10 days. In order to optimise the design in a practical time we need to get that down to 12 hours. At that point we will just hand over a RDQ to a big iron company.

However in the interim we are on our own funding and that is limited.

[dinos22]

Not an option......THE ONLY CHOICE BTW, they go retail channels in Q3...

lol why the ONLY choice in capital letters

[s7e9h3n]

lol why the ONLY choice in capital letters

Hehehe...you'll see....SOON...