Extend Your Server With The DELL C410X
In December 2010 I was at the Stanford HPC Conference where I took a glance at Certainty, it was there that I first heard of the Dell C410X Since then I've gathered as much information as possible on that device, and I thought to assemble it here.
So what is the DELL C410X and why is it so interesting for Systems Architecture?
The official webpage describes it as "[...]a 3U, external PCIe expansion chassis that can support 1-8 server connections to up to 16 GPU cards" in other words, a way to extend the hardware capabilities of your typical rack server via PCIe. The innovation in this device comes from the form factor to host up to 16 PCI-e cards within 3U, now that's rack density indeed...
The C410x was designed by Dell Data Center Solutions Architect Joe Sekel and so this gear is a true Data Center piece of hardware, N+1 power supplies, hot swappable modules, ipmi, etc. Check out these videos with Dell Data Center Solutions System Engineer Chris Petersen they speak for themselves.
You couple that beast with a C6100 (the only option currently available) and suddenly your servers have the same versatility in terms of PCI-E slots as your Alien Ware 1337 gaming machine.
A typical farm server blade usually comes with a max of 2 PCI-E slots but with this config each of the 4 servers in the C6100 have access to 4 PCI-E Slots, and that, dear reader, means Systems Architect like me have a new toy to play with, who hoo!
The first batch of "generic C410X" with empty PCI-E slots is supposed to be available in February 2011 and yes we have been speaking with DELL at my current job to start prototyping =D.
If you are really interested in the C410x I highly recommend to watch this technical explanation about the bandwidths and switching internal.
If you investigate the Titanium (Dell codename for the C410x) don't get confused by their "Nvidia Tesla only" PR that you may find here and there, they are definitively opening it to put other PCIe cards. There is a limitation though induced by the the original "Tesla only" design, it seems not all PCIe cards will fit in the generic version of the C410x depending on their form factors.
Not convinced yet? Imagine populating a C410x with ioDrive Duo cards from FusionIO or maybe a mix of HBAs, 10GE, 1GE, cards available to the same host for whatever your architectural needs are.
That PCIe versatility within that density is unprecedented in the Data Center space this 4 PCI-E slots to 1 Server ratio, with a density of up to 8 servers/16 PCI-E cards in 7U for the C410X/C6100 is going to make a nice weapon to add to the arsenal.
Personally since I'm currently working a lot with HD video signals I'm looking into populate the expansion chassis cards with Nvidia Quadro Digital Video Pipeline, but this PCI-e solution from Nvidia will get it's well deserved own post.
Categories
Tags
Tagnetic Poetry by Roy Tanck and Merel Zwart requires Flash Player 9 or better.
My Delicious
Blogs I Read
- All Commands
- Amazon Web Services Blog
- Cloud Scaling
- Danger Room @ Wired
- IT @ Slash Dot
- Linux @ SlashDot
- Mac Os X Hints
- Open Source @ Ars Technica
- Presentation Zen
- Robots.net
- Stay Safe Online
- Threat Level @ Wired
- Transnets (in French)
Books
Planned books:
None
Current books:
-
An Introduction to Information Theory: Symbols, Signals and Noise by John R. Pierce
-
RHCSA/RHCE Red Hat Linux Certification Study Guide (Exams EX200 by Michael Jang
Recent books:
- Cybernetics, Second Edition: or the Control and Communication in the Animal and the Machine by Norbert Wiener
- The Cuckoo’s Egg: Tracking a Spy Through the Maze of Computer Espionage by Clifford Stoll
- Apple Training Series: Xsan 2 Administration: A Guide to Designing, Deploying, and Maintaining Xsan by Robert Kite
- Snow Crash (Bantam Spectra Book) by Neal Stephenson
- The Code Book: The Science of Secrecy from Ancient Egypt to Quantum Cryptography by Simon Singh
April 26th, 2011 - 15:55
Greetings, Have you found a list of what will fit inside the “tacos” of the c410x? We just tested a geForce GTX 580 and it wont fit even with its grill and plastic cover off. Its about 1/4in too tall and the plastic swing gate of the taco that holds the video card in place won’t close on it.
April 26th, 2011 - 16:31
I don’t think there’s such a list yet and I don’t know if Dell will create it, so a it’s going to be trial and error for PCIe cards that do not have a standard form factor. When I spoke with Dell in Dec. it seemed that PCIe full length cards wouldn’t fit, but they weren’t able at that time to give me the definitive specs for the fitting PCIe form factors. What I’m pretty sure at this point -I’m waiting for my C410X Unit to ship- is that anything similar to the Tesla M2070 should be ok. The M2070 specs says “9.75” Pcie x16 form factor”
April 26th, 2011 - 16:34
I was trying to remember where did I see something more precise and I found it on the C410X product spec page it states: PCIe Form Factors
Half-height/half-length and full-height/half-length, up to 225W per module passively cooled
http://dell.to/9ecDay
April 28th, 2011 - 15:29
So here is some more info.
There is a general purpose sled part # 331-1709 that will not fit any full height cards (4.376in). There is a riser on the sled which puts the over the height of the sled. Also I do not see any power cables with the sled. So even if you buy a Tesla card from somewhere else, it will not fit on this sled.
Dell has another Sled that ships with the tesla card they sell. So its a M2070Q + Sled together. The sled is different and can have a full height card on it. Currently, even with some smooth talking, I couldn’t get Dell to sell me just the sled they are putting the Tesla cards in. Now even If I got my hands on just that sled, it might be very similar sled and the Tesla cards themselves are different that Dell sells, Im not going to know until the sleds with the tesla cards get here.
I did test a half height card (Quadroo 600) and it fit perfectly, but I would like more power than just a few quadro 600s.
I will hopefully remember to respond back when the Tesla sleds get in and ill see if there is a difference and ill try to fit a geforce 580 on the sled.
May 2nd, 2011 - 14:26
Did you have that #331-1709 listed in your original quote from DELL? I just looked in mine and that is not appearing. I’ll log updates using the comments of this post.
May 13th, 2011 - 16:43
The c410x doesnt come with any sleds, you have to order them separately. The 331-1709 are the universal sleds without GPUs. The sled with the GPU is 331-1708 (m2070Q GPU)
May 13th, 2011 - 15:33
so we just got the c401x in and the tesla m2070Q + sleds
First of all, this thing is a beauty.
Second, the sleds are different. They have power on board, and no riser as I expected. The tesla cards look like they are standard tesla cards. Basically the universal sled that Dell sells will not fit full height GPUs, you need to buy the tesla card + sled combo to get a full height GPU into the c410x
May 17th, 2011 - 19:44
We are starting to receive the different parts, first arrived are the “iPass” PCI-e cards and cables, interesting to see that these have NVidia stamps while shipped in Dell wrapping, unusual.
May 19th, 2011 - 16:23
Ya the HIC card and the cable is NVIDIA You can even get the exact one from NVIDIA site because they also have a GPU expander similar to the C410x but holds only 4 tesla cards and its designed for 1 server I believe.
Ours is going in the data center right now, we for some reason didnt have the proper power cables to handle the 208v on the c410x so had to get them shipped out.
July 4th, 2011 - 10:56
Jamie did you get your remoteFX working with this c410x? We aren’t seeing great results yet.
July 7th, 2011 - 12:26
I just received the C6100 and still have to order the empty sleds for the C410X, plus I have another project that has been “niced” and is now above in my priorities stack…I’m planning to get it running during the fall.
November 8th, 2011 - 16:25
Just wanted to post a comment about an interesting piece of hardware somehow similar to the C410x. IT’s called the GPU-Xpander and the maker is Cubix
http://www.cubix.com/products/gpu-xpander