Home Lab Xeon

The current home lab setup consists of an Intel Core i3-2100 with 16GB of DDR3, a USB drive for ESXi (on 6.5 right now) and a 3TB WD for the VMs. While the Intel i3 performs perfectly for my needs, I came across a Xeon E3-1220 (SR00F, Ivy Bridge), which should be even better!

For the specs, we have the following differences:

Model Intel Xeon E3-1220 Intel Core i3-2100
Released: Q2-2011 Q1-2011
Manufacturing process: 32nm 32nm
Price originally: 189-203 US dollars (more in euroland) 120 USD
Core count: 4 Cores 2 cores
Hyperthreading No Yes
Base Freq: 3.10 GHz 3.1 GHz
Turbo Freq: 3.40 GHz No
TDP: 80 W 65W
Max Memory: 32 GB ECC DDR3 32 GB Non-ECC DDR3
L1 Cache: 128 + 128 KB 64 + 64 KB
L2 Cache: 1 MB 512 KB
L3 Cache: 8 MB 3 MB

So we can see that the Xeon part is 4 core processor, without hyperthreading, so real cores as opposed to the i3’s threads. It’s more power hungry, which is to be expected, but can also Turbo at a higher frequency than the i3. Also, the Xeon has more cache, which is also to be expected with a server grade component.

A notable thing is that the Xeon, being a server part, does not include the GPU components, so I’ll have to add a GPU at least for the installation. I run the server headless anyway, but I want to see it POST at least. I think I’ll have to add a PCI card for this it has no PCI slots so, as I only have one PCIe slot (well there are some x1 slots but I have no such cards), and that’s used by the NIC. The motherboard is an Asrock H61M-DGS R2.0 which has one x16 slot and one x1 slot. Maybe I’ll do it all headless and hope it posts? Or take out the NIC for the installation?

Some yahoo also tried running an x16 card in an x1 slot here. Might try that but since I have to melt off one end of the x1 slot, probably not.

There are apparently some x1 graphics cards, but I don’t have one as I mentioned. An option could be the Zotac GeForce GT 710, which can be had for 60 euros as of this post.

Preparations

I went to the pharmacy to get some pure isopropyl alcohol. It wasn’t on the shelf, so I had to ask for it. I told the lady I need some isopropyl alcohol, as pure as possible. She looked at me funny and said they had some in stock. I told her I’m using it to clean electronics, so she wouldn’t suspect I’m some sort of cringey soon-to-be-blind  (not sure if you get blind from this stuff, but it can’t be good for you) wannabe alcoholic, to which she replied that she doesn’t know what i’ll do with it, or how it will work for that. She got the bottle, which is described as “100 ml Isopropyl Alcohol”. There is a mention of cleaning vinyl disks and tape recorder heads on the back, so I was vindicated. There’s no indication of purity on the bottle, but the manufacturer lists above 99.8% purity here. Doesn’t exactly match the bottle, but it’s close.

Why did I get isopropyl alcohol? Well, because people on the internet said it’s good for cleaning off residual thermal paste from processors and CPU coolers. With common sense 2.0, I can also deduce that anything with a high alcoholic content will evaporate, and not leave behind anything conductive to mess things up. Oh and it cost 6,30€ at the local pharmacy. It’s not listed on the website (or it says it’s no longer a part of their selection).

Let’s see how it performs. I’m using cotton swabs, but I suppose I could use a paper towel. If it leaves behind cotton pieces, I’ll switch to something else.

The Xeon originally had a passive CPU block and a bunch of loud, small case fans, but I will use the same cooler as for the i3.

Take out the i3 and the cooler. Clean the cooler off with the isopropyl:

Isopropyl worked wonders

Put in the E3, new thermal paste. I used some trusty Arctic Silver 5.

Termal paste added, note artistic pattern

Re-attach the cooler and we’re off to the races. I’ll note here that I hate the push through and turn type attachments of the stock Intel cooler. Oh well, it’ll work.

 

Powering on

Powering the thing on was the exciting part. Will there be blue smoke? Will it boot headless? Will it get stuck in some POST screen and require me to press a button to move on? Maybe even go into the BIOS to save settings for the new CPU?

Strangely enough, after a while, I started getting ping replies from ESXi meaning the box had booted.

There’s really nothing left to do. ESXi 6.5 recognizes the new CPU and VMs started booting shortly after.

Xeon E3 running on ESXi 6.5

MicroATX Home Server Build – Part 1

Today I officially started my new home server build by ordering a case. The requirements for building a new home server are the following:

  • It needs to be physically small
  • It needs to be able to operate quietly
  • It needs to utilize some current hardware to reduce cost
  • It needs to be able to run VMware ESXi 6
  • Needs to support 32GB RAM for future requirements
  • Needs to accommodate or contain at least 2 Intel Gigabit NICs

Having run a number of machine at home in the past three decades, some of these have become more or less must-haves. Others are more of a nice-to-have. I’ve had some real server hardware running at home, but most of the hand-me-down stuff has been large, powerhungry and/or loud to the point where running it has been a less than pleasurable experience.

The last candidate was an HP Proliant 350 G5 (or so?), which was otherwise nice, but too loud.

You will note that power isn’t a requirement. I don’t care, really. My monthly power bills for a 2.5 person household of 100 m^2 is in the neighborhood of a few dozen euros. I really don’t know, or care. I’m finally at a position where I can pick one expense that I don’t have to look at so closely. For me, that expense is power. Case closed.

The conditions I’ve set forth rule out using a classic desktop machine cum server thing. Those are usually not quiet, they use weird form factors for the motherboard, seldom support large amounts of RAM etc. etc. A proper modern server can be very quiet, and quite scalable as most readers will know. A new 3rd or 4th generation Xeon machine in the 2U or Tower form factor can be nigh silent when running at lower loads, and support hundreds of gigabytes of RAM. They are, however, outside my price range, and do not observe the “Needs to utilize some current hardware to reduce cost”-condition.

Astute readers will also pipe up with, “Hey, this will probably mean you won’t use ECC memory! That’s bad!”. And I’ll agree! However, ECC is not a top priority for me, as I am not running data or time sensitive applications on this machine. Data will reside elsewhere, and be backuped to yet another “elsewhere”, so even if there is a crash, with loss of data (which is still unlikely, even *with* non-ECC memory), I’ll just roll back a day or so, not losing much of anything. A motherboard supporting ECC would be nice, but definitely not a requirement.

Ruling out classic desktop workstations and expensive server builds I am left with two choices:

  1. Get a standard mATX case + motherboard
  2. Get a server grade mATX motherboard and some suitable case

The case would probably end up being around the same choice, as the only criteria is that it is small, and can accommodate fans that are quiet (meaning non-small fans). The motherboard presents a bigger question, and is one that I have yet to solve.

I could either go with a Supermicro, setting me back between 200-400 €, and get a nice server grade board, possibly with an integrated intel nic, out of band management etc., or I could go with a desktop motherboard that just happens to support 32GB of memory. There are such motherboards around for less than 100€ (For instance, Intel B85 chipset motherboards from many vendors).

Here’s the tricky part: I could utilize my current i5-2500 (Socket LGA1155) in this build, and associated memory. This would mean that the motherboard would obviously need to support that socket. Note! The 1155 socket is not the current Intel socket. We’re now at generation 6 (Skylake), which uses an altogether different socket (Socket 1151), which is not compatible with generations 2&3 (which used 1155), generation 4&5 (which used 1150).

Using my current processor would save some money. Granted, I’d have to upgrade the machine currently running that processor (meaning a motherboard, cpu and memory upgrade, probably to Haswell or Broadwell, i.e. Socket 1150), meaning the cost would be transferred there. But then again, I tend to run the most modern hardware on my main workstation, as it’s the one I use as my daily driver. The server has usually been re-purposed older hardware.

Case selection

I’ve basically decided on the form factor, which will be micro ATX (or mATX or µATX or whatever), so I can go ahead an buy a case. Out of some options, I picked something that is fairly spacey inside, and somewhat pretty on the outside, which doesn’t cost over 100€. The choice I ended up with was the Bitfenix Prodigy mATX Black.

Here’s the case, picture from Bitfenix (all rights belong to them etc.):

bitfenix_prodigy

Some features include:

  • mATX or mITX form factor
  • 2 internal 3.5″ slots
  • Suitable for a standard PS2 standard ATX PSU (which I happen to have lying around)
  • Not garish or ugly by my standards

I ordered the case today from CDON, who had it for 78,95€ + shipping (which was 4,90€). Delivery will happen in the next few days.

The current working idea is to get an mATX motherboard which supports my i5-2500 and 32GB of DDR3 memory. I’ve been looking at some boards from Gigabyte, Asrock and MSI. MSI is pretty much out, just because I’ve had a lot of bad experience with their kit in the past. May be totally unjustified, but that’s the way it feels right now.

I haven’t still ruled out getting a Supermicro board, something like this one: http://www.supermicro.nl/products/motherboard/Xeon/C202_C204/X9SCM-F.cfm but that would rule out using my current CPU and memory. I’d have to get a new CPU, which, looking at the spec, would either be a Xeon E3 or a 2nd or 3rd generation i3 (as i5’s and i7’s are for some reason not supported). i3 would probably do well, but I would take a substantial CPU performance hit going from Xeon or i5 down to i3. I’d lose 2 cores at least, which are nice to have in a virtualized environment, such as this.

Getting the board would set me back about 250€ and the CPU, even if I got it used would probably be around 100€. Compare this against an 80-100€ desktop motherboard, use existing CPU, existing memory (maybe?). Then again, I’ll have to upgrade my main workstation if I steal the CPU from there. Oh well. More thinking is in order, me thinks.

 

Last minute edit:

The hardware I have at my disposal is as follows:

  • Intel NICs in the PCI form factor
  • Some quad-NIC thing, non intel, PCIe
  • Corsair ATX power supply
  • Various fans
  • If I cannibalize my main rig:
    • i5-2500
    • 16GB DDR3 memory (4x4GB)