MicroATX Home Server Build– Part 4

After a longish break, here’s the next installment! So the server has been in production now since last September, and is running very well. After the previous post, this is what’s happened:

  • Installed ESXi 6.0 update 1 + some post u1 patches
  • Installed three VMs: Openbsd 5.8 PF router/firewall machine, Windows Server 2016 Technical Preview to run Veeam 9 on and an Ubuntu PXE server to test out PXE deployment
  • Added a 4 port gigabit NIC that I got second hand

In this post, I’ll be writing mostly about ESXi 6.0 and how I’ve configured various things in there.

For the hypervisor, I bought a super small USB memory, specifically a Verbatim Store n’ Stay (I believe this is the model name) 8GB, which looks like a small Bluetooth dongle. It’s about as small as they get. Here’s a picture of it plugged in:

The Verbatim Store N Go plugged in
The Verbatim Store N Go plugged in

Using another USB stick created with Rufus, which had the ESXi 6u1 installation media on it, I installed ESXi on the Verbatim. Nothing worth mentioning here. Post-installation, I turned on ESXi Shell and SSH, because I like having that local console and SSH access for multiple reasons, one of them I’ll get to shortly (hint: it’s about updating).

Since I didn’t want to use the Realtek NIC on the motherboard to do anything, I used one of the ports on the 4 port card for the VMkernel management port. One of the ports I configured as internal and one as external. The external port is hooked up straight to my cable modem, and it will be passed through straight to the OpenBSD virtual machine, so it can get an address from the service provider. The cable modem is configured as a bridge.

The basic network connections therefore look like this:

Simple graph of my home network
Simple graph of my home network

After the installation, multiple ESXi patches have been released. Those can be found under my.vmware.com, using this link: https://my.vmware.com/group/vmware/patch#search. Patches for ESXi can be installed in two ways: either through vCenter  Update Manager (VUM) or by hand over ssh/local esxi shell. Since I will not be running vCenter Server, VUM is out of the question. Installing patches manually requires you to have a datastore on the ESXi server where you can store the patch while you are installing. The files are .zip files (you don’t decompress them before installation), and are usually a few hundred megabytes in size.

To install a patch, I uploaded the zip file to my datastore (in this case the 2TB internal SATA drive) and through SSH logged on to the host. From there, you just run: esxcli software vib install -d /vmfs/volumes/volumename/patchname.zip

Patches most often require reboots so prepare for one, but you don’t have to do it right away.

Update 2 installed on a standalone ESXi host through SSH
Update 2 installed on a standalone ESXi host through SSH

Edit: As I’m writing this, I noticed Update 2 has been released. I’ll have to install that shortly..  Here’s the KB for Update 2 http://kb.vmware.com/kb/2142184

A one-host environment is hardly a configuration challenge, but some of the stuff that I’ve set up includes:

  • Don’t display a warning about SSH being on (this is under Configuration -> Advanced Settings -> UserVars -> UserVars.SuppressShellWarning “1”)
  • Set hostnames, DNS, etc. under Configuration -> DNS and Routing (also made sure that the ESXi host has a proper dns A record and PTR, too; things just work better this way)
  • Set NTP server to something proper under Configuration -> Time Configuration

For the network, nothing complicated was done as mentioned earlier. The management interface is on vmnic0, vswitch 0. It has a vmkernel port which has the management ip address. You can easily share management and virtual machine networking if you want to, though that’s not a best practice. In that scenario, you would create a port group under the same vswitch, and call it something like Virtual Machine port group for instance. That port group doesn’t get an IP, it’s just a network location you can refer to when you are assigning networking for your VMs. What ever settings are on the physical port / vswitch / port group apply to VMs that have been assigned to that port group.

By the way, after the install of Update 2, I noticed something cool on the ESXi host web page:

VMware Host..client?

Hold on, this looks very familiar to the vSphere web client which has been available for vCenter since 5.1?

Very familiar!
Very familiar!

Very familiar in fact! This looks awesome! Looks like yet another piece that VMware needs to kill of the vSphere Client. Not sure I’m ready to give it up just yet, but the lack of a tool to configure a stand-alone host was one of the key pieces missing so far.

Host web client after login
Host web client after login

In the next  post I will be looking at my VMs and how I use them in my environment.

Relevant links:

The Host UI web client was previously a Fling, something you could install but that wasn’t released with ESXi https://labs.vmware.com/flings/esxi-embedded-host-client
But now it’s official: http://pubs.vmware.com/Release_Notes/en/vsphere/60/vsphere-esxi-60u2-release-notes.html

MicroATX Home Server Build – Part 1

Today I officially started my new home server build by ordering a case. The requirements for building a new home server are the following:

  • It needs to be physically small
  • It needs to be able to operate quietly
  • It needs to utilize some current hardware to reduce cost
  • It needs to be able to run VMware ESXi 6
  • Needs to support 32GB RAM for future requirements
  • Needs to accommodate or contain at least 2 Intel Gigabit NICs

Having run a number of machine at home in the past three decades, some of these have become more or less must-haves. Others are more of a nice-to-have. I’ve had some real server hardware running at home, but most of the hand-me-down stuff has been large, powerhungry and/or loud to the point where running it has been a less than pleasurable experience.

The last candidate was an HP Proliant 350 G5 (or so?), which was otherwise nice, but too loud.

You will note that power isn’t a requirement. I don’t care, really. My monthly power bills for a 2.5 person household of 100 m^2 is in the neighborhood of a few dozen euros. I really don’t know, or care. I’m finally at a position where I can pick one expense that I don’t have to look at so closely. For me, that expense is power. Case closed.

The conditions I’ve set forth rule out using a classic desktop machine cum server thing. Those are usually not quiet, they use weird form factors for the motherboard, seldom support large amounts of RAM etc. etc. A proper modern server can be very quiet, and quite scalable as most readers will know. A new 3rd or 4th generation Xeon machine in the 2U or Tower form factor can be nigh silent when running at lower loads, and support hundreds of gigabytes of RAM. They are, however, outside my price range, and do not observe the “Needs to utilize some current hardware to reduce cost”-condition.

Astute readers will also pipe up with, “Hey, this will probably mean you won’t use ECC memory! That’s bad!”. And I’ll agree! However, ECC is not a top priority for me, as I am not running data or time sensitive applications on this machine. Data will reside elsewhere, and be backuped to yet another “elsewhere”, so even if there is a crash, with loss of data (which is still unlikely, even *with* non-ECC memory), I’ll just roll back a day or so, not losing much of anything. A motherboard supporting ECC would be nice, but definitely not a requirement.

Ruling out classic desktop workstations and expensive server builds I am left with two choices:

  1. Get a standard mATX case + motherboard
  2. Get a server grade mATX motherboard and some suitable case

The case would probably end up being around the same choice, as the only criteria is that it is small, and can accommodate fans that are quiet (meaning non-small fans). The motherboard presents a bigger question, and is one that I have yet to solve.

I could either go with a Supermicro, setting me back between 200-400 €, and get a nice server grade board, possibly with an integrated intel nic, out of band management etc., or I could go with a desktop motherboard that just happens to support 32GB of memory. There are such motherboards around for less than 100€ (For instance, Intel B85 chipset motherboards from many vendors).

Here’s the tricky part: I could utilize my current i5-2500 (Socket LGA1155) in this build, and associated memory. This would mean that the motherboard would obviously need to support that socket. Note! The 1155 socket is not the current Intel socket. We’re now at generation 6 (Skylake), which uses an altogether different socket (Socket 1151), which is not compatible with generations 2&3 (which used 1155), generation 4&5 (which used 1150).

Using my current processor would save some money. Granted, I’d have to upgrade the machine currently running that processor (meaning a motherboard, cpu and memory upgrade, probably to Haswell or Broadwell, i.e. Socket 1150), meaning the cost would be transferred there. But then again, I tend to run the most modern hardware on my main workstation, as it’s the one I use as my daily driver. The server has usually been re-purposed older hardware.

Case selection

I’ve basically decided on the form factor, which will be micro ATX (or mATX or µATX or whatever), so I can go ahead an buy a case. Out of some options, I picked something that is fairly spacey inside, and somewhat pretty on the outside, which doesn’t cost over 100€. The choice I ended up with was the Bitfenix Prodigy mATX Black.

Here’s the case, picture from Bitfenix (all rights belong to them etc.):


Some features include:

  • mATX or mITX form factor
  • 2 internal 3.5″ slots
  • Suitable for a standard PS2 standard ATX PSU (which I happen to have lying around)
  • Not garish or ugly by my standards

I ordered the case today from CDON, who had it for 78,95€ + shipping (which was 4,90€). Delivery will happen in the next few days.

The current working idea is to get an mATX motherboard which supports my i5-2500 and 32GB of DDR3 memory. I’ve been looking at some boards from Gigabyte, Asrock and MSI. MSI is pretty much out, just because I’ve had a lot of bad experience with their kit in the past. May be totally unjustified, but that’s the way it feels right now.

I haven’t still ruled out getting a Supermicro board, something like this one: http://www.supermicro.nl/products/motherboard/Xeon/C202_C204/X9SCM-F.cfm but that would rule out using my current CPU and memory. I’d have to get a new CPU, which, looking at the spec, would either be a Xeon E3 or a 2nd or 3rd generation i3 (as i5’s and i7’s are for some reason not supported). i3 would probably do well, but I would take a substantial CPU performance hit going from Xeon or i5 down to i3. I’d lose 2 cores at least, which are nice to have in a virtualized environment, such as this.

Getting the board would set me back about 250€ and the CPU, even if I got it used would probably be around 100€. Compare this against an 80-100€ desktop motherboard, use existing CPU, existing memory (maybe?). Then again, I’ll have to upgrade my main workstation if I steal the CPU from there. Oh well. More thinking is in order, me thinks.


Last minute edit:

The hardware I have at my disposal is as follows:

  • Intel NICs in the PCI form factor
  • Some quad-NIC thing, non intel, PCIe
  • Corsair ATX power supply
  • Various fans
  • If I cannibalize my main rig:
    • i5-2500
    • 16GB DDR3 memory (4x4GB)