My First home lab: 2023 HP z440 Edition

Here is the mostly finished product:

How it all started…

I have wanted to build a homelab for quite some time, but it has been something I have never really made the time to do. For most of my career, I could work with tons of enterprise-grade equipment directly, and for all intents and purposes, I had my own dedicated lab in my company’s datacenter. So overall, I didn’t really need to build my own.

Fast forward to 2023, and I have been working for VMware for approximately a year and a half. Sure, I have access to many excellent dedicated labs, on-demand physical labs, and various nested labs through vCloud Director, but it’s not the same as managing my own gear… Sometimes I just want to hug a server 😂

The Host Build

So earlier this year, I started exploring host options that I could get for a decent price and landed on the HP z440. My main reason for selecting this build was that it could support up to 256 GB of ram and was a desktop format. My original theory was that I would just run a single HP z440…. fast forward to today, and I now have seven HP z440s in my homelab 😎

HP z440 BOM:

• HP z440 Base (700w PSU, 14c/28t – Intel Xeon 2690v4 proc, usually memory fan included)
• 256 gb ( 8 * 32gb DDR4 PC4-2400T ) – Shoot for less than $30 per dimm
• HP NC560 SFP+ Dual Port 10 gb NIC
• 3.2 TB NVMe Dell Branded: Samsung SM1715 U.2 form factor – vSAN ESA
• U.2 to PCIe Adapter
• 240gb T-Force Vulcan Z Sata drive – ESXi Boot
• Radeon HD7470 1GB Graphics Card – cheap option for display

Total Approx Host Cost: $700-800ish – depending on the deals you can get on memory/eBay offers, etc.

Parts depicted below:

Fully assembled HP z440 build:

Some critical observations on the HP z440:

• If you can, ensure you get a unit with the memory fan included. Any large memory configurations will cause the system to pause on boot if you don’t have the fan installed. You can manually override it and get it to continue to boot, but it can be a pain to deal with when performing cluster operations, etc. Buying the fans individually usually run 40-60 bucks on eBay, and considering the total price of the unit, not a great situation for the overall cost of the unit.
• It does not come with integrated graphics support. So you will need to install a cheap graphics card. It is worth noting that it does have an OG PCI port, so if you want to find an old PCI VGA card, you can keep all of the PCI-Express slots open for important cards, etc.
• These units either ship with a 700W PSU or a 525W PSU — its in your best interest to get one with a 700W model
• The gaming community picked up these units as a potential cheap gaming PC build – so there are many different listings on eBay with various configurations – figure out the details of what unit you are going after.
• Fully populating unit with 256 GB of ram will run you 200 to 300 USD, depending on how good of a deal you can get. If you don’t need that much, going with 16 GB DIMMs and maxing out at 128 GB is a bit cheaper.
• The HP z440 is built like an absolute tank – It has been quite some time since I have handled a case that is so sturdy – truly a pleasure to work with.

Networking:

From a networking perspective, I decided to keep it simple and continue to stay within the Ubiquiti ecosystem. I had wanted to upgrade my overall network gear for quite some time, and the homelab buildout seemed like a perfect time to pull the trigger.

Networking BOM:

• UDM-PRO
• USW-Aggregation
• TES Smart 8 Port KVM switch
• USW-24 – not in use, will move for my network drops elsewhere

First and foremost – I want to call out the USW-Agg switch — at $269, it’s quite a deal to get 8 ports of 10 Gb SFP+ connectivity. I was considering going a 10GB copper route but found the switching and cards to be more expensive. I was lucky in the fact that the USW-Agg, the Unifi DAC cables, the HP (intel 520?) 10GB NIC card, and the default ESXi 8.0 install all worked together out of the box.

The Uni-Fi gear was super simple to set up and looks amazing 🤩 Creating VLANs, configuring jumbo frames, etc., are really easy to do.

Here’s a view of the network stack before the 7th host was added.

See some of the cable management (before the 7th host as well):

I am also running two pfSense boxes – a physical install ( strapped to the side of the network stack) and a VM to serve as both DNS resolvers and NTP services for my gear.

Hypervisor: VMware, of course, 😁

Now that I covered the base, let’s dive into some of the details of the hypervisor setup.

• I initially used the ESXi 8.0 default build with these hosts and had no issues
• I was able to upgrade to 8.0.1 without issues as well.
• I manually bypassed the CPU warning since the Xeon 2690v4 is not on the official CPU-supported list.
• All the gear listed in BOM above worked without additional drivers or VIBs using the vanilla ESXi installer.

Disclaimer: This stack is not certified, or supported by VMware. I am using gear that is not certified and not supported. Don’t run this stack in a real environment. This is simply for my homelab purposes.

A view from my Center:

Total Resource Stats:

• CPU: 98 physical cores, 196 threads, ~ 254 GHz
• Memory: 1.75 TB of RAM
• Storage: ~22TB of NVMe storage for vSAN ESA

vSAN ESA:

I was lucky enough to take advantage of the new vSAN ESA architecture – this saved me a considerable amount of money as I didn’t have to worry about purchasing cache drives in addition to my storage drives. It also simplifies the overall stack.

Quick performance view of the cluster –

I need to really stress test it in the future – I suspect my 10gb networking will be the bottleneck.

If you made it this far- thanks for reading! But I have very much enjoyed this homelab build and look forward to the many projects I can run with this hardware in the future!

👀 Keep your eyes peeled for my future home lab content 👀

Check out the power consumption of this blog here:

HP z440 HomeLab: Power Consumption Update