Overview of my homelab:
My approach to homelabs was born from my love of gaming hardware. If I'm spending money on hardware, it better run everything I throw at it, including PC games, real good. This is typically not hard - a good CPU, max RAM and a good video card, with a power supply that can last for years and multiple expansions will get you far. This may be why I don't like Apple hardware, which typically gives you less hardware for the same dollar amount, with the excuse of "but the software is so good".
My last build was in 2011, and almost 7 years later, it's still running pretty well. Ever since I got it, I went max RAM. I have added a PCIe enterprise SSD (FusionIO doesn't show up in Windows, you will see it in the actual pics) and changed SSDs as they have come down in price.
32GBs are overkill for gaming but for VMware homelabs they were adequate in 2011 and they are the bare minimum today - RAM is the main limiting factor when running VMs. 32GB is what you can get from an Intel NUC, one of the current homelab darlings, with a normal desktop's noise levels. As you can see I have it connected to a TV, using a microsoft wireless mouse/keyboard.
M.2 SSDs
The M.2 form factor works for both SATA and NVMe SSDs. The important thing is the key, which can B (sata) or M (PCIe, used with NVMe). It's wikipedia article is a good read. M.2 SSDs are becoming more popular since they are smaller than a 2.5 drive and NVMe is much faster than SATA; but using up a PCIe slot for one storage device can be expensive and consumer/server boards normally don't have that many PCIe slots. Some boards are putting M.2 slots directly on the motherboard, both in lay-flat and vertical positions. I've seen server boards that treat them like DIMMs.
Most M.2 you would consider come in the 2280 size, but enterprise SSDs will most likely come in the 22110 size.
Selecting a M.2 PCIe adapter card
I'm a big fan of Paul Braren's TinkerTry.com site. I've called him "the God of homelabs" and I mean that - he has spent considerable time blogging in the prosumer segment of homelabs, investigating and publishing particularly useful tips for VMware homelab enthusiasts. I'm a big proponent of his SuperMicro bundles - I especially like that he always thinks of them in multiboot scenarios so they can perform more than just homelab functions. SuperMicro offers much better specs than any gaming rig you can build, since they are server boards, allowing for 128GB of RAM and on-board 10G NICs. VMware recently became his employer, which is great news for everybody, and bodes for some good VSAN content, which is known for a difficult HCL for homelabs. Don't miss out on his SuperMicro vs NUC comparison chart.
Paul did an excellent article showing PCIe adapter cards for NVMe M.2 SSDs. Ideally, if you are going to put an adapter card, you would use a x16 slot and put 4 NVME SSDs on it; using up a slot for just one drive is ok until you need the slot, so it's better to plan ahead. However, the reviewed card that can do multiple NVMe SSDs is really expensive right now, something I can't explain since in theory it's only routing the PCIe lanes and providing power. It does mark all checkboxes though - supports 22110 drives and 4 M.2 NVMe SSDs at the same time; it's the only card we've all found that can do this.
There's a cheap 4 drive card but this only works for mSATA SSDs, which are not NVMe - this is the card featured on the AzureStack mini server, the Addonics AD4MSPX2-A. If you already have mSATA SSDs this is the card to get.
SilverStone Dual M.2 (NVMe & SATA)
While checking around for a multiple NVME card I found this card
This is the SilverStone Technology Dual M2 to PCI-E X4 and SATA 6G Adapter Card (ECM20). I liked it because
1) has two 2280 m.2 slots, one in SATA and another in NVMe, able to run both M.2 simultaneously,
2) it's cheap, $18 last time I checked
3) it's just an adapter, no drivers or interfering silicon, using a x4 PCIe slot
This gave me the sweet spot that I was looking for, multiple M.2 slots while being cheap and simple. While I won't be able to run 22110 M.2 SSDs, I most likely won't be able to afford them anyways ;)
While waiting on a good M.2 NVMe drive to drop down in price I decided to get a cheap SATA B key M.2 SSD. I found a $100 512GB refurbished Samsung 850 EVO, which is a very good drive for cheap, and ordered it, obviously a little wary.
Good value SSD options I can recommend today:
NVME: the Samsung are probably the best performing but I can't justify the cost difference, my choice would be Intel 600p
Once everything arrived I took some pictures to share:
2280 is pretty small!
Notice the max size is 2280 for the M.2 SSD
I only had either x1 or x16- used the last free PCIE X16 slot
Notice the card holds and powers the SATA SSD only, you still need to connect a SATA cable to it
At least the SMART data for the drive shows it healthy, and it has been fine so far
With this, once I find a good value on a 1TB NVMe SSD I can add it to the same card and get another year or so of use from this motherboard. This is a card I'm pretty sure I can move to my next build, hopefully on a x4 slot. The next build will very likely use NVMe only, and will most probably have slots for 3DXP/Optane; such a build will take at least another year to reach my budget projections.
Feel free to ask me questions on this build on twitter.
