Introduction
After 10 years of service, and several rounds of upgrades, my old Nehalem based server is running into its limits. With 6x 3T hard drives, 3 graphic cards, 32G RAM (double the 'max supported'), and a quad core i7 @ 2.8 Ghz, it is more or less maxed out. I'm running mostly into performance issues w.r.t. CPU and RAM. Yes, the disks are slow too, and adding an SSD might just squeeze out another 2 years of use, but it would mean buying components that I probably won't be using after an upgrade, and would not solve my main problems, being CPU and RAM maxed out.
Selecting the components
Since I find myself running virtual machines more and more, I am craving for more RAM, as well as CPU power. To allow for sufficient potential growth in the future, I would need a CPU that has at least 8 cores, and starting with at least 64G RAM, with upgrade path to 256G in the future. Having more than 6 SATA ports, and M.2 SSD would be nice.
In short, I have some requirements and wishes:
| Required | Nice to have |
|---|---|
|
|
Case: Fractal Design Define 7 XL
Initially I had planned to re-use the existing case, but then seeing how cramped it was with only 6 disks, and a big (Noctua NH-D14) cooler, I decided to reconsider. For some time I've been keeping an eye on Fractal Design cases, I really liked the node 304 (good for building a NAS), and also the Define 5 and 6. So when they came with a Define 7 and Define 7 XL, I was very interested. The XL version offers quite a bit more room, handy if you want to cater for extra large mainboards, water-cooling, and/or many hdd drives. Since I already have 6 disks to start out with, and expect to add at least 2 more, and want to leave room for expansion in the future, the choice fell on the Define 7 XL.
Features:
- Can be configured to hold up to 18 3.5" hdd drives
- Room for 360mm radiator in front or 420mm on top
- Can hold ATX, E-ATX, EE-ATX, SSI-CEB and SSI-EEB mainboards
- Has 9 + 3 slot brackets
- Sidepanels can be easily removed
- Dustfilter on the bottom
PSU: Fractal Design ION+ 860
Not much to say here, chose a Fractal Design PSU because that will likely work well with the Case, 860W is the max they provide, otherwise I might have chosen a bigger PSU. Should be sufficient though for now, and of course it can be easily replaced later.
Mainboard: ASRock Rack X399D8A-2T
Ever since the first Threadripper CPU's were introduced, I've been keeping an eye out for them, so much performance, and good value for money. Also the Threadripper CPU's have way more (64 total) PCIe lanes when compared to any other CPU in the same class. And for some reason, I always seem to be short of PCIe lanes, so this seemed like a great platform. Just when I was about to build this machine, PCIe 4.0 and TRX40 chipsets were announced, and for a while I was tempted to wait for those to come to market. However seeing the first mainboards appear with even less PCIe slots than the previous generation, it kind of killed my appetite, and I found the 2nd generation more interesting. At one point my eye fell onto the ASRock Rack X399D8A-2T mainboard, albeit not a 2nd gen but rather 1st gen Threadripper mainboard. Despite that, it can take a 2nd gen CPU, is a server class mainboard with IPMI (meaning: integrated ASpeed 2500 Graphics and remote console over ethernet). It also features dual M.2 sockets, as well as 5 PCIe x16 plus 1 PCIe x4 slots. It can hold up to 256G of RAM, and has 10 Sata ports, and even provides dual 10G ethernet ports. All in all, it surpasses my minimum requirements, and is close to my 'nice to have' criteria.
CPU: AMD Ryzen Threadripper 2970WX
Since I went for a 1st/2nd gen compatible mainboard (see above), this means that it may be hard to find a CPU upgrade in the future, hence I am probably best of going for a 16+ core thread ripper right from the start. I considered a 1950 for a while, but then noticed major price drops on the 2nd gen series, especially on the 2970WX, which features a good 24 Cores! When I set out to buy a CPU, I had not expected that this model would be affordable within my budget, but now with recent price cuts, it is a very interesting purchase, so I decided to go for it. Have looked at the 2990 as well, but that one seems to be still in high demand, and prices are still quite high on those.
CPU Cooler: Fractal Design Celcius S36 Black
Although I was very happy with the cooling performance of the Noctua NH-D14 that i used on my previous machine, it was not going to fit a threadripper CPU, so I need something new. The Noctua did have one drawback, which is that due to its size, it severely hindered accessibility to the mainboard. Since the case has plenty of room, I decided to go for a AIO watercooler solution. The Celsius was an obvious choice, since it comes with the same fans that the case has, and thus keeps things nicely consistent. Reviews seem to indicate that this is a top 3 model, so cooling performance should be fine. Wanted to place this in the front, however the hoses were too short for that (cables or hoses too short is something of an issue with a large case like this). Luckily it fits fine on top.
RAM: Trident Z RGB, 3200Mhz, 64GB
Having run into memory issues with my 32G server, I really wanted to go for 64G right from the start, since that would bring quite some relief for my virtual machines. I also planned to replace a few VMs with docker containers, so that could further reduce memory demand. Initially I chose a different type of memory, but that suddenly went out of stock. The G.Skill memory was similar in spec, so went for this. Nice addition was the RGB feature, which I did not care about at first, but seems to look very nice through the dark tinted side window of my case.
Graphics: ASpeed AST2500
This is the onboard IPMI graphics card, which i use as console for my VM host. Very convenient, since it provides a console over the network.
Graphics: 3x NVidia Quadro K420
Why such an out of date graphics card, and 3 of those? Why not a single more modern card? Well, to be honest, these were working fine in my previous system, and i don't care much for 3D graphics performance. My main purpose for these cards is to give my VMs a proper Discrete GPU so that their UI is properly accelerated. The few games I have on steam seem to be ok with this card, and until that changes, I see no need to upgrade. Windows 10, Ubuntu 20.04 and Macos Catalina all seem to get along with this card just fine. Having 3 of these means they can all run in parallel, and I can even hook up 3 monitors, keyboards and mice, and have 3 full blown terminals :-)
Graphics: Sapphire Pulse Radeon RX570 ITX
Since i recently upgraded to dual 4k screens, and have found a few nice games on steam that would like to run on a more advanced graphics card than the K420 mentioned above, I decided to purchase a more advanced graphics card and put it in the last slot, it's a pcie 3 x16 slot, which i reserved especially in case i might want to add a more powerfull graphics card. After carefull consideration, I went with the Saphire Pulse RX 570, according to reviews, this is one of the few AMD cards that is:
- is not troubled by the Reset bug (a lot of AMD cards are troubled by this, which is causing issues when rebooting VMs) and
- will work out of the box with macOS Catalina (as well as Big Sur and Monterey) and
- has enough performance to work well with the games i want to run
USB3: Sonnet Allegro Pro USB 3
This card has 4 separate USB3 root ports (4 chips), which makes it very handy to pass through a USB host controller to a VM, and connect high performance USB3 devices. Now it is easy to give each VM its own USB host controller, which can be combined with the 3 GPU's.
RAID Controller: LSI SAS 9211-8i
Since I would like to run a NAS in a separate VM, I decided to add an extra controller card. This one has great support for OpenMediaVault, but also for FreeNAS and other dedicated NAS systems.
SSD: 2x Samsung 970 EVO Plus 1TB M.2
Samsung still has the best NVMe SSD's currently on the market. Great performance does come at a price, but value for money on these is still great.
HDD: 2x Western Digital WD60EFAX-68SHWN0
Because my 8T Seagate seemed to be getting many errors after getting a bit hot this summer, I had to replace it. Decided to go with two new 6T drives, so that I can run them as Raid5/Raid1 in combination with two of my other 3T drives in the NAS VM. My other 4x 3T drives will be configured as a Raid 10 on the host to serve as storage for my VM disk images. I can use some of the SSD space to cache these images, that should improve performance dramatically.
HDD: 2x Western Digital WD60EFAX-68JH4N1
A recent drive failure for one of my old 3T drives prompted me to buy a new pair of drives. Since i liked the WD60EFAX, i bought another two of those, albeit a different version.
Putting it together
Comments