On Mon, Sep 18, 2023 at 6:13 AM Frank Steinmetzger <war...@gmx.de> wrote: > > Am Mon, Sep 18, 2023 at 12:17:20AM -0500 schrieb Dale: > > […] > > The downside, only micro ATX and > > mini ITX mobo. This is a serious down vote here. > > Why is that bad? µATX comes with up to four PCIe slots. Even for ten drives, > you only need one SATA expander (with four or six on-board). Perhaps a fast > network card if one is needed, that makes two slots.
Tend to agree. The other factor here is that desktop-oriented CPUs tend to not have a large number of PCIe lanes free for expansion slots, especially if you want 1-2 NVMe slots. (You also have to watch out as the lanes for those can be shared with some of the expansion slots so you can't use both.) If you want to consider a 10GbE+ card I'd definitely get something with integrated graphics, because a NIC is going to need a 4-8x port most likely (maybe there are expensive ones that use later generations and fewer lanes). On most motherboards you may only get one slot with that kind of bandwidth. > Speaking of RAM; might I interest you in server-grade hardware? The reason > being that you can then use ECC memory, which is a nice perk for storage. That and way more PCIe lanes. That said, it seems super-expensive, both in terms of dollars, and power use. Is there any entry point into server-grade hardware that is reasonably priced, and which can idle at something reasonable (certainly under 50W)? > > I was hoping to turn > > my current rig into a NAS. The mobo and such parts. This won't be a > > option with this case. Otherwise, it gives ideas on what I'm looking > > for. And not. ;-) > > I was going to upgrade my 9 years old Haswell system at some point to a new > Ryzen build. Have been looking around for parts and configs for perhaps two > years now but I can’t decide (perhaps some remember previous ramblings about > that). The latest zen generation is VERY nice, but also pretty darn expensive. Going back to zen3 might get you more for the money, depending on how big you're scaling up. A big part of the cost of zen4 is the motherboard, so if you're building something very high end where the CPU+RAM dominates, then zen4 may be a better buy. If you just want a low-core system then you're paying a lot just to get started. RE NAS: I used to build big boxes with lots of drives on ZFS. These days I'm using distributed filesystems (I've migrated from MooseFS to Ceph, though both have their advantages). The advantage of distributed filesystems is that you can build them out of a bunch of cheap boxes, vs trying to find one box that you can cram a dozen hard drives into. They're just much easier to expand. Plus you get host-level redundancy. Ceph is better for HA - I can literally reboot every host in my network (one at a time) and all my essential services stay running. MooseFS performs much better at small scale on hard drives, but depends on a master node for the FOSS version, so if that goes down the cluster is down (the locking behavior also seems to have issues - I've had corruption issues with sqllite and such with it). When you start getting up to a dozen drives the cost of getting them to all work on a single host starts going up. You need big cases, expansion cards, etc. Then when something breaks you need to find a replacement quickly from a limited pool of options. If I lose a node on my Rook cluster I can just go to newegg and look at $150 used SFF PCs, then install the OS and join the cluster and edit a few lines of YAML and the disks are getting formatted... > ¹ There was once a time when ECC was supported by all boards and CPUs. But > then someone invented market segmentation to increase profits through > upselling. Yeah, zen1 used to support ECC on most motherboards. Then later motherboards dropped support. Definitely a case of market segmentation. This is part of why I like storage implementations that have more robustness built into the software. Granted, it is still only as good as your clients, but with distributed storage I really don't want to be paying for ECC on all of my nodes. If the client calculates a checksum and it remains independent of the data, then any RAM corruption should be detectable as a mismatch (that of course assumes the checksum is preserved and not re-calculated at any point). -- Rich
