Hi, I wrote a private email to a friend. The content may be of interest to some on the list, so here it is.
Do you know much about NVMe over PCIe for SSDs? I don't, only having bought SATA SSDs in the past. The Pi 5 has a ×1 PCIe connector, meaning one channel of two twisted pairs to allow a full-duplex connection. A ×4 PCIe has four of them allowing data to be sent in parallel, 4 b at a time. Raspberry Pi have recently released their PCIe HAT+ for plugging in an M.2 NVMe SSD, though only in one of two short form factors. https://www.raspberrypi.com/news/m-2-hat-on-sale-now-for-12/ That's handy. For $12 one can plug in large reliable storage and boot from it. Crucial, really Micron, don't make M.2 SSDs that short, but Samsung do so I'll probably plump for them as a second preference. But it's only one drive. What would be really nice is two in a RAID 1 mirror. The two peers sat at either end of the PCIe can negotiate the ×𝑛 channels downwards. This is how a ×4 can plug into a ×16. Some controllers can also support ‘bifurcation’, splitting a ×16, say, into a ×8 and two ×4 so a passive adapter card can plug into the ×16 and provide three connectors. This also needs BIOS or OS support. (Bifurcation seems the wrong word but perhaps it's split by a multiple of two each time.) (Sorry if you know all this. By putting it in writing, I'm having to think it through for myself.) Anyway, the Pi 5's ×1 can't be split. So a passive adapter, one which is effectively just wiring, wouldn't work. My slight understanding is an active adapter, meaning it has a PCIe controller, is required to bring forth a ×16 or whatever by intelligently multiplex. Obviously, it's all squeezed down a ×1 in the end, but if performance was key then the Pi 5 wouldn't be used. It's just a compact way to get considerable RAID 1 storage to hand. Ah, I see a reference to a ‘PCI switch’ rather than controller to multiplex. That's led me to https://pineboards.io/products/hatdrive-dual-2xnvme-2230-2242-gen-2-for-raspberry-pi-5 which uses an ASMedia device. So such things do exist. €40 seems a little high, but they're low volume. NVMe has other standardised transports, e.g. TCP/IP with TLS. This suggests it's just a ‘simple’ matter of software to have a RAID 1 with one M.2 drive dangling off the local ×1 PCIe and the other over Ethernet. Ethernet can stretch from LAN to WAN: a second local Pi 5 on the back of the TV running LibreElec, or an off-site non-Pi. The Linux kernel has NVMe support since 5.0; package nvme-cli drives it. There's also the BSD-licensed SPDK which is a user-mode C implementation of NMVe along with a network stack; to avoid the kernel overhead. https://spdk.io/doc/about.html. Which makes me wonder what the minimal hardware is to deploy either and drive local NVMe storage for network access. Not corporate-level data-centre SAN but home, hobbyist, small-office level. Easiest would be some SoC, ARM or RISC-V, running Linux. SPDK on a non-Linux SoC with PCIe would seem possible. https://spdk.io/doc/porting.html -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2024-06-04 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk