On Thu, Aug 05, 2021 at 11:05:41AM +0000, Eric Wong wrote: > > - I will bring up the rest of the nodes throughout the week, so > > x-lore.kernel.org will become more geoip-balanced. I will share any other > > observations once I have more data. Once all 4 nodes are up, I will share > > this more widely with kernel devs so they can kick some tires and report > > whether they are seeing decreased performance compared to current > > lore.kernel.org. It's entirely possible that my plan to use > > mirrors.edge.kernel.org nodes for this isn't one of my brightest ideas, in > > which case I may bring up several dedicated instances in multiple clouds > > instead. > > Increasing the size of the SSD caches would net the most > dramatic improvement (or going SSD-only). Even consumer grade > SSDs (MLC/TLC) leave enterprise HDDs in the dust.
The more I poke at this, the more it seems that using faster storage is a much better plan than trying to make this work reasonably well with our distro mirroring nodes. Occasional high IO latency will go largely unnoticed when it's apt or dnf that's fronting the requests, but web or b4 calls that make someone wait for 10+ seconds seems like a recipe for some very irate developers. I actually have a good opportunity to move our current git.kernel.org systems to newer hardware at Equinix Metal. We'll go from 6 older nodes to 3 newer nodes, each with more CPU/RAM and 1.2TB total of SSD. This should give us plenty of room to colocate lore.kernel.org on the same systems as git.kernel.org, and it's a much better plan than trying to make spinning rust work well for this purpose. > > Thanks for all your work, Eric. > > You're welcome and thanks for the support. It's been a very > rough time, especially with the pandemic still dragging on :< And we thought we were all done this spring, eh? Ah well, this too shall pass. Best regards, -K
