Re: [gentoo-user] Hard drive pricing and the near future
On 06/10/2021 21:20, Laurence Perkins wrote: But currently the WD Purples and the Seagate Skyhawks should all be CMR. Seagate have said the Iromwolf range will remain CMR. BarraCuda is now all SMR (renamed from Barracuda, presumably to say it's still the budget range, but the old drives are pre-SMR). I get the impression that FireCuda are the "budget CMR" range, but don't quote me on that. Obviously I have no power to hold them to that, so do not take this as any kind of guarantee. But if they change it without warning they're likely to have a lot of unhappy customers. AS WD found out when loads of "marketed for NAS" WD Reds got returned "not fit for purpose" when people put them in NASs. Cheers, Wol
Re: [gentoo-user] Hard drive pricing and the near future
Am Wed, Oct 06, 2021 at 10:24:07PM +0200 schrieb Frank Steinmetzger: > Am Tue, Oct 05, 2021 at 06:32:24PM -0500 schrieb Dale: > > Howdy all, > > […] > > If anyone reading this does track the pricing of drives, are they on the > > rise, stable, dropping or what? Is this a good time to expand while it > > is more cost effective? > > Well I’m on ye Olde Continent and here I use a price search engine called > geizhals (German for scrooge). But they have an English equivalent ;-) and > of course use European pricing schemes (meaning including taxes, which you > do different). > https://skinflint.co.uk/?m=1 > > The big difference here: it allows you to filter and sort by all sorts of > criteria, such as being (non-)SMR or price/TB. And it shows you the prices > of lots of different shops, including Amazon. So while it may not give you > an idea of the US market, it gives you one of available drive models. Ah, and another reason why I wanted to suggest this site: it also remembers the price development of an item. If you view an item’s details, you have its price history on the top right of the screen. -- Grüße | Greetings | Qapla’ Please do not share anything from, with or about me on any social network. The only thing that makes some people bearable is their absence. signature.asc Description: PGP signature
Re: [gentoo-user] Hard drive pricing and the near future
Am Tue, Oct 05, 2021 at 06:32:24PM -0500 schrieb Dale: > Howdy all, > […] > If anyone reading this does track the pricing of drives, are they on the > rise, stable, dropping or what? Is this a good time to expand while it > is more cost effective? Well I’m on ye Olde Continent and here I use a price search engine called geizhals (German for scrooge). But they have an English equivalent ;-) and of course use European pricing schemes (meaning including taxes, which you do different). https://skinflint.co.uk/?m=1 The big difference here: it allows you to filter and sort by all sorts of criteria, such as being (non-)SMR or price/TB. And it shows you the prices of lots of different shops, including Amazon. So while it may not give you an idea of the US market, it gives you one of available drive models. For instance, here are all internal non-SMR 3.5″ drives of 6 TB and up: https://skinflint.co.uk/?cat=hde7s=1080_SATA+1.5Gb%2Fs%7E1080_SATA+3Gb%2Fs%7E1080_SATA+6Gb%2Fs%7E13810_6000%7E3772_3.5%7E8457_non-SMR I like to buy naked drives, because I don’t like “cable salad”, i.e. having multiple external cases, each needing its own special power supply. Instead, I’d opt for a hard drive dock which lets you insert a hard disk as if it were an SD card. Actually, I use this internal bay for my PC case: https://skinflint.co.uk/sharkoon-sata-quickport-internal-multi-a862667.html It eats standard SATA drives and is connected via standard connectors to my PSU and mobo. > However, in a year or so, I'm getting fiber internet, dang fast at that. > It's starts at 200Mb but still over a 100 times faster than current > connection. It goes all the way up to 1Gb. God help us all. ROFL But when are you ever going to watch it all? -- Grüße | Greetings | Qapla’ Please do not share anything from, with or about me on any social network. Filling your car becomes much cheaper if you install a smaller tank. signature.asc Description: PGP signature
RE: [gentoo-user] Hard drive pricing and the near future
> On Wed, Oct 6, 2021 at 3:37 PM Laurence Perkins wrote: > > I think what to look for there would be if there's a way to align the BTRFS > > chunks to the SMR blocks. > > There are definitely ways to implement filesystems that are more compatible > with SMR. You basically want something like a log-based filesystem. A COW > filesystem is actually a really good candidate as they don't do in-place > writes ever, and all you need to do is defer block frees and garbage collect > and so on to make it more log-based. COW is a good compromise. Log-based filesystems like NILFS2 generally have terrible performance. Their only advantage in this situation is that they don't suffer any detectable performance loss from SMR. EXT4 has new formatting options that tell it to sequentialize the metadata writes as much as possible. Supposedly it helps quite a bit, but I haven't tested it myself yet. > > One of the issues though is that these drives obfuscate how they work. > The filesystem has no way to intentionally write to the CMR vs SMR regions. It depends on the drive. There are ATA commands you can use and at least some drives will tell you what their zones are and can be told to give the host more control over what goes where. https://zonedstorage.io/getting-started/smr-disk/ has a good description. Definitely avoid the Drive Managed Interface ones except maybe for cheap, bulk storage. But the Host Aware and Host Managed ones can perform pretty well on some IO loads with proper FS tuning. > > This is why drive-managed SMR really shouldn't be a thing. > Host-managed SMR makes a LOT more sense, because then the filesystem can > mitigate most of the issues and not end up fighting the drive firmware, whose > behavior isn't even standardized. Yup. Only reason for drive-managed is for drop-in replacement of low-throughput systems that couldn't otherwise handle it. > > > But the manufacturers decided to continue manufacturing CMR disks for the > > surveillance industry, so I haven't had to worry about it just yet. > > There are lots of CMR drives out there. It is just that you have to be > careful as nothing is well-documented and it is all subject to change. It is > like trying to figure out how many channels a DIMM has or what its timing > capabilities are. > > I saw a good price on an Exos drive and I believe those are all CMR. > This is why I use slickdeals - you can set up searches and get alerts when a > price drops. It also picks up stuff like Best Buy who often has some of the > best prices on USB enclosures for whatever reason when they go on sale. Almost everything marketed as being for the surveillance industry is going to be CMR. We often run the drives at 80% or better throughput and a lot of it is non-sequential. SMRs just can't keep up. So there was a lot of pushback from the surveillance industry about not changing the underlying storage technology without clearly labeling it as such. Prior to that the announcement from Seagate and I think WD was that everything larger than 12TB was going to be SMR. (I work on the software side of things, I get hardware stuff second hand, so my understanding may be fuzzy.) But currently the WD Purples and the Seagate Skyhawks should all be CMR. Obviously I have no power to hold them to that, so do not take this as any kind of guarantee. But if they change it without warning they're likely to have a lot of unhappy customers. > > -- > Rich > > LMP
Re: [gentoo-user] Hard drive pricing and the near future
On Wed, Oct 6, 2021 at 3:37 PM Laurence Perkins wrote: > I think what to look for there would be if there's a way to align the BTRFS > chunks to the SMR blocks. There are definitely ways to implement filesystems that are more compatible with SMR. You basically want something like a log-based filesystem. A COW filesystem is actually a really good candidate as they don't do in-place writes ever, and all you need to do is defer block frees and garbage collect and so on to make it more log-based. One of the issues though is that these drives obfuscate how they work. The filesystem has no way to intentionally write to the CMR vs SMR regions. This is why drive-managed SMR really shouldn't be a thing. Host-managed SMR makes a LOT more sense, because then the filesystem can mitigate most of the issues and not end up fighting the drive firmware, whose behavior isn't even standardized. > But the manufacturers decided to continue manufacturing CBR disks for the > surveillance industry, so I haven't had to worry about it just yet. There are lots of CMR drives out there. It is just that you have to be careful as nothing is well-documented and it is all subject to change. It is like trying to figure out how many channels a DIMM has or what its timing capabilities are. I saw a good price on an Exos drive and I believe those are all CMR. This is why I use slickdeals - you can set up searches and get alerts when a price drops. It also picks up stuff like Best Buy who often has some of the best prices on USB enclosures for whatever reason when they go on sale. -- Rich
RE: [gentoo-user] Hard drive pricing and the near future
> On Wed, Oct 6, 2021 at 12:16 PM Laurence Perkins wrote: > > > > Other option, depending on exactly what your use case is would be to look > > into your choice of filesystem. SMR doesn't like random writes into one of > > its chunks unless it has enough idle time to go back and straighten it out > > later is all. There are now format options for ext4 to align its metadata > > to the SMR sections and to make it avoid random writes as much as it can. > > Additionally BTRFS, ZFS, and NILFS2 are all structured such that they tend > > to write from one end of the disk to the other and then jump back to the > > beginning, so they see little if any degradation from SMR. > > Unless something has changed, it was ZFS rebuilds that caused a lot of the > initial fuss on Linux. Drives were getting dropped from pools due to > timeouts/etc during rebuilds. I'm not sure how sequential the IO is for ZFS > rebuilds. I think btrfs seems a bit smarter about scrubs in general. > > -- > Rich > Good to know. I don't use ZFS myself, and I suspect the benchmarks I looked at when I was checking on how best to deal with SMR drives didn't try a pool rebuild for ZFS. Scrub on BTRFS is read-only unless there are errors that need correcting, which should be rare. BTRFS balance operations might be affected, but those aren't needed as often as they used to be and should still operate linearly on whole chunks. I think what to look for there would be if there's a way to align the BTRFS chunks to the SMR blocks. But the manufacturers decided to continue manufacturing CBR disks for the surveillance industry, so I haven't had to worry about it just yet. LMP
Re: [gentoo-user] Hard drive pricing and the near future
On Wed, Oct 6, 2021 at 12:16 PM Laurence Perkins wrote: > > Other option, depending on exactly what your use case is would be to look > into your choice of filesystem. SMR doesn't like random writes into one of > its chunks unless it has enough idle time to go back and straighten it out > later is all. There are now format options for ext4 to align its metadata to > the SMR sections and to make it avoid random writes as much as it can. > Additionally BTRFS, ZFS, and NILFS2 are all structured such that they tend to > write from one end of the disk to the other and then jump back to the > beginning, so they see little if any degradation from SMR. Unless something has changed, it was ZFS rebuilds that caused a lot of the initial fuss on Linux. Drives were getting dropped from pools due to timeouts/etc during rebuilds. I'm not sure how sequential the IO is for ZFS rebuilds. I think btrfs seems a bit smarter about scrubs in general. -- Rich
RE: [gentoo-user] Hard drive pricing and the near future
> -Original Message- > From: Dale > Sent: Tuesday, October 5, 2021 4:32 PM > To: gentoo-user@lists.gentoo.org > Subject: [gentoo-user] Hard drive pricing and the near future > > > Howdy all, > > I still have quite a bit of drive storage but I've read that prices on drives > are on the rise. Thing is, I don't track them much. I'm looking at buying a > 8TB drive and I've researched to make sure I'm getting a PMR/CMR drive. I'm > avoiding a SMR since it doesn't perform as well in my use case. I tend to > stick with Seagate, WD and other major makers. > > If anyone reading this does track the pricing of drives, are they on the > rise, stable, dropping or what? Is this a good time to expand while it is > more cost effective? I shop around on ebay, Amazon and others before buying. > I'm not opposed to buying used since I can sometimes find one that was > pulled and sometimes has only a few hours of use. I found one once that only > had like 10 hours on it. Still got it too. > > One reason I'm wanting to do this now is price. However, in a year or so, > I'm getting fiber internet, dang fast at that. It's starts at 200Mb but > still over a 100 times faster than current connection. It goes all the way > up to 1Gb. God help us all. ROFL > > Thoughts?? > > Dale > > :-) :-) > If you want to avoid SMR, look for the drives built for video surveillance systems. Most manufacturers have a line for that purpose. For WD it's the "purple" ones. Other option, depending on exactly what your use case is would be to look into your choice of filesystem. SMR doesn't like random writes into one of its chunks unless it has enough idle time to go back and straighten it out later is all. There are now format options for ext4 to align its metadata to the SMR sections and to make it avoid random writes as much as it can. Additionally BTRFS, ZFS, and NILFS2 are all structured such that they tend to write from one end of the disk to the other and then jump back to the beginning, so they see little if any degradation from SMR. LMP
RE: [gentoo-user] 2021-08-24-eudev-retirement
> Jack wrote: > > On 2021.10.05 15:32, the...@sys-concept.com wrote: > >> I have "eudev" installed but I think it will be obsolete as of > >> Jan.1/22 according to news: 2021-08-24-eudev-retirement > >> > >> Does converting from: sys-fs/eudev > >> to: sys-fs/udev > >> > >> is as simple as: emerge -C sys-fs/eudev emerge sys-fs/udev > > Have you read the news item? It's not quite that literal about what > > to do, but does imply that should work. However, on 15 Sep there was > > a post on the gentoo-dev list with a link to > > https://github.com/eudev-project/eudev (see README.md) which says that > > there is a new team taking up maintenance of eudev, so it is not > > abandoned, and so it is not clear whether eudev will actually be > > dropped or not. > > > > > > > When I switched from udev to eudev, it was as simple as uninstalling udev and > installing eudev. I'd think as you do the reverse would work now as well but > as you point out, it may not. I'd recommend going to a console to do this > and logging out of any GUIs as well. Of course restarting udev afterwards as > well. > > I also saw the post about new maintainers. I'm hopeful that it will keep > being maintained even tho it can be a headache for devs at times. Thing is, > I've read a lot of distros use eudev and avoid the systemd version as much as > possible, even tho eudev still has the same code from my understanding. > Still, I'd rather stick with what works for me. > > Maybe we will know pretty soon what the status of this will be. > > Dale > > :-) :-) > > I was also glad to hear about the new maintainers. The day after I heard the project had served its purpose and was being abandoned I went to update one of my few systems with udev instead of eudev and the build crashed and it took me a couple hours to trick it into compiling correctly. I was a little worried that might end up being prophetic. Whatever the exact differences are, eudev still seems to build a little more reliably. Maybe it's just that the project does a better job of informing people about changes to the build requirements... LMP
