Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-15 Thread hw
On Mon, 2022-11-14 at 15:08 -0500, Michael Stone wrote: > On Mon, Nov 14, 2022 at 08:40:47PM +0100, hw wrote: > > Not really, it was just an SSD.  Two of them were used as cache and they > > failed > > was not surprising.  It's really unfortunate that SSDs fail particulary fast > > when used for

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread Linux-Fan
hw writes: On Fri, 2022-11-11 at 21:26 +0100, Linux-Fan wrote: > hw writes: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > > > On Thu, Nov 10, 2022 at

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 21:26 +0100, Linux-Fan wrote: > hw writes: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > > > On Thu, Nov 10, 2022 at 05:34:32PM

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread Michael Stone
On Mon, Nov 14, 2022 at 08:40:47PM +0100, hw wrote: Not really, it was just an SSD. Two of them were used as cache and they failed was not surprising. It's really unfortunate that SSDs fail particulary fast when used for purposes they can be particularly useful for. If you buy hard drives

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 14:48 -0500, Michael Stone wrote: > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > There was no misdiagnosis.  Have you ever had a failed SSD?  They usually > > just > > disappear. > > Actually, they don't; that's a somewhat unusual failure mode. What else happens?

Re: Sorry for the misattribution [was: ZFS performance (was: Re: deduplicating file systems: VDO] withDebian?)

2022-11-14 Thread hw
On Sat, 2022-11-12 at 07:27 +0100, to...@tuxteam.de wrote: > On Fri, Nov 11, 2022 at 07:22:19PM +0100, to...@tuxteam.de wrote: > > [...] > > > I think what hede was hinting at was that early SSDs had a (pretty) > > limited number of write cycles [...] > > As was pointed out to me, the OP wasn't

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 17:05 +, Curt wrote: > On 2022-11-11, wrote: > > > > I just contested that their failure rate is higher than that of HDDs. > > This is something which was true in early days, but nowadays it seems > > to be just a prejudice. > > If he prefers extrapolating his

Sorry for the misattribution [was: ZFS performance (was: Re: deduplicating file systems: VDO] withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 07:22:19PM +0100, to...@tuxteam.de wrote: [...] > I think what hede was hinting at was that early SSDs had a (pretty) > limited number of write cycles [...] As was pointed out to me, the OP wasn't hede. It was hw. Sorry for the mis-attribution. Cheers -- t

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Linux-Fan
hw writes: On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > > > And mind you, SSDs are *designed to fail* the

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Michael Stone
On Fri, Nov 11, 2022 at 02:05:33PM -0500, Dan Ritter wrote: 300TB/year. That's a little bizarre: it's 9.51 MB/s. Modern high end spinners also claim 200MB/s or more when feeding them continuous writes. Apparently WD thinks that can't be sustained more than 5% of the time. Which makes sense for

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Michael Stone
On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: There was no misdiagnosis. Have you ever had a failed SSD? They usually just disappear. Actually, they don't; that's a somewhat unusual failure mode. I have had a couple of ssd failures, out of hundreds. (And I think mostly from a

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Dan Ritter
to...@tuxteam.de wrote: > > I think what hede was hinting at was that early SSDs had a (pretty) > limited number of write cycles per "block" [1] before failure; they had > (and have) extra blocks to substitute broken ones and do a fair amount > of "wear leveling behind the scenes. So it made

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Dan Ritter
Jeffrey Walton wrote: > On Fri, Nov 11, 2022 at 2:01 AM wrote: > > > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > >... Here's a report > > by folks who do lots of HDDs and SDDs: > > > >

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 12:53:21PM -0500, Jeffrey Walton wrote: > On Fri, Nov 11, 2022 at 2:01 AM wrote: > > > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > >... Here's a report > > by folks who do lots of HDDs and SDDs: > >

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Jeffrey Walton
On Fri, Nov 11, 2022 at 2:01 AM wrote: > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: >... Here's a report > by folks who do lots of HDDs and SDDs: > > https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1-2021/ > > The

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 05:05:51PM -, Curt wrote: > On 2022-11-11, wrote: > > > > I just contested that their failure rate is higher than that of HDDs. [...] > If he prefers extrapolating his anecdotal personal experience to a > general rule rather than applying a verifiable general rule to

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Curt
On 2022-11-11, wrote: > > I just contested that their failure rate is higher than that of HDDs. > This is something which was true in early days, but nowadays it seems > to be just a prejudice. If he prefers extrapolating his anecdotal personal experience to a general rule rather than applying

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 09:12:36AM +0100, hw wrote: > Backblaze does all kinds of things. whatever. > > The gist, for disks playing similar roles (they don't use yet SSDs for bulk > > storage, because of the costs): 2/1518 failures for SSDs, 44/1669 for HDDs. > > > > I'll leave the maths as an

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread hw
On Fri, 2022-11-11 at 08:01 +0100, to...@tuxteam.de wrote: > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > [...] > > > Why would anyone use SSDs for backups?  They're way too expensive for that. > > Possibly. > > > So far,

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread tomas
On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: [...] > Why would anyone use SSDs for backups? They're way too expensive for that. Possibly. > So far, the failure rate with SSDs has been not any better than the failure > rate > of

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > > > And mind you, SSDs are *designed to fail* the sooner the more

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread Michael Stone
On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > And mind you, SSDs are *designed to fail* the sooner the more data you write > to > them.  They have their uses, maybe even for

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > And mind you, SSDs are *designed to fail* the sooner the more data you write > > to > > them.  They have their uses, maybe even for storage if you're so desperate, > > but > > not for

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread Michael Stone
On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: And mind you, SSDs are *designed to fail* the sooner the more data you write to them. They have their uses, maybe even for storage if you're so desperate, but not for backup storage. It's unlikely you'll "wear out" your SSDs faster than you

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 02:19 -0500, gene heskett wrote: > On 11/10/22 00:37, David Christensen wrote: > > On 11/9/22 00:24, hw wrote: > >  > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: > > [...] > Which brings up another suggestion in two parts: > > 1: use amanda, with tar and

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-09 Thread gene heskett
On 11/10/22 00:37, David Christensen wrote: On 11/9/22 00:24, hw wrote: > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: > Hmm, when you can backup like 3.5TB with that, maybe I should put FreeBSD on my > server and give ZFS a try.  Worst thing that can happen is that it