Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-15 Thread hw
On Mon, 2022-11-14 at 15:08 -0500, Michael Stone wrote: > On Mon, Nov 14, 2022 at 08:40:47PM +0100, hw wrote: > > Not really, it was just an SSD.  Two of them were used as cache and they > > failed > > was not surprising.  It's really unfortunate that SSDs fail particulary fast > > when used for

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-15 Thread hw
On Mon, 2022-11-14 at 20:37 +0100, Linux-Fan wrote: > hw writes: > > > On Fri, 2022-11-11 at 22:11 +0100, Linux-Fan wrote: > > > hw writes: > > > > On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote: > [...] > > How do you intend to copy files at any other level than at file level?  At  > > that

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-14 Thread David Christensen
On 11/14/22 13:48, hw wrote: On Fri, 2022-11-11 at 21:55 -0800, David Christensen wrote: Lots of snapshots slows down commands that involve snapshots (e.g.  'zfs list -r -t snapshot ...').  This means sysadmin tasks take longer when the pool has more snapshots. Hm, how long does it take?

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread Linux-Fan
hw writes: On Fri, 2022-11-11 at 21:26 +0100, Linux-Fan wrote: > hw writes: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > > > On Thu, Nov 10, 2022 at

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 21:55 -0800, David Christensen wrote: > [...] > As with most filesystems, performance of ZFS drops dramatically as you > approach 100% usage.  So, you need a data destruction policy that keeps > storage usage and performance at acceptable levels. > > Lots of snapshots

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 21:26 +0100, Linux-Fan wrote: > hw writes: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > > > On Thu, Nov 10, 2022 at 05:34:32PM

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread Michael Stone
On Mon, Nov 14, 2022 at 08:40:47PM +0100, hw wrote: Not really, it was just an SSD. Two of them were used as cache and they failed was not surprising. It's really unfortunate that SSDs fail particulary fast when used for purposes they can be particularly useful for. If you buy hard drives

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 14:48 -0500, Michael Stone wrote: > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > There was no misdiagnosis.  Have you ever had a failed SSD?  They usually > > just > > disappear. > > Actually, they don't; that's a somewhat unusual failure mode. What else happens?

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-14 Thread Linux-Fan
hw writes: On Fri, 2022-11-11 at 22:11 +0100, Linux-Fan wrote: > hw writes: > > On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote: > > [...] > > > >  If you do not value the uptime making actual (even > > > scheduled) copies of the data may be recommendable over > > > using a RAID

Re: Sorry for the misattribution [was: ZFS performance (was: Re: deduplicating file systems: VDO] withDebian?)

2022-11-14 Thread hw
On Sat, 2022-11-12 at 07:27 +0100, to...@tuxteam.de wrote: > On Fri, Nov 11, 2022 at 07:22:19PM +0100, to...@tuxteam.de wrote: > > [...] > > > I think what hede was hinting at was that early SSDs had a (pretty) > > limited number of write cycles [...] > > As was pointed out to me, the OP wasn't

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-14 Thread hw
On Fri, 2022-11-11 at 17:05 +, Curt wrote: > On 2022-11-11, wrote: > > > > I just contested that their failure rate is higher than that of HDDs. > > This is something which was true in early days, but nowadays it seems > > to be just a prejudice. > > If he prefers extrapolating his

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-13 Thread David Christensen
On 11/13/22 13:02, hw wrote: On Fri, 2022-11-11 at 07:55 -0500, Dan Ritter wrote: hw wrote: On Thu, 2022-11-10 at 20:32 -0500, Dan Ritter wrote: Linux-Fan wrote: [...] * RAID 5 and 6 restoration incurs additional stress on the other   disks in the RAID which makes it more likely that one

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-13 Thread hw
On Fri, 2022-11-11 at 22:11 +0100, Linux-Fan wrote: > hw writes: > > > On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote: > > [...] > > > >  If you do not value the uptime making actual (even > > > scheduled) copies of the data may be recommendable over > > > using a RAID because

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-13 Thread hw
On Fri, 2022-11-11 at 07:55 -0500, Dan Ritter wrote: > hw wrote: > > On Thu, 2022-11-10 at 20:32 -0500, Dan Ritter wrote: > > > Linux-Fan wrote: > > > > > > > > > [...] > > > * RAID 5 and 6 restoration incurs additional stress on the other > > >   disks in the RAID which makes it more likely

Re: MTBF interpretations (Re: ZFS performance)

2022-11-13 Thread hede
On Sat, 12 Nov 2022 07:52:32 -0500 Dan Ritter wrote: > No, my interpretation is that the average (mean) lifetime > between failures should be the listed value. At 114 years, half > of the population of drives should still be working. > > This is obviously not congruent with reality. I'd say

Re: MTBF interpretations (Re: ZFS performance)

2022-11-12 Thread Stefan Monnier
>> > Claimed MTBF: 1 million hours. Believe it or not, this is par >> > for the course for high-end disks. >> > >> > 24 hours a day, 365 days a year: 8760 hours per year. >> > 100/8760 = 114 years. >> > >> > So, no: MTBF numbers must be presumed to be malicious lies. >> >> With your

Re: MTBF interpretations (Re: ZFS performance)

2022-11-12 Thread Dan Ritter
hede wrote: > On Fri, 11 Nov 2022 14:05:33 -0500 Dan Ritter wrote: > > > Claimed MTBF: 1 million hours. Believe it or not, this is par > > for the course for high-end disks. > > > > 24 hours a day, 365 days a year: 8760 hours per year. > > 100/8760 = 114 years. > > > > So, no: MTBF

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-12 Thread Dan Ritter
David Christensen wrote: > The Intel Optane Memory Series products are designed to be cache devices -- > when using compatible hardware, Windows, and Intel software. My hardware > should be compatible (Dell PowerEdge T30), but I am unsure if FreeBSD 12.3-R > will see the motherboard NVMe slot or

MTBF interpretations (Re: ZFS performance)

2022-11-12 Thread hede
On Fri, 11 Nov 2022 14:05:33 -0500 Dan Ritter wrote: > Claimed MTBF: 1 million hours. Believe it or not, this is par > for the course for high-end disks. > > 24 hours a day, 365 days a year: 8760 hours per year. > 100/8760 = 114 years. > > So, no: MTBF numbers must be presumed to be

Sorry for the misattribution [was: ZFS performance (was: Re: deduplicating file systems: VDO] withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 07:22:19PM +0100, to...@tuxteam.de wrote: [...] > I think what hede was hinting at was that early SSDs had a (pretty) > limited number of write cycles [...] As was pointed out to me, the OP wasn't hede. It was hw. Sorry for the mis-attribution. Cheers -- t

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-11 Thread David Christensen
On 11/11/22 00:43, hw wrote: On Thu, 2022-11-10 at 21:14 -0800, David Christensen wrote: On 11/10/22 07:44, hw wrote: On Wed, 2022-11-09 at 21:36 -0800, David Christensen wrote: On 11/9/22 00:24, hw wrote:   > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: Taking snapshots is

Re: ZFS performance

2022-11-11 Thread Stefan Monnier
Michael Stone [2022-11-11 14:59:46] wrote: > On Fri, Nov 11, 2022 at 02:05:33PM -0500, Dan Ritter wrote: >>300TB/year. That's a little bizarre: it's 9.51 MB/s. Modern >>high end spinners also claim 200MB/s or more when feeding them >>continuous writes. Apparently WD thinks that can't be sustained

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-11 Thread Linux-Fan
hw writes: On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote: [...] > If you do not value the uptime making actual (even > scheduled) copies of the data may be recommendable over > using a RAID because such schemes may (among other advantages) > protect you from accidental

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Linux-Fan
hw writes: On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > > > And mind you, SSDs are *designed to fail* the

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Michael Stone
On Fri, Nov 11, 2022 at 02:05:33PM -0500, Dan Ritter wrote: 300TB/year. That's a little bizarre: it's 9.51 MB/s. Modern high end spinners also claim 200MB/s or more when feeding them continuous writes. Apparently WD thinks that can't be sustained more than 5% of the time. Which makes sense for

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-11 Thread Michael Stone
On Fri, Nov 11, 2022 at 09:03:45AM +0100, hw wrote: On Thu, 2022-11-10 at 23:12 -0500, Michael Stone wrote: The advantage to RAID 6 is that it can tolerate a double disk failure. With RAID 1 you need 3x your effective capacity to achieve that and even though storage has gotten cheaper, it

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Michael Stone
On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: There was no misdiagnosis. Have you ever had a failed SSD? They usually just disappear. Actually, they don't; that's a somewhat unusual failure mode. I have had a couple of ssd failures, out of hundreds. (And I think mostly from a

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Dan Ritter
to...@tuxteam.de wrote: > > I think what hede was hinting at was that early SSDs had a (pretty) > limited number of write cycles per "block" [1] before failure; they had > (and have) extra blocks to substitute broken ones and do a fair amount > of "wear leveling behind the scenes. So it made

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Dan Ritter
Jeffrey Walton wrote: > On Fri, Nov 11, 2022 at 2:01 AM wrote: > > > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > >... Here's a report > > by folks who do lots of HDDs and SDDs: > > > >

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 12:53:21PM -0500, Jeffrey Walton wrote: > On Fri, Nov 11, 2022 at 2:01 AM wrote: > > > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > >... Here's a report > > by folks who do lots of HDDs and SDDs: > >

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Jeffrey Walton
On Fri, Nov 11, 2022 at 2:01 AM wrote: > > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: >... Here's a report > by folks who do lots of HDDs and SDDs: > > https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1-2021/ > > The

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 05:05:51PM -, Curt wrote: > On 2022-11-11, wrote: > > > > I just contested that their failure rate is higher than that of HDDs. [...] > If he prefers extrapolating his anecdotal personal experience to a > general rule rather than applying a verifiable general rule to

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread Curt
On 2022-11-11, wrote: > > I just contested that their failure rate is higher than that of HDDs. > This is something which was true in early days, but nowadays it seems > to be just a prejudice. If he prefers extrapolating his anecdotal personal experience to a general rule rather than applying

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-11 Thread Dan Ritter
hw wrote: > On Thu, 2022-11-10 at 20:32 -0500, Dan Ritter wrote: > > Linux-Fan wrote: > > > > > > [...] > > * RAID 5 and 6 restoration incurs additional stress on the other > >   disks in the RAID which makes it more likely that one of them > >   will fail. The advantage of RAID 6 is that it

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread tomas
On Fri, Nov 11, 2022 at 09:12:36AM +0100, hw wrote: > Backblaze does all kinds of things. whatever. > > The gist, for disks playing similar roles (they don't use yet SSDs for bulk > > storage, because of the costs): 2/1518 failures for SSDs, 44/1669 for HDDs. > > > > I'll leave the maths as an

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-11 Thread DdB
Am 11.11.2022 um 07:36 schrieb hw: > That's on https://docs.freebsd.org/en/books/handbook/zfs/ > > I don't remember where I read about 8, could have been some documentation > about > FreeNAS. Well, OTOH there do exist some considerations, which may have lead to that number sticking somewhere,

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-11 Thread hw
On Thu, 2022-11-10 at 21:14 -0800, David Christensen wrote: > On 11/10/22 07:44, hw wrote: > > On Wed, 2022-11-09 at 21:36 -0800, David Christensen wrote: > > > On 11/9/22 00:24, hw wrote: > > >   > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: > > [...] > > > > Taking snapshots is

Re: ZFS performance

2022-11-11 Thread hede
On 10.11.2022 16:44, hw wrote: I accidentally trash files on occasion.  Being able to restore them quickly and easily with a cp(1), scp(1), etc., is a killer feature. indeed I'd say the same and I do use a file based backup solution and love having cp, scp, etc. Still, having a tool

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-11 Thread hw
On Fri, 2022-11-11 at 08:01 +0100, to...@tuxteam.de wrote: > On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > > [...] > > > Why would anyone use SSDs for backups?  They're way too expensive for that. > > Possibly. > > > So far,

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-11 Thread hw
On Thu, 2022-11-10 at 23:12 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 08:32:36PM -0500, Dan Ritter wrote: > > * RAID 5 and 6 restoration incurs additional stress on the other > >  disks in the RAID which makes it more likely that one of them > >  will fail. > > I believe that's mostly

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 20:32 -0500, Dan Ritter wrote: > Linux-Fan wrote: > > > [...] > * RAID 5 and 6 restoration incurs additional stress on the other >   disks in the RAID which makes it more likely that one of them >   will fail. The advantage of RAID 6 is that it can then recover >   from

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote: > hw writes: > > > On Wed, 2022-11-09 at 19:17 +0100, Linux-Fan wrote: > > > hw writes: > > > > On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > > > > > Le 09/11/2022 à 12:41, hw a écrit : > > [...] > > > > > I'd > > > > have to use

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread tomas
On Fri, Nov 11, 2022 at 07:15:07AM +0100, hw wrote: > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: [...] > Why would anyone use SSDs for backups? They're way too expensive for that. Possibly. > So far, the failure rate with SSDs has been not any better than the failure > rate > of

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 14:28 +0100, DdB wrote: > Am 10.11.2022 um 13:03 schrieb Greg Wooledge: > > If it turns out that '?' really is the filename, then it becomes a ZFS > > issue with which I can't help. > > just tested: i could create, rename, delete a file with that name on a > zfs filesystem

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 08:48 -0500, Dan Ritter wrote: > hw wrote: > > And I've been reading that when using ZFS, you shouldn't make volumes with > > more > > than 8 disks.  That's very inconvenient. > > > Where do you read these things? I read things like this: "Sun™ recommends that the number

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > > > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > > > And mind you, SSDs are *designed to fail* the sooner the more

Re: weird directory entry on ZFS volume (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread David Christensen
On Thu, Nov 10, 2022 at 05:54:00AM +0100, hw wrote: ls -la insgesamt 5 drwxr-xr-x  3 namefoo namefoo    3 16. Aug 22:36 . drwxr-xr-x 24 root    root    4096  1. Nov 2017  .. drwxr-xr-x  2 namefoo namefoo    2 21. Jan 2020  ? namefoo@host /srv/datadir $ ls -la '?' ls: Zugriff auf ? nicht möglich:

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-10 Thread David Christensen
On 11/10/22 07:44, hw wrote: On Wed, 2022-11-09 at 21:36 -0800, David Christensen wrote: On 11/9/22 00:24, hw wrote:  > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: Be careful that you do not confuse a ~33 GiB full backup set, and 78 snapshots over six months of that same

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Michael Stone
On Thu, Nov 10, 2022 at 08:32:36PM -0500, Dan Ritter wrote: * RAID 5 and 6 restoration incurs additional stress on the other disks in the RAID which makes it more likely that one of them will fail. I believe that's mostly apocryphal; I haven't seen science backing that up, and it hasn't

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread Michael Stone
On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote: On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > And mind you, SSDs are *designed to fail* the sooner the more data you write > to > them.  They have their uses, maybe even for

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Dan Ritter
Linux-Fan wrote: > I think the arguments of the RAID5/6 critics summarized were as follows: > > * Running in a RAID level that is 5 or 6 degrades performance while > a disk is offline significantly. RAID 10 keeps most of its speed and > RAID 1 only degrades slightly for most use cases. > >

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread DdB
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Am 10.11.2022 um 22:37 schrieb Linux-Fan: > Ext4 still does not offer snapshots. The traditional way to do > snapshots outside of fancy BTRFS and ZFS file systems is to add LVM > to the equation although I do not have any useful experience with >

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Linux-Fan
hw writes: On Wed, 2022-11-09 at 19:17 +0100, Linux-Fan wrote: > hw writes: > > On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > > > Le 09/11/2022 à 12:41, hw a écrit : [...] > > I'd > > have to use mdadm to create a RAID5 (or use the hardware RAID but that > > isn't > > AFAIK

Re: weird directory entry on ZFS volume (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread Greg Wooledge
On Thu, Nov 10, 2022 at 06:54:31PM +0100, hw wrote: > Ah, yes. I tricked myself because I don't have hd installed, It's just a symlink to hexdump. lrwxrwxrwx 1 root root 7 Jan 20 2022 /usr/bin/hd -> hexdump unicorn:~$ dpkg -S usr/bin/hd bsdextrautils: /usr/bin/hd unicorn:~$ dpkg -S

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote: > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: > > And mind you, SSDs are *designed to fail* the sooner the more data you write > > to > > them.  They have their uses, maybe even for storage if you're so desperate, > > but > > not for

Re: weird directory entry on ZFS volume (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 09:30 -0500, Greg Wooledge wrote: > On Thu, Nov 10, 2022 at 02:48:28PM +0100, hw wrote: > > On Thu, 2022-11-10 at 07:03 -0500, Greg Wooledge wrote: > > [...] > > printf '%s\0' * | hexdump > > 000 00c2 6177 7468     > > 007 > > I dislike this

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread Michael Stone
On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote: And mind you, SSDs are *designed to fail* the sooner the more data you write to them. They have their uses, maybe even for storage if you're so desperate, but not for backup storage. It's unlikely you'll "wear out" your SSDs faster than you

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 10:47 +0100, DdB wrote: > Am 10.11.2022 um 06:38 schrieb David Christensen: > > What is your technique for defragmenting ZFS? > well, that was meant more or less a joke: there is none apart from > offloading all the data, destroying and rebuilding the pool, and filling > it

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-10 Thread hw
On Thu, 2022-11-10 at 02:19 -0500, gene heskett wrote: > On 11/10/22 00:37, David Christensen wrote: > > On 11/9/22 00:24, hw wrote: > >  > On Tue, 2022-11-08 at 17:30 -0800, David Christensen wrote: > > [...] > Which brings up another suggestion in two parts: > > 1: use amanda, with tar and

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-10 Thread hw
; are plenty of articles.  Here is a general article I found recently: > > https://klarasystems.com/articles/choosing-the-right-zfs-pool-layout/ Thanks! If I make a zpool for backups (or anything else), I need to do some reading beforehand anyway. > MySQL appears to have the ability to use

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Dan Ritter
Brad Rogers wrote: > On Thu, 10 Nov 2022 08:48:43 -0500 > Dan Ritter wrote: > > Hello Dan, > > >8 is not a magic number. > > Clearly, you don't read Terry Pratchett. :-) In the context of ZFS, 8 is not a magic number. May you be ridiculed by Pictsies. -dsr-

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Brad Rogers
On Thu, 10 Nov 2022 08:48:43 -0500 Dan Ritter wrote: Hello Dan, >8 is not a magic number. Clearly, you don't read Terry Pratchett. :-) -- Regards _ "Valid sig separator is {dash}{dash}{space}" / ) "The blindingly obvious is never immediately apparent" / _)rad

Re: weird directory entry on ZFS volume (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread Greg Wooledge
On Thu, Nov 10, 2022 at 02:48:28PM +0100, hw wrote: > On Thu, 2022-11-10 at 07:03 -0500, Greg Wooledge wrote: > good idea: > > printf %s * | hexdump > 000 77c2 6861 0074 > 005 Looks like there might be more than one file here. > > If you misrepresented the

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Dan Ritter
hw wrote: > And I've been reading that when using ZFS, you shouldn't make volumes with > more > than 8 disks. That's very inconvenient. Where do you read these things? The number of disks in a zvol can be optimized, depending on your desired redundancy method, total number of drives, and

weird directory entry on ZFS volume (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 07:03 -0500, Greg Wooledge wrote: > On Thu, Nov 10, 2022 at 05:54:00AM +0100, hw wrote: > > ls -la > > insgesamt 5 > > drwxr-xr-x  3 namefoo namefoo    3 16. Aug 22:36 . > > drwxr-xr-x 24 root    root    4096  1. Nov 2017  .. > > drwxr-xr-x  2 namefoo namefoo    2 21. Jan

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread DdB
Am 10.11.2022 um 13:03 schrieb Greg Wooledge: > If it turns out that '?' really is the filename, then it becomes a ZFS > issue with which I can't help. just tested: i could create, rename, delete a file with that name on a zfs filesystem just as with any other fileystem. But: i recall having

block devices vs. partitions (Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 10:59 +0100, DdB wrote: > Am 10.11.2022 um 04:46 schrieb hw: > > On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote: > > > Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: > > > [...] > [...] > > > > > Why would partitions be better than the block device

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread hw
On Thu, 2022-11-10 at 10:34 +0100, Christoph Brinkhaus wrote: > Am Thu, Nov 10, 2022 at 04:46:12AM +0100 schrieb hw: > > On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote: > > > Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: > > > [...] > [...] > > > > > > > Why would partitions

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Greg Wooledge
On Thu, Nov 10, 2022 at 05:54:00AM +0100, hw wrote: > ls -la > insgesamt 5 > drwxr-xr-x 3 namefoo namefoo3 16. Aug 22:36 . > drwxr-xr-x 24 rootroot4096 1. Nov 2017 .. > drwxr-xr-x 2 namefoo namefoo2 21. Jan 2020 ? > namefoo@host /srv/datadir $ ls -la '?' > ls: Zugriff auf ?

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread DdB
Am 10.11.2022 um 04:46 schrieb hw: > On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote: >> Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: >> [...] >>> FreeBSD has ZFS but can't even configure the disk controllers, so that won't >>> work.  >> >> If I understand you right you mean

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-10 Thread DdB
Am 10.11.2022 um 06:38 schrieb David Christensen: > What is your technique for defragmenting ZFS? well, that was meant more or less a joke: there is none apart from offloading all the data, destroying and rebuilding the pool, and filling it again from the backup. But i do it from time to time if

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-10 Thread Christoph Brinkhaus
Am Thu, Nov 10, 2022 at 04:46:12AM +0100 schrieb hw: > On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote: > > Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: > > [...] > > > FreeBSD has ZFS but can't even configure the disk controllers, so that > > > won't > > > work.  > > > >

Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)

2022-11-09 Thread gene heskett
https://klarasystems.com/articles/choosing-the-right-zfs-pool-layout/ MySQL appears to have the ability to use raw disks.  Tuned correctly, this should give the best results: https://dev.mysql.com/doc/refman/8.0/en/innodb-system-tablespace.html#innodb-raw-devices If ZFS performance is not

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread David Christensen
On 11/9/22 01:35, DdB wrote: > But i am satisfied with zfs performance from spinning rust, if i dont fill up the pool too much, and defrag after a while ... What is your technique for defragmenting ZFS? David

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread David Christensen
On 11/9/22 05:29, didier gaumet wrote: - *BSDs nowadays have departed from old ZFS code and use the same source code stack as Linux (OpenZFS) AIUI FreeBSD 12 and prior use ZFS-on-Linux code, while FreeBSD 13 and later use OpenZFS code. On 11/9/22 05:44, didier gaumet wrote: > I was

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread David Christensen
it. Constructing a ZFS pool to match the workload is not easy. STFW there are plenty of articles. Here is a general article I found recently: https://klarasystems.com/articles/choosing-the-right-zfs-pool-layout/ MySQL appears to have the ability to use raw disks. Tuned correctly,

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-09 Thread hw
On Wed, 2022-11-09 at 19:17 +0100, Linux-Fan wrote: > hw writes: > > > On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > > > Le 09/11/2022 à 12:41, hw a écrit : > > [...] > > > > I am really not so well aware of ZFS state but my impression was that: > > > - FUSE implementation of ZoL

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-09 Thread hw
On Wed, 2022-11-09 at 18:26 +0100, Christoph Brinkhaus wrote: > Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: > [...] > > FreeBSD has ZFS but can't even configure the disk controllers, so that won't > > work.  > > If I understand you right you mean RAID controllers? yes > According to

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-09 Thread Linux-Fan
hw writes: On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > Le 09/11/2022 à 12:41, hw a écrit : [...] > I am really not so well aware of ZFS state but my impression was that: > - FUSE implementation of ZoL (ZFS on Linux) is deprecated and that, > Ubuntu excepted (classic module?),

Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-09 Thread Christoph Brinkhaus
Am Wed, Nov 09, 2022 at 06:11:34PM +0100 schrieb hw: Hi hw, > On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > > Le 09/11/2022 à 12:41, hw a écrit : > > [...] > > > In any case, I'm currently tending to think that putting FreeBSD with ZFS > > > on > > > my > > > server might be the

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread hw
On Wed, 2022-11-09 at 17:29 +0100, DdB wrote: > Am 09.11.2022 um 12:41 schrieb hw: > > In any case, I'm currently tending to think that putting FreeBSD with ZFS on > > my > > server might be the best option.  But then, apparently I won't be able to > > configure the controller cards, so that won't

else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))

2022-11-09 Thread hw
On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote: > Le 09/11/2022 à 12:41, hw a écrit : > [...] > > In any case, I'm currently tending to think that putting FreeBSD with ZFS on > > my > > server might be the best option.  But then, apparently I won't be able to > > configure the controller

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread DdB
Am 09.11.2022 um 12:41 schrieb hw: > In any case, I'm currently tending to think that putting FreeBSD with ZFS on > my > server might be the best option. But then, apparently I won't be able to > configure the controller cards, so that won't really work. And ZFS with Linux > isn't so great

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread D. R. Evans
hw wrote on 11/9/22 04:41: configure the controller cards, so that won't really work. And ZFS with Linux isn't so great because it keeps fuse in between. That isn't true. I've been using ZFS with Debian for years without FUSE, through the ZFSonLinux project. The only slightly

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread didier gaumet
Le 09/11/2022 à 12:41, hw a écrit : [...] In any case, I'm currently tending to think that putting FreeBSD with ZFS on my server might be the best option. But then, apparently I won't be able to configure the controller cards, so that won't really work. And ZFS with Linux isn't so great

Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread hw
u have less capacity ... > Due to snapshots and increments, i am now backing up only once in 2 > weeks, which takes somewhat around 1 hour bcoz of a slow connection. But > i am satisfied with zfs performance from spinning rust, if i dont fill > up the pool too much, and defrag after a while

ZFS performance (was: Re: deduplicating file systems: VDO with Debian?)

2022-11-09 Thread DdB
irrors (much faster than raid) and left the raid only on the slower backup server. Due to snapshots and increments, i am now backing up only once in 2 weeks, which takes somewhat around 1 hour bcoz of a slow connection. But i am satisfied with zfs performance from spinning rust, if i dont f