Re: Feature requests: online backup - defrag - change RAID level

2019-09-17 Thread General Zed
Quoting Zygo Blaxell : On Tue, Sep 17, 2019 at 06:07:24AM -0400, General Zed wrote: Quoting Zygo Blaxell : > I doubt that on a 50TB filesystem you need to read the whole tree...are > you going to globally optimize 50TB at once? That will take a while. I need to read the whole free

Re: Feature requests: online backup - defrag - change RAID level

2019-09-17 Thread General Zed
Quoting Zygo Blaxell : On Mon, Sep 16, 2019 at 10:30:39PM -0400, General Zed wrote: Quoting Zygo Blaxell : > and I think that's impossible so I start from designs > that make forward progress with a fixed allocation of resources. Well, that's not useless, but it's kind

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting General Zed : Quoting Zygo Blaxell : On Sun, Sep 15, 2019 at 01:54:07PM -0400, General Zed wrote: Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 09:50:38PM -0400, General Zed wrote: > > Quoting Zygo Blaxell : > > > On Fri, Sep 13, 2019 at 01:05:52AM -0400, Ge

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting Zygo Blaxell : On Sun, Sep 15, 2019 at 01:54:07PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Fri, Sep 13, 2019 at 09:50:38PM -0400, General Zed wrote: > > > > Quoting Zygo Blaxell : > > > > > On Fri, Sep 13, 2019 at 01:05

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting Zygo Blaxell : On Mon, Sep 16, 2019 at 07:42:51AM -0400, General Zed wrote: Quoting Zygo Blaxell : > Your defrag ideas are interesting, but you should spend a lot more > time learning the btrfs fundamentals before continuing. Right now > you do not understand what btrfs i

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting General Zed : Quoting Zygo Blaxell : On Sun, Sep 15, 2019 at 01:54:07PM -0400, General Zed wrote: Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 09:50:38PM -0400, General Zed wrote: > > Quoting Zygo Blaxell : > > > On Fri, Sep 13, 2019 at 01:05:52AM -0400, Ge

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting Zygo Blaxell : On Sun, Sep 15, 2019 at 01:54:07PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Fri, Sep 13, 2019 at 09:50:38PM -0400, General Zed wrote: > > > > Quoting Zygo Blaxell : > > > > > On Fri, Sep 13, 2019 at 01:05

Re: Feature requests: online backup - defrag - change RAID level

2019-09-16 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 05:23:21PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Wed, Sep 11, 2019 at 07:21:31PM -0400, webmas...@zedlx.com wrote: > > > > Quoting Zygo Blaxell : [...etc...] > > > On Wed, Sep 11, 2019 at 01:20:53PM -0

Re: Feature requests: online backup - defrag - change RAID level

2019-09-15 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 09:28:49PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Fri, Sep 13, 2019 at 05:25:20AM -0400, General Zed wrote: > > > > Quoting General Zed : > > > > > Quoting Zygo Blaxell : > > > > >

Re: Feature requests: online backup - defrag - change RAID level

2019-09-15 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 09:50:38PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Fri, Sep 13, 2019 at 01:05:52AM -0400, General Zed wrote: > > > > Quoting Zygo Blaxell : > > > > > On Thu, Sep 12, 2019 at 08:26

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 01:05:52AM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: > > > > Quoting Zygo Blaxell : > > > > > Don't forget you have to write new checksum

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 01:05:52AM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: > > > > Quoting Zygo Blaxell : > > > > > Don't forget you have to write new checksum

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 05:25:20AM -0400, General Zed wrote: Quoting General Zed : > Quoting Zygo Blaxell : > > > On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: > > > > > > Quoting Zygo Blaxell : > > > > >

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting Zygo Blaxell : On Fri, Sep 13, 2019 at 07:04:28AM -0400, Austin S. Hemmelgarn wrote: On 2019-09-12 19:54, Zygo Blaxell wrote: > On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: > > > > Quoting Chris Murphy : > > > > > On Thu, Sep 12, 2019 a

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting "Austin S. Hemmelgarn" : On 2019-09-13 12:54, General Zed wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-12 18:21, General Zed wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-12 15:18, webmas...@zedlx.com wrote: Quoting "Austin S.

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting General Zed : Quoting "Austin S. Hemmelgarn" : On 2019-09-12 18:57, General Zed wrote: Quoting Chris Murphy : On Thu, Sep 12, 2019 at 3:34 PM General Zed wrote: Quoting Chris Murphy : On Thu, Sep 12, 2019 at 1:18 PM wrote: It is normal and common for defrag op

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting "Austin S. Hemmelgarn" : On 2019-09-12 18:57, General Zed wrote: Quoting Chris Murphy : On Thu, Sep 12, 2019 at 3:34 PM General Zed wrote: Quoting Chris Murphy : On Thu, Sep 12, 2019 at 1:18 PM wrote: It is normal and common for defrag operation to use some

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting General Zed : Quoting General Zed : Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: At worst, it just has to completely write-out "all metadata",

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting "Austin S. Hemmelgarn" : On 2019-09-12 18:21, General Zed wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-12 15:18, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-11 17:37, webmas...@zedlx.com wrote: Quoting "Aust

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting General Zed : Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: > > At worst, it just has to completely write-out "all metadata", all the

Re: Feature requests: online backup - defrag - change RAID level

2019-09-13 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: > > > > Quoting Chris Murphy : > > > > > On Thu, Sep 12, 2019 a

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : > On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: > > > > At worst, it just has to completely write-out "all metadata", all the way up > &g

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : You mean: all metadata size is 156 GB on one of your systems. However, you don't typically have to put ALL metadata in RAM. You need just some parts needed for defrag operatio

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : You mean: all metadata size is 156 GB on one of your systems. However, you don't typically have to put ALL metadata in RAM. You need just some parts needed for defrag operatio

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 08:26:04PM -0400, General Zed wrote: Quoting Zygo Blaxell : > Don't forget you have to write new checksum and free space tree pages. > In the worst case, you'll need about 1GB of new metadata pages for each > 128MB you def

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Wed, Sep 11, 2019 at 04:01:01PM -0400, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : > Not necessarily. Even ignoring the case of data deduplication (which > needs to be considered if you care at all about enterprise usage, and is > part of the who

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Thu, Sep 12, 2019 at 06:57:26PM -0400, General Zed wrote: Quoting Chris Murphy : > On Thu, Sep 12, 2019 at 3:34 PM General Zed wrote: > > > > > > Quoting Chris Murphy : > > > > > On Thu, Sep 12, 2019 at 1:18 PM wrote: >

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Chris Murphy : On Thu, Sep 12, 2019 at 3:34 PM General Zed wrote: Quoting Chris Murphy : > On Thu, Sep 12, 2019 at 1:18 PM wrote: >> >> It is normal and common for defrag operation to use some disk space >> while it is running. I estimate that a reasona

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting "Austin S. Hemmelgarn" : On 2019-09-12 15:18, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-11 17:37, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-11 13:20, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 201

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Zygo Blaxell : On Wed, Sep 11, 2019 at 07:21:31PM -0400, webmas...@zedlx.com wrote: Quoting Zygo Blaxell : > On Wed, Sep 11, 2019 at 01:20:53PM -0400, webmas...@zedlx.com wrote: > > > > Quoting "Austin S. Hemmelgarn" : > > > > > On 2019-09-10 19:32, webmas...@zedlx.com wrote: > > >

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting "Austin S. Hemmelgarn" : On 2019-09-12 15:18, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-11 17:37, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 2019-09-11 13:20, webmas...@zedlx.com wrote: Quoting "Austin S. Hemmelgarn" : On 201

Re: Feature requests: online backup - defrag - change RAID level

2019-09-12 Thread General Zed
Quoting Chris Murphy : On Thu, Sep 12, 2019 at 1:18 PM wrote: It is normal and common for defrag operation to use some disk space while it is running. I estimate that a reasonable limit would be to use up to 1% of total partition size. So, if a partition size is 100 GB, the defrag can use 1