Re: Big disk space usage difference, even after defrag, on identical data

2015-04-13 Thread Duncan
Gian-Carlo Pascutto posted on Mon, 13 Apr 2015 16:06:39 +0200 as excerpted: >> Defrag should force the rewrite of entire files and take care of this, >> but obviously it's not returning to "clean" state. I forgot what the >> default minimum file size is if -t isn't set, maybe 128 MiB? But a -t1

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-13 Thread Zygo Blaxell
On Mon, Apr 13, 2015 at 04:06:39PM +0200, Gian-Carlo Pascutto wrote: > On 13-04-15 07:06, Duncan wrote: > > >> So what can explain this? Where did the 66G go? > > > > Out of curiosity, does a balance on the actively used btrfs help? > > > > You mentioned defrag -v -r -clzo, but didn't use the -f

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-13 Thread Gian-Carlo Pascutto
On 13-04-15 07:06, Duncan wrote: >> So what can explain this? Where did the 66G go? > > Out of curiosity, does a balance on the actively used btrfs help? > > You mentioned defrag -v -r -clzo, but didn't use the -f (flush) or -t > (minimum size file) options. Does adding -f -t1 help? Unfortuna

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-13 Thread Gian-Carlo Pascutto
On 13-04-15 06:04, Zygo Blaxell wrote: >> I would think that compression differences or things like >> fragmentation or bookending for modified files shouldn't affect >> this, because the first filesystem has been >> defragmented/recompressed and didn't shrink. >> >> So what can explain this? Whe

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-13 Thread Duncan
Zygo Blaxell posted on Mon, 13 Apr 2015 00:04:36 -0400 as excerpted: > A database ends up maxing out at about a factor of two space usage > because it tends to write short uniform-sized bursts of pages randomly, > so we get a pattern a bit like bricks in a wall: > > 0 MB AA BB CC DD EE FF

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-12 Thread Duncan
Gian-Carlo Pascutto posted on Sat, 11 Apr 2015 21:59:50 +0200 as excerpted: > That's a 66G difference for the same data with the same compress option. > The used size here is much more in line with what I'd have expected > given the nature of the data. > > I would think that compression differenc

Re: Big disk space usage difference, even after defrag, on identical data

2015-04-12 Thread Zygo Blaxell
On Sat, Apr 11, 2015 at 09:59:50PM +0200, Gian-Carlo Pascutto wrote: > Linux mozwell 3.19.0-trunk-amd64 #1 SMP Debian 3.19.1-1~exp1 > (2015-03-08) x86_64 GNU/Linux > btrfs-progs v3.19.1 > > I have a btrfs volume that's been in use for a week or 2. It has about > ~560G of uncompressible data (video

Big disk space usage difference, even after defrag, on identical data

2015-04-11 Thread Gian-Carlo Pascutto
Linux mozwell 3.19.0-trunk-amd64 #1 SMP Debian 3.19.1-1~exp1 (2015-03-08) x86_64 GNU/Linux btrfs-progs v3.19.1 I have a btrfs volume that's been in use for a week or 2. It has about ~560G of uncompressible data (video files, tar.xz, git repos, ...) and ~200G of data that compresses 2:1 with LZO (P