Re: Bug in 'btrfs filesystem du' ?

[Andrei Borzenkov]

28.06.2016 20:20, Andrei Borzenkov пишет:
> 28.06.2016 19:55, Henk Slager пишет:
>> On Tue, Jun 28, 2016 at 2:56 PM, M G Berberich  
>> wrote:
>>> Hello,
>>>
>>> Am Montag, den 27. Juni schrieb Henk Slager:
 On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich  
 wrote:
> Am Montag, den 27. Juni schrieb M G Berberich:
>> after a balance ‘btrfs filesystem du’ probably shows false data about
>> shared data.
>
> Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.

 With  btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct
 about shared data.
>>>
>>> I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and
>>> 4.61.
>> Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516
>> (current stock opensuse tumbleweed) I cannot reproduce the problem.
>>
> 
> I confirm the same behavior on openSUSE Tumbleweed with kernel 4.6.2-1.2
> and btrfsprogs 4.5.3-1.2 using provided script.
> 

I realized that it sounded ambiguous. I do see the reported bug on
openSUSE Tumbleweed using the same versions as you. After rebalance
shared data disappears completely in du output.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in 'btrfs filesystem du' ?

[Andrei Borzenkov]

28.06.2016 19:55, Henk Slager пишет:
> On Tue, Jun 28, 2016 at 2:56 PM, M G Berberich  
> wrote:
>> Hello,
>>
>> Am Montag, den 27. Juni schrieb Henk Slager:
>>> On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich  
>>> wrote:
 Am Montag, den 27. Juni schrieb M G Berberich:
> after a balance ‘btrfs filesystem du’ probably shows false data about
> shared data.

 Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
>>>
>>> With  btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct
>>> about shared data.
>>
>> I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and
>> 4.61.
> Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516
> (current stock opensuse tumbleweed) I cannot reproduce the problem.
> 

I confirm the same behavior on openSUSE Tumbleweed with kernel 4.6.2-1.2
and btrfsprogs 4.5.3-1.2 using provided script.

>  The later kernel with two patches to make the kernel work:
>> https://lkml.org/lkml/2016/6/1/310 https://lkml.org/lkml/2016/6/1/311 .
> ... so these seem to cause the problem
> 
> 
>> You can see the script¹ I used (do-btrfs-du-test) and the logs at
>> http://m-berberich.de/btrfs/
>>
>> In all four cases, ‘btrfs fi du -s .’ reports
>>
>>  Total   Exclusive  Set shared  Filename
>>   59.38MiB   0.00B29.69MiB  .
>>
>> befor balance and
>>
>>  Total   Exclusive  Set shared  Filename
>>   59.38MiB59.38MiB   0.00B  .
>>
>> after balance.
>>
>> Disclaimer: The script works for me, no guaranty at all.
>>
>> MfG
>> bmg
>> __
>> ¹ Disclaimer: The script works for me, no guaranty at all.
>> --
>> „Des is völlig wurscht, was heut beschlos- | M G Berberich
>>  sen wird: I bin sowieso dagegn!“  | m...@m-berberich.de
>> (SPD-Stadtrat Kurt Schindler; Regensburg)  |
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in 'btrfs filesystem du' ?

[Henk Slager]

On Tue, Jun 28, 2016 at 2:56 PM, M G Berberich  wrote:
> Hello,
>
> Am Montag, den 27. Juni schrieb Henk Slager:
>> On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich  
>> wrote:
>> > Am Montag, den 27. Juni schrieb M G Berberich:
>> >> after a balance ‘btrfs filesystem du’ probably shows false data about
>> >> shared data.
>> >
>> > Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
>>
>> With  btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct
>> about shared data.
>
> I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and
> 4.61.
Also with kernel 4.6.2-1-default and btrfs-progs v4.5.3+20160516
(current stock opensuse tumbleweed) I cannot reproduce the problem.

 The later kernel with two patches to make the kernel work:
> https://lkml.org/lkml/2016/6/1/310 https://lkml.org/lkml/2016/6/1/311 .
... so these seem to cause the problem


> You can see the script¹ I used (do-btrfs-du-test) and the logs at
> http://m-berberich.de/btrfs/
>
> In all four cases, ‘btrfs fi du -s .’ reports
>
>  Total   Exclusive  Set shared  Filename
>   59.38MiB   0.00B29.69MiB  .
>
> befor balance and
>
>  Total   Exclusive  Set shared  Filename
>   59.38MiB59.38MiB   0.00B  .
>
> after balance.
>
> Disclaimer: The script works for me, no guaranty at all.
>
> MfG
> bmg
> __
> ¹ Disclaimer: The script works for me, no guaranty at all.
> --
> „Des is völlig wurscht, was heut beschlos- | M G Berberich
>  sen wird: I bin sowieso dagegn!“  | m...@m-berberich.de
> (SPD-Stadtrat Kurt Schindler; Regensburg)  |
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in 'btrfs filesystem du' ?

[M G Berberich]

Hello,

Am Montag, den 27. Juni schrieb Henk Slager:
> On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich  
> wrote:
> > Am Montag, den 27. Juni schrieb M G Berberich:
> >> after a balance ‘btrfs filesystem du’ probably shows false data about
> >> shared data.
> >
> > Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.
> 
> With  btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct
> about shared data.

I tested with kernels 4.6.3 and 4.7-rc5 and with btrfs-progs 4.5.2 and
4.61. The later kernel with two patches to make the kernel work:
https://lkml.org/lkml/2016/6/1/310 https://lkml.org/lkml/2016/6/1/311 .

You can see the script¹ I used (do-btrfs-du-test) and the logs at
http://m-berberich.de/btrfs/

In all four cases, ‘btrfs fi du -s .’ reports

 Total   Exclusive  Set shared  Filename
  59.38MiB   0.00B29.69MiB  .

befor balance and

 Total   Exclusive  Set shared  Filename
  59.38MiB59.38MiB   0.00B  .

after balance.

Disclaimer: The script works for me, no guaranty at all.

MfG
bmg
__
¹ Disclaimer: The script works for me, no guaranty at all.
-- 
„Des is völlig wurscht, was heut beschlos- | M G Berberich
 sen wird: I bin sowieso dagegn!“  | m...@m-berberich.de
(SPD-Stadtrat Kurt Schindler; Regensburg)  | 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in 'btrfs filesystem du' ?

[Henk Slager]

On Mon, Jun 27, 2016 at 3:33 PM, M G Berberich  wrote:
> Am Montag, den 27. Juni schrieb M G Berberich:
>> after a balance ‘btrfs filesystem du’ probably shows false data about
>> shared data.
>
> Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.

With  btrfs-progs v4.6.1 and kernel 4.7-rc5, the numbers are correct
about shared data.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bug in 'btrfs filesystem du' ?

[M G Berberich]

Am Montag, den 27. Juni schrieb M G Berberich:
> after a balance ‘btrfs filesystem du’ probably shows false data about
> shared data.

Oh, I forgot: I have btrfs-progs v4.5.2 and kernel 4.6.2.

MfG
bmg

-- 
„Des is völlig wurscht, was heut beschlos- | M G Berberich
 sen wird: I bin sowieso dagegn!“  | m...@m-berberich.de
(SPD-Stadtrat Kurt Schindler; Regensburg)  | 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bug in 'btrfs filesystem du' ?

[M G Berberich]

Hello,

after a balance ‘btrfs filesystem du’ probably shows false data about
shared data.

To reproduce, create a (smal) btrfs-filesystem, copy some data in a
directory, then ‘cp -a --reflink’ the data. Now all data is shared and
‘btrfs fi du’ shows it correct. In my case:

 Total   Exclusive  Set shared  Filename
  59.38MiB29.69MiB29.69MiB  .

after a balance ‘btrfs fi du’ shows no shared data any more, but all
data as exclusive. In my case:

 Total   Exclusive  Set shared  Filename
  59.38MiB59.38MiB   0.00B  .

As ‘btrfs fi df’ still shows used=29.69MiB, the problem probabaly is
in btrfs-tools.

Test-session log:

# dd if=/dev/urandom of=dev-btrfs bs=4K count=10
10+0 Datensätze ein
10+0 Datensätze aus
40960 bytes (410 MB, 391 MiB) copied, 24.7574 s, 16.5 MB/s

# mkfs.btrfs dev-btrfs
btrfs-progs v4.5.2
See http://btrfs.wiki.kernel.org for more information.
 
Label:  (null)
UUID:   698a2755-8ecb-468d-9577-9a48947361ea
Node size:  16384
Sector size:4096
Filesystem size:390.62MiB
Block group profiles:
  Data: single8.00MiB
  Metadata: DUP  40.00MiB
  System:   DUP  12.00MiB
SSD detected:   no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   IDSIZE  PATH
1   390.62MiB  dev-btrfs
 

# mount /tmp/dev-btrfs /mnt/

# cd /mnt/

# btrfs fi du -s .
 Total   Exclusive  Set shared  Filename
 0.00B   0.00B   0.00B  .

# cp -a /scratch/kernel/linux-4.6/drivers/usb .

# btrfs fi du -s .
 Total   Exclusive  Set shared  Filename
  28.96MiB28.96MiB   0.00B  .

# btrfs fi df .
Data, single: total=56.00MiB, used=3.61MiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=32.00MiB, used=192.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B

# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated:136.00MiB
Device unallocated:  254.62MiB
Device missing:  0.00B
Used: 32.06MiB
Free (estimated):280.94MiB  (min: 153.62MiB)
Data ratio:   1.00
Metadata ratio:   2.00
Global reserve:   16.00MiB  (used: 0.00B)
 
Data,single: Size:56.00MiB, Used:29.69MiB
   /dev/loop0 56.00MiB
 
Metadata,DUP: Size:32.00MiB, Used:1.17MiB
   /dev/loop0 64.00MiB
 
System,DUP: Size:8.00MiB, Used:16.00KiB
   /dev/loop0 16.00MiB
 
Unallocated:
   /dev/loop0254.62MiB

# cp -a --reflink usb usb2

# btrfs fi du -s .
 Total   Exclusive  Set shared  Filename
  59.38MiB29.69MiB29.69MiB  .

# btrfs fi df .
Data, single: total=56.00MiB, used=29.69MiB
System, DUP: total=8.00MiB, used=16.00KiB
Metadata, DUP: total=32.00MiB, used=1.17MiB
GlobalReserve, single: total=16.00MiB, used=0.00B

# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated:136.00MiB
Device unallocated:  254.62MiB
Device missing:  0.00B
Used: 32.06MiB
Free (estimated):280.94MiB  (min: 153.62MiB)
Data ratio:   1.00
Metadata ratio:   2.00
Global reserve:   16.00MiB  (used: 0.00B)
 
Data,single: Size:56.00MiB, Used:29.69MiB
   /dev/loop0 56.00MiB
 
Metadata,DUP: Size:32.00MiB, Used:1.17MiB
   /dev/loop0 64.00MiB
 
System,DUP: Size:8.00MiB, Used:16.00KiB
   /dev/loop0 16.00MiB
 
Unallocated:
   /dev/loop0254.62MiB

# btrfs balance start .
WARNING:
 
Full balance without filters requested. This operation is very
intense and takes potentially very long. It is recommended to
use the balance filters to narrow down the balanced data.
Use 'btrfs balance start --full-balance' option to skip this
warning. The operation will start in 10 seconds.
Use Ctrl-C to stop it.
10 9 8 7 6 5 4 3 2 1
Starting balance without any filters.
Done, had to relocate 4 out of 4 chunks

# btrfs fi du -s .
 Total   Exclusive  Set shared  Filename
  59.38MiB59.38MiB   0.00B  .

# btrfs fi df .
Data, single: total=48.00MiB, used=29.69MiB
System, DUP: total=24.00MiB, used=16.00KiB
Metadata, DUP: total=24.00MiB, used=2.08MiB
GlobalReserve, single: total=16.00MiB, used=0.00B

# btrfs fi usage .
Overall:
Device size: 390.62MiB
Device allocated:144.00MiB
Device unallocated:  246.62MiB
Device missing:  0.00B
Used: 33.88MiB
Free (estimated):264.94MiB  (min: 141.62MiB)
Data ratio:   1.00
Metadata ratio:   2.00
Global reserve:   16.00MiB  (used: 0.00B)
 
Data,single: Size:48.00MiB, Used:29.69MiB
   /dev/loop0 48.00MiB
 
Metadata,DUP: Size:24.00MiB,