Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10

2016-05-25 Thread Henk Slager 
There is a division by 2 missing in the code. With that added, the
RAID10 numbers make more sense. See also:
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/53989

More detail in here:
https://www.spinics.net/lists/linux-btrfs/msg52882.html

And if you want to look at allocation in a different way, this might
be interesting:
https://github.com/knorrie/btrfs-heatmap/blob/master/show_usage.py

On Mon, May 23, 2016 at 3:34 PM, Marco Lorenzo Crociani
 wrote:
> Hi,
> as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
> I have a 4 partitions RAID10 btrfs filesystem almost full.
> 'btrfs filesystem usage' reports wrong "Unallocated" indications.
>
> Linux 4.5.3
> btrfs-progs v4.5.3
>
>
> # btrfs fi usage /data/
>
> Overall:
> Device size:  13.93TiB
> Device allocated:  13.77TiB
> Device unallocated: 167.54GiB
> Device missing: 0.00B
> Used:  13.44TiB
> Free (estimated): 244.39GiB(min: 244.39GiB)
> Data ratio:  2.00
> Metadata ratio:  2.00
> Global reserve: 512.00MiB(used: 0.00B)
>
> Data,single: Size:8.00MiB, Used:0.00B
>/dev/sda4   8.00MiB
>
> Data,RAID10: Size:6.87TiB, Used:6.71TiB
>/dev/sda4   1.72TiB
>/dev/sdb3   1.72TiB
>/dev/sdc3   1.72TiB
>/dev/sdd3   1.72TiB
>
> Metadata,single: Size:8.00MiB, Used:0.00B
>/dev/sda4   8.00MiB
>
> Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
>/dev/sda4   4.75GiB
>/dev/sdb3   4.75GiB
>/dev/sdc3   4.75GiB
>/dev/sdd3   4.75GiB
>
> System,single: Size:4.00MiB, Used:0.00B
>/dev/sda4   4.00MiB
>
> System,RAID10: Size:16.00MiB, Used:768.00KiB
>/dev/sda4   4.00MiB
>/dev/sdb3   4.00MiB
>/dev/sdc3   4.00MiB
>/dev/sdd3   4.00MiB
>
> Unallocated:
>/dev/sda4   1.76TiB
>/dev/sdb3   1.76TiB
>/dev/sdc3   1.76TiB
>/dev/sdd3   1.76TiB
>
> --
> # btrfs fi show /data/
> Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
> Total devices 4 FS bytes used 6.72TiB
> devid1 size 3.48TiB used 3.44TiB path /dev/sda4
> devid2 size 3.48TiB used 3.44TiB path /dev/sdb3
> devid3 size 3.48TiB used 3.44TiB path /dev/sdc3
> devid4 size 3.48TiB used 3.44TiB path /dev/sdd3
>
> --
> # btrfs fi df /data/
> Data, RAID10: total=6.87TiB, used=6.71TiB
> Data, single: total=8.00MiB, used=0.00B
> System, RAID10: total=16.00MiB, used=768.00KiB
> System, single: total=4.00MiB, used=0.00B
> Metadata, RAID10: total=19.00GiB, used=14.15GiB
> Metadata, single: total=8.00MiB, used=0.00B
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> --
> # df -h
> /dev/sda4 7,0T  6,8T245G  97% /data
>
> Regards,
>
> --
> Marco Crociani
> Prisma Telecom Testing S.r.l.
> via Petrocchi, 4  20127 MILANO  ITALY
> Phone:  +39 02 26113507
> Fax:  +39 02 26113597
> e-mail:  mar...@prismatelecomtesting.com
> web:  http://www.prismatelecomtesting.com
>
> Questa email (e I suoi allegati) costituisce informazione riservata e
> confidenziale e può essere soggetto a legal privilege. Può essere utilizzata
> esclusivamente dai suoi destinatari legittimi.  Se avete ricevuto questa
> email per errore, siete pregati di informarne immediatamente il mittente e
> quindi cancellarla.  A meno che non siate stati a ciò espressamente
> autorizzati, la diffusione o la riproduzione di questa email o del suo
> contenuto non sono consentiti.
>
>  Salvo che questa email sia espressamente qualificata come offerta o
> accettazione contrattuale, il mittente non intende con questa email dare
> vita ad un vincolo giuridico e questa email non può essere interpretata
> quale offerta o accettazione che possa dare vita ad un contratto. Qualsiasi
> opinione manifestata in questa email è un'opinione personale del mittente,
> salvo che il mittente dichiari espressamente che si tratti di un'opinione di
> Prisma Engineering.
>
>
> ***
>
>  This e-mail (including any attachments) is private and confidential, and
> may be privileged.  It is for the exclusive use of the intended
> recipient(s).  If you have received this email in error, please inform the
> sender immediately and then delete this email.  Unless you have been given
> specific permission to do so, please do not distribute or copy this email or
> its contents.
>  Unless the text of this email specifically states that it is a contractual
> offer or acceptance, the sender does not intend to create a legal
> relationship and this email shall not constitute an offer or acceptance
> which could

Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10

2016-05-23 Thread Hugo Mills 
On Mon, May 23, 2016 at 08:49:04PM +0200, Zoiled wrote:
[snip]
> For what it's worth...  I have a 8 disk (7x 300GB + 1x 500GB) data
> raid10, metadata raid1 setup and I get the following output of
> btrfs...
> 
> Label: 'xxyyzz'  uuid: 12345678-9abc-def1-2345-6789abcdef01
> Total devices 8 FS bytes used 1.05TiB
> devid1 size 268.05GiB used 265.94GiB path /dev/sda1
> devid2 size 279.40GiB used 277.22GiB path /dev/sdb
> devid3 size 279.40GiB used 277.32GiB path /dev/sdc
> devid4 size 279.40GiB used 278.73GiB path /dev/sdd
> devid5 size 279.40GiB used 277.72GiB path /dev/sde
> devid6 size 279.40GiB used 278.61GiB path /dev/sdf
> devid7 size 279.40GiB used 278.82GiB path /dev/sdg
> devid8 size 465.76GiB used 230.99GiB path /dev/sdh
> 
> # btrfs filesystem usage -T /
> Overall:
> Device size:   2.35TiB
> Device allocated:  2.11TiB
> Device unallocated:  244.83GiB
> Device missing:  0.00B
> Used:  2.11TiB
> Free (estimated):122.57GiB  (min: 122.57GiB)
> Data ratio:   2.00
> Metadata ratio:   2.00
> Global reserve:  512.00MiB  (used: 0.00B)
> 
>  Data   Data  Metadata System
> Id Path  RAID1  RAID10RAID1RAID1 Unallocated
> -- - -- -  - ---
>  1 /dev/sda1  - 132.97GiB- - 135.08GiB
>  2 /dev/sdb   - 138.61GiB- - 140.79GiB
>  3 /dev/sdc   - 138.66GiB- - 140.74GiB
>  4 /dev/sdd   - 138.87GiB  1.00GiB - 139.53GiB
>  5 /dev/sde 1.00GiB 137.86GiB  1.00GiB - 139.53GiB
>  6 /dev/sdf   - 138.81GiB  1.00GiB - 139.59GiB
>  7 /dev/sdg 1.00GiB 138.38GiB  1.00GiB  64.00MiB 138.96GiB
>  8 /dev/sdh   - 113.46GiB  4.00GiB  64.00MiB 348.24GiB
> -- - -- -  - ---
>Total1.00GiB   1.05TiB  4.00GiB  64.00MiB 1.29TiB
> Used  1007.30MiB   1.05TiB  1.66GiB 400.00KiB
> 
> What I don't get is... how can I have 244.8 GB unallocated when the
> table below clearly shows that there is as much as 1.29TiB
> unallocated does not appear to make sense to me at least...

   This is exactly the issue. The Unallocated value(s) from btrfs fi
usage on at least RAID-10 are simply wrong, any way you look at it.

   Hugo.

-- 
Hugo Mills | Great films about cricket: The Third Man
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4  |


signature.asc
Description: Digital signature

Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10

2016-05-23 Thread Zoiled 

Marco Lorenzo Crociani wrote:

Hi,
as I wrote today in IRCI experienced an issue with 'btrfs filesystem 
usage'.

I have a 4 partitions RAID10 btrfs filesystem almost full.
'btrfs filesystem usage' reports wrong "Unallocated" indications.

Linux 4.5.3
btrfs-progs v4.5.3


# btrfs fi usage /data/

Overall:
Device size:  13.93TiB
Device allocated:  13.77TiB
Device unallocated: 167.54GiB
Device missing: 0.00B
Used:  13.44TiB
Free (estimated): 244.39GiB(min: 244.39GiB)
Data ratio:  2.00
Metadata ratio:  2.00
Global reserve: 512.00MiB(used: 0.00B)

Data,single: Size:8.00MiB, Used:0.00B
   /dev/sda4   8.00MiB

Data,RAID10: Size:6.87TiB, Used:6.71TiB
   /dev/sda4   1.72TiB
   /dev/sdb3   1.72TiB
   /dev/sdc3   1.72TiB
   /dev/sdd3   1.72TiB

Metadata,single: Size:8.00MiB, Used:0.00B
   /dev/sda4   8.00MiB

Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
   /dev/sda4   4.75GiB
   /dev/sdb3   4.75GiB
   /dev/sdc3   4.75GiB
   /dev/sdd3   4.75GiB

System,single: Size:4.00MiB, Used:0.00B
   /dev/sda4   4.00MiB

System,RAID10: Size:16.00MiB, Used:768.00KiB
   /dev/sda4   4.00MiB
   /dev/sdb3   4.00MiB
   /dev/sdc3   4.00MiB
   /dev/sdd3   4.00MiB

Unallocated:
   /dev/sda4   1.76TiB
   /dev/sdb3   1.76TiB
   /dev/sdc3   1.76TiB
   /dev/sdd3   1.76TiB

-- 


# btrfs fi show /data/
Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
Total devices 4 FS bytes used 6.72TiB
devid1 size 3.48TiB used 3.44TiB path /dev/sda4
devid2 size 3.48TiB used 3.44TiB path /dev/sdb3
devid3 size 3.48TiB used 3.44TiB path /dev/sdc3
devid4 size 3.48TiB used 3.44TiB path /dev/sdd3

-- 


# btrfs fi df /data/
Data, RAID10: total=6.87TiB, used=6.71TiB
Data, single: total=8.00MiB, used=0.00B
System, RAID10: total=16.00MiB, used=768.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID10: total=19.00GiB, used=14.15GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=512.00MiB, used=0.00B

-- 


# df -h
/dev/sda4 7,0T  6,8T245G  97% /data

Regards,

For what it's worth...  I have a 8 disk (7x 300GB + 1x 500GB) data 
raid10, metadata raid1 setup and I get the following output of btrfs...


Label: 'xxyyzz'  uuid: 12345678-9abc-def1-2345-6789abcdef01
Total devices 8 FS bytes used 1.05TiB
devid1 size 268.05GiB used 265.94GiB path /dev/sda1
devid2 size 279.40GiB used 277.22GiB path /dev/sdb
devid3 size 279.40GiB used 277.32GiB path /dev/sdc
devid4 size 279.40GiB used 278.73GiB path /dev/sdd
devid5 size 279.40GiB used 277.72GiB path /dev/sde
devid6 size 279.40GiB used 278.61GiB path /dev/sdf
devid7 size 279.40GiB used 278.82GiB path /dev/sdg
devid8 size 465.76GiB used 230.99GiB path /dev/sdh

# btrfs filesystem usage -T /
Overall:
Device size:   2.35TiB
Device allocated:  2.11TiB
Device unallocated:  244.83GiB
Device missing:  0.00B
Used:  2.11TiB
Free (estimated):122.57GiB  (min: 122.57GiB)
Data ratio:   2.00
Metadata ratio:   2.00
Global reserve:  512.00MiB  (used: 0.00B)

 Data   Data  Metadata System
Id Path  RAID1  RAID10RAID1RAID1 Unallocated
-- - -- -  - ---
 1 /dev/sda1  - 132.97GiB- - 135.08GiB
 2 /dev/sdb   - 138.61GiB- - 140.79GiB
 3 /dev/sdc   - 138.66GiB- - 140.74GiB
 4 /dev/sdd   - 138.87GiB  1.00GiB - 139.53GiB
 5 /dev/sde 1.00GiB 137.86GiB  1.00GiB - 139.53GiB
 6 /dev/sdf   - 138.81GiB  1.00GiB - 139.59GiB
 7 /dev/sdg 1.00GiB 138.38GiB  1.00GiB  64.00MiB 138.96GiB
 8 /dev/sdh   - 113.46GiB  4.00GiB  64.00MiB 348.24GiB
-- - -- -  - ---
   Total1.00GiB   1.05TiB  4.00GiB  64.00MiB 1.29TiB
Used  1007.30MiB   1.05TiB  1.66GiB 400.00KiB

What I don't get is... how can I have 244.8 GB unallocated when the 
table below clearly shows that there is as much as 1.29TiB 
unallocated does not appear to make sense to me at least...

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfs filesystem usage - Wrong Unallocated indications - RAID10

2016-05-23 Thread Nicholas D Steeves 
On 23 May 2016 at 09:34, Marco Lorenzo Crociani
 wrote:
> Hi,
> as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
> I have a 4 partitions RAID10 btrfs filesystem almost full.
> 'btrfs filesystem usage' reports wrong "Unallocated" indications.
>
> Linux 4.5.3
> btrfs-progs v4.5.3
>
>
> # btrfs fi usage /data/
>
> Overall:
> Device size:   13.93TiB
> Device allocated:   13.77TiB
> Device unallocated: 167.54GiB

I wonder if this is related to whatever caused the free space cache
bug for Ivan Pilipenko and myself (linux 4.4.10, btrfs-progs 4.4.1)?

Cheers,
Nicholas
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs filesystem usage - Wrong Unallocated indications - RAID10

2016-05-23 Thread Marco Lorenzo Crociani 

Hi,
as I wrote today in IRCI experienced an issue with 'btrfs filesystem usage'.
I have a 4 partitions RAID10 btrfs filesystem almost full.
'btrfs filesystem usage' reports wrong "Unallocated" indications.

Linux 4.5.3
btrfs-progs v4.5.3


# btrfs fi usage /data/

Overall:
Device size:  13.93TiB
Device allocated:  13.77TiB
Device unallocated: 167.54GiB
Device missing: 0.00B
Used:  13.44TiB
Free (estimated): 244.39GiB(min: 244.39GiB)
Data ratio:  2.00
Metadata ratio:  2.00
Global reserve: 512.00MiB(used: 0.00B)

Data,single: Size:8.00MiB, Used:0.00B
   /dev/sda4   8.00MiB

Data,RAID10: Size:6.87TiB, Used:6.71TiB
   /dev/sda4   1.72TiB
   /dev/sdb3   1.72TiB
   /dev/sdc3   1.72TiB
   /dev/sdd3   1.72TiB

Metadata,single: Size:8.00MiB, Used:0.00B
   /dev/sda4   8.00MiB

Metadata,RAID10: Size:19.00GiB, Used:14.15GiB
   /dev/sda4   4.75GiB
   /dev/sdb3   4.75GiB
   /dev/sdc3   4.75GiB
   /dev/sdd3   4.75GiB

System,single: Size:4.00MiB, Used:0.00B
   /dev/sda4   4.00MiB

System,RAID10: Size:16.00MiB, Used:768.00KiB
   /dev/sda4   4.00MiB
   /dev/sdb3   4.00MiB
   /dev/sdc3   4.00MiB
   /dev/sdd3   4.00MiB

Unallocated:
   /dev/sda4   1.76TiB
   /dev/sdb3   1.76TiB
   /dev/sdc3   1.76TiB
   /dev/sdd3   1.76TiB

--
# btrfs fi show /data/
Label: 'data'  uuid: df6639d5-3ef2-4ff6-a871-9ede440e2dae
Total devices 4 FS bytes used 6.72TiB
devid1 size 3.48TiB used 3.44TiB path /dev/sda4
devid2 size 3.48TiB used 3.44TiB path /dev/sdb3
devid3 size 3.48TiB used 3.44TiB path /dev/sdc3
devid4 size 3.48TiB used 3.44TiB path /dev/sdd3

--
# btrfs fi df /data/
Data, RAID10: total=6.87TiB, used=6.71TiB
Data, single: total=8.00MiB, used=0.00B
System, RAID10: total=16.00MiB, used=768.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID10: total=19.00GiB, used=14.15GiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=512.00MiB, used=0.00B

--
# df -h
/dev/sda4 7,0T  6,8T245G  97% /data

Regards,

--
Marco Crociani
Prisma Telecom Testing S.r.l.
via Petrocchi, 4  20127 MILANO  ITALY
Phone:  +39 02 26113507
Fax:  +39 02 26113597
e-mail:  mar...@prismatelecomtesting.com
web:  http://www.prismatelecomtesting.com

Questa email (e I suoi allegati) costituisce informazione riservata e 
confidenziale e può essere soggetto a legal privilege. Può essere utilizzata 
esclusivamente dai suoi destinatari legittimi.  Se avete ricevuto questa email 
per errore, siete pregati di informarne immediatamente il mittente e quindi 
cancellarla.  A meno che non siate stati a ciò espressamente autorizzati, la 
diffusione o la riproduzione di questa email o del suo contenuto non sono 
consentiti.

 Salvo che questa email sia espressamente qualificata come offerta o 
accettazione contrattuale, il mittente non intende con questa email dare vita 
ad un vincolo giuridico e questa email non può essere interpretata quale 
offerta o accettazione che possa dare vita ad un contratto. Qualsiasi opinione 
manifestata in questa email è un'opinione personale del mittente, salvo che il 
mittente dichiari espressamente che si tratti di un'opinione di Prisma 
Engineering.


***

 This e-mail (including any attachments) is private and confidential, and may 
be privileged.  It is for the exclusive use of the intended recipient(s).  If 
you have received this email in error, please inform the sender immediately and 
then delete this email.  Unless you have been given specific permission to do 
so, please do not distribute or copy this email or its contents.
 Unless the text of this email specifically states that it is a contractual 
offer or acceptance, the sender does not intend to create a legal relationship 
and this email shall not constitute an offer or acceptance which could give 
rise to a contract. Any views expressed in this communication are those of the 
individual sender, except where the sender specifically states them to be the 
views of Prisma Engineering.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html