Sorry, fighting with this technology called "email" :)
Hopefully better wrapped outputs:
On 13. 03. 19 22:58, Jakub Husák wrote:
Hi,
I added another disk to my 3-disk raid5 and ran a balance command.
After few hours I looked to output of `fi usage` to see that no data
are being used on the new disk. I got the same result even when
balancing my raid5 data or metadata.
Next I tried to convert my raid5 metadata to raid1 (a good idea
anyway) and the new disk started to fill immediately (even though it
received the whole amount of metadata with replicas being spread among
the other drives, instead of being really "balanced". I know why this
happened, I don't like it but I can live with it, let's not go off
topic here :)).
Now my usage output looks like this:
# btrfs filesystem usage /mnt/data1
WARNING: RAID56 detected, not implemented
Overall:
Device size: 10.91TiB
Device allocated: 316.12GiB
Device unallocated: 10.61TiB
Device missing: 0.00B
Used: 58.86GiB
Free (estimated): 0.00B (min: 8.00EiB)
Data ratio: 0.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Data,RAID5: Size:4.59TiB, Used:4.06TiB
/dev/mapper/crypt-sdb 2.29TiB
/dev/mapper/crypt-sdc 2.29TiB
/dev/mapper/crypt-sde 2.29TiB
Metadata,RAID1: Size:158.00GiB, Used:29.43GiB
/dev/mapper/crypt-sdb 53.00GiB
/dev/mapper/crypt-sdc 53.00GiB
/dev/mapper/crypt-sdd 158.00GiB
/dev/mapper/crypt-sde 52.00GiB
System,RAID1: Size:64.00MiB, Used:528.00KiB
/dev/mapper/crypt-sdc 32.00MiB
/dev/mapper/crypt-sdd 64.00MiB
/dev/mapper/crypt-sde 32.00MiB
Unallocated:
/dev/mapper/crypt-sdb 393.04GiB
/dev/mapper/crypt-sdc 393.01GiB
/dev/mapper/crypt-sdd 2.57TiB
/dev/mapper/crypt-sde 394.01GiB
I'm now running `fi balance -dusage=10` (and rising the usage limit).
I can see that the unallocated space is rising as it's freeing the
little used chunks but still no data are being stored on the new disk.
I it some bug? Is `fi usage` not showing me something (as it states
"WARNING: RAID56 detected, not implemented")? Or is there just too
much free space on the first set of disks that the balancing is not
bothering moving any data?
If so, shouldn't it be really balancing (spreading) the data among all
the drives to use all the IOPS capacity, even when the raid5
redundancy constraint is currently satisfied?
# uname -a
Linux storage 4.19.0-0.bpo.2-amd64 #1 SMP Debian 4.19.16-1~bpo9+1
(2019-02-07) x86_64 GNU/Linux
# btrfs --version
btrfs-progs v4.17
# btrfs fi show
Label: none uuid: xxxxxxxxxxxxxxxxx
Total devices 4 FS bytes used 4.09TiB
devid 2 size 2.73TiB used 2.34TiB path /dev/mapper/crypt-sdc
devid 3 size 2.73TiB used 2.34TiB path /dev/mapper/crypt-sdb
devid 4 size 2.73TiB used 2.34TiB path /dev/mapper/crypt-sde
devid 5 size 2.73TiB used 158.06GiB path /dev/mapper/crypt-sdd
# btrfs fi df .
Data, RAID5: total=4.59TiB, used=4.06TiB
System, RAID1: total=64.00MiB, used=528.00KiB
Metadata, RAID1: total=158.00GiB, used=29.43GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
Thanks
Jakub
