Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote: On Sat, May 5, 2012 at 10:52 PM, Hugo Mills h...@carfax.org.uk wrote: On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote: Hello list, recently reformatted my home partition from XFS to RAID1 btrfs. I used the default options to mkfs.btrfs except for enabling raid1 for data as well as metadata. Filesystem is made up of two 1TB drives. mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 Total devices 2 FS bytes used 888.06GB devid 1 size 931.51GB used 931.51GB path /dev/sdb1 devid 2 size 931.51GB used 931.49GB path /dev/sdc1 Btrfs Btrfs v0.19 mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home Data, RAID1: total=893.48GB, used=886.60GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=38.00GB, used=2.54GB Metadata: total=8.00MB, used=0.00 As can be seen I don't have a lot of free space left and while I am planning on adding more storage soon I would like to gain a little breathing room until I can do this. While I don't have a lot of space remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. 2.5GB used out of 38GB. Does this excess become available automatically to the file system when the block groups in Data, RAID1 are exhausted or, if not, is there a way to manually reallocate them? Youre best bet at the moment is to try a partial balance of metadata chunks: # btrfs balance start -m /home That will rewrite all of your metadata, putting it through the allocator again, and removing the original allocated chunks. This should have the effect of reducing the allocation of metadata chunks. You will need a 3.3 kernel, or later, and an up-to-date userspace from cmason's git repository. Gave this a shot and it did help. mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ uname -r 3.3.4-2-ARCH mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ sudo ./btrfs balance start -m /home Done, had to relocate 40 out of 934 chunks mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ ./btrfs filesystem df /home Data, RAID1: total=900.97GB, used=880.06GB Data: total=8.00MB, used=0.00 System, RAID1: total=32.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=30.50GB, used=2.54GB There is now 8GB less in Metadata and I was able to delete some files as well to free up space. There is still a lot of wasted space in the metadata block groups. It seems that it allocates more metadata block groups than required for my filesystem. This will do until I am able to add a couple of devices to the system. Is there anyway to adjust the block group allocation strategy at filesystem creation? No. Chunk allocator currently allocates a lot more chunks than actually needed, and it impacts both balancing and normal operation. Try this: # btrfs balance start -musage=10 /home This is suboptimal, but it should get rid of more chunks. Thanks, Ilya -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 01:26:45PM +0300, Ilya Dryomov wrote: On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote: There is now 8GB less in Metadata and I was able to delete some files as well to free up space. There is still a lot of wasted space in the metadata block groups. It seems that it allocates more metadata block groups than required for my filesystem. This will do until I am able to add a couple of devices to the system. Is there anyway to adjust the block group allocation strategy at filesystem creation? No. Chunk allocator currently allocates a lot more chunks than actually needed, and it impacts both balancing and normal operation. Try this: # btrfs balance start -musage=10 /home This is suboptimal, but it should get rid of more chunks. While we're talking about it, what is the parameter to the usage option? I'm assuming it selects chunks which are less than some amount full -- but is the value a percentage, or a quantity in megabytes (power-of-10 or power-of-2), or something else? Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Doughnut furs ache me, Omar Dorlin. --- signature.asc Description: Digital signature
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sun, May 06, 2012 at 11:37:27AM +0100, Hugo Mills wrote: On Sun, May 06, 2012 at 01:26:45PM +0300, Ilya Dryomov wrote: On Sun, May 06, 2012 at 01:07:06PM +1000, Mike Sampson wrote: There is now 8GB less in Metadata and I was able to delete some files as well to free up space. There is still a lot of wasted space in the metadata block groups. It seems that it allocates more metadata block groups than required for my filesystem. This will do until I am able to add a couple of devices to the system. Is there anyway to adjust the block group allocation strategy at filesystem creation? No. Chunk allocator currently allocates a lot more chunks than actually needed, and it impacts both balancing and normal operation. Try this: # btrfs balance start -musage=10 /home This is suboptimal, but it should get rid of more chunks. While we're talking about it, what is the parameter to the usage option? I'm assuming it selects chunks which are less than some amount full -- but is the value a percentage, or a quantity in megabytes (power-of-10 or power-of-2), or something else? It's a percentage, so the command above will balance out chunks that are less than 10 percent full. I'll update btrfs man page and the wiki page you started as soon as I can. Thanks, Ilya -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Is it possible to reclaim block groups once they are allocated to data or metadata?
Hello list, recently reformatted my home partition from XFS to RAID1 btrfs. I used the default options to mkfs.btrfs except for enabling raid1 for data as well as metadata. Filesystem is made up of two 1TB drives. mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 Total devices 2 FS bytes used 888.06GB devid1 size 931.51GB used 931.51GB path /dev/sdb1 devid2 size 931.51GB used 931.49GB path /dev/sdc1 Btrfs Btrfs v0.19 mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home Data, RAID1: total=893.48GB, used=886.60GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=38.00GB, used=2.54GB Metadata: total=8.00MB, used=0.00 As can be seen I don't have a lot of free space left and while I am planning on adding more storage soon I would like to gain a little breathing room until I can do this. While I don't have a lot of space remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. 2.5GB used out of 38GB. Does this excess become available automatically to the file system when the block groups in Data, RAID1 are exhausted or, if not, is there a way to manually reallocate them? Regards, Mike -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote: Hello list, recently reformatted my home partition from XFS to RAID1 btrfs. I used the default options to mkfs.btrfs except for enabling raid1 for data as well as metadata. Filesystem is made up of two 1TB drives. mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 Total devices 2 FS bytes used 888.06GB devid1 size 931.51GB used 931.51GB path /dev/sdb1 devid2 size 931.51GB used 931.49GB path /dev/sdc1 Btrfs Btrfs v0.19 mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home Data, RAID1: total=893.48GB, used=886.60GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=38.00GB, used=2.54GB Metadata: total=8.00MB, used=0.00 As can be seen I don't have a lot of free space left and while I am planning on adding more storage soon I would like to gain a little breathing room until I can do this. While I don't have a lot of space remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. 2.5GB used out of 38GB. Does this excess become available automatically to the file system when the block groups in Data, RAID1 are exhausted or, if not, is there a way to manually reallocate them? Youre best bet at the moment is to try a partial balance of metadata chunks: # btrfs balance start -m /home That will rewrite all of your metadata, putting it through the allocator again, and removing the original allocated chunks. This should have the effect of reducing the allocation of metadata chunks. You will need a 3.3 kernel, or later, and an up-to-date userspace from cmason's git repository. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- All hope abandon, Ye who press Enter here. --- signature.asc Description: Digital signature
Re: Is it possible to reclaim block groups once they are allocated to data or metadata?
On Sat, May 5, 2012 at 10:52 PM, Hugo Mills h...@carfax.org.uk wrote: On Sat, May 05, 2012 at 10:37:01PM +1000, Mike Sampson wrote: Hello list, recently reformatted my home partition from XFS to RAID1 btrfs. I used the default options to mkfs.btrfs except for enabling raid1 for data as well as metadata. Filesystem is made up of two 1TB drives. mike@mercury (0) pts/3 ~ $ sudo btrfs filesystem show Label: none uuid: f08a8896-e03e-4064-9b94-9342fb547e47 Total devices 2 FS bytes used 888.06GB devid 1 size 931.51GB used 931.51GB path /dev/sdb1 devid 2 size 931.51GB used 931.49GB path /dev/sdc1 Btrfs Btrfs v0.19 mike@mercury (0) pts/3 ~ $ btrfs filesystem df /home Data, RAID1: total=893.48GB, used=886.60GB Data: total=8.00MB, used=0.00 System, RAID1: total=8.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=38.00GB, used=2.54GB Metadata: total=8.00MB, used=0.00 As can be seen I don't have a lot of free space left and while I am planning on adding more storage soon I would like to gain a little breathing room until I can do this. While I don't have a lot of space remaining in Data, RAID1 I do have a good chunk in Metadata, RAID1. 2.5GB used out of 38GB. Does this excess become available automatically to the file system when the block groups in Data, RAID1 are exhausted or, if not, is there a way to manually reallocate them? Youre best bet at the moment is to try a partial balance of metadata chunks: # btrfs balance start -m /home That will rewrite all of your metadata, putting it through the allocator again, and removing the original allocated chunks. This should have the effect of reducing the allocation of metadata chunks. You will need a 3.3 kernel, or later, and an up-to-date userspace from cmason's git repository. Gave this a shot and it did help. mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ uname -r 3.3.4-2-ARCH mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ sudo ./btrfs balance start -m /home Done, had to relocate 40 out of 934 chunks mike@mercury (0) pts/3 ~/.../btrfs-progs git:master $ ./btrfs filesystem df /home Data, RAID1: total=900.97GB, used=880.06GB Data: total=8.00MB, used=0.00 System, RAID1: total=32.00MB, used=136.00KB System: total=4.00MB, used=0.00 Metadata, RAID1: total=30.50GB, used=2.54GB There is now 8GB less in Metadata and I was able to delete some files as well to free up space. There is still a lot of wasted space in the metadata block groups. It seems that it allocates more metadata block groups than required for my filesystem. This will do until I am able to add a couple of devices to the system. Is there anyway to adjust the block group allocation strategy at filesystem creation? Thanks Hugo, Mike -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
