Re: [PATCH v2 02/10] btrfs-progs: Add support to suppress tree block csum error output.

2015-01-28 Thread Qu Wenruo
Original Message Subject: Re: [PATCH v2 02/10] btrfs-progs: Add support to suppress tree block csum error output. From: David Sterba To: Qu Wenruo Date: 2015年01月29日 02:16 On Mon, Jan 19, 2015 at 02:45:04PM +0800, Qu Wenruo wrote: Add new open ctree flag OPEN_CTREE_SUPPRESS

Re: [PATCH RFC v3 1/5] Revert "btrfs: add support for processing pending changes" related commits

2015-01-28 Thread David Sterba
On Mon, Jan 26, 2015 at 02:05:18PM +0800, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH RFC v3 1/5] Revert "btrfs: add support for > processing pending changes" related commits > From: Qu Wenruo > To: dste...@suse.cz, linux-btrfs@vger.kernel.org, miao...@huawei.co

[PATCH RESEND v4 0/8] Fix freeze/sysfs deadlock in better method.

2015-01-28 Thread Qu Wenruo
Patchset to solve the previous found freeze/sysfs deadlock. Unlike previous pending_changes, which uses transaction commits to ensure mount option doesn't change during transaction. This idea is great in concept, but will introduce extra and somewhat duplicated ro/freeze check, and the original sy

[PATCH RESEND v4 5/8] btrfs: Use btrfs_test_trans_opt() to handle INODE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h| 1 - fs/btrfs/disk-io.c | 4 fs/btrfs/free-space-cache.c |

Re: Get the diff from a file in two snapshots.

2015-01-28 Thread Stef Bon
Aha, thanks a lot Zygo for the hint about fiemap. I will try this, it could indeed save an awful lot of I/O. Stef Bon -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majo

[PATCH RESEND v4 1/8] Revert "btrfs: add support for processing pending changes" related commits

2015-01-28 Thread Qu Wenruo
This reverts commit 572d9ab7845 ~ a6f69dc8018. This pending commits patches introduce deadlock with freeze, and fix for it will introduce extra checks on freeze and read only case. For mount option change, later patches will introduce per-transaction mount option to keep mount options consistent

[PATCH RESEND v4 3/8] btrfs: Introduce per-transaction mount_opt to keep mount option consistent during transaction.

2015-01-28 Thread Qu Wenruo
Before this patch, mount_opt is not consistent during a transaction. btrfs_parse_options() can race with transaction. Now each transaction will keep a copy of fs_info->mount_opt upon creation, and new btrfs_test_trans_opt() macro is introduced to get the mount_opt in the transaction. Signed-off-b

[PATCH RESEND v4 8/8] btrfs: Use mnt_want_write() to protect sysfs feature change.

2015-01-28 Thread Qu Wenruo
Just like label change, use mnt_want_write() to do a correct protection. Signed-off-by: Qu Wenruo --- fs/btrfs/sysfs.c | 25 - 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index cdfa6b9..71e50ba 100644 --- a/fs/btrfs/s

[PATCH RESEND v4 4/8] btrfs: Use btrfs_test_trans_opt() to handle SPACE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/transaction.c | 7 --- 2 files changed, 5 insertions(+), 4 d

[PATCH RESEND v4 6/8] vfs: Add get_vfsmount_sb() function to get vfsmount from a given sb.

2015-01-28 Thread Qu Wenruo
There are sysfs interfaces in some fs, only btrfs yet, which will modify on-disk data. Unlike normal file operation routine we can use mnt_want_write_file() to protect the operation, change through sysfs won't to be binded to any file in the filesystem. So we can only extract the first vfsmount of

[PATCH RESEND v4 2/8] btrfs: Make btrfs_parse_options() parse mount option in a atomic way

2015-01-28 Thread Qu Wenruo
Current btrfs_parse_options() is not atomic, which can set and clear a bit, especially for nospace_cache case. For example, if a fs is mounted with nospace_cache, btrfs_parse_options() will set SPACE_CACHE bit first(since cache_generation is non-zeo) and clear the SPACE_CACHE bit due to nospace_ca

[PATCH RESEND v4 7/8] btrfs: Use mnt_want_write() to protect label change.

2015-01-28 Thread Qu Wenruo
Use the new vfs API get_vfsmount_sb() to get a vfsmount and use it to do the write protection of the label change transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/sysfs.c | 24 +++- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/s

Re: [PATCH 1/2] btrfs-progs: Make option parsing more robust to code modifications

2015-01-28 Thread Hugo Mills
On Wed, Jan 28, 2015 at 09:15:42AM +0800, Qu Wenruo wrote: > Original Message > Subject: [PATCH 1/2] btrfs-progs: Make option parsing more robust to > code modifications > From: Hugo Mills > To: , > Date: 2015年01月27日 23:05 > >The current approach to option parsing, where long-on

Re: [PATCH v4 0/8] Fix freeze/sysfs deadlock in better method.

2015-01-28 Thread Qu Wenruo
Oh, I'm very sorry for the patchset which includes other unrelated/out-of-date patches. I'll resend them with correct patchese. Thanks, Qu Original Message Subject: [PATCH v4 0/8] Fix freeze/sysfs deadlock in better method. From: Qu Wenruo To: Date: 2015年01月29日 09:27 Patchs

Indefinite hang in reserve_metadata_bytes on kernel 3.18.3

2015-01-28 Thread Steven Schlansker
[ Please CC me on responses, I am not subscribed to the list ] Hello linux-btrfs, I am running an cluster of Docker containers managed by Apache Mesos. Until recently, we'd found btrfs to be the most reliable storage backend for Docker. But now we are having troubles where large numbers of ou

[PATCH v4 2/8] btrfs: Introduce per-transaction mount_opt to keep mount option consistent during transaction.

2015-01-28 Thread Qu Wenruo
Before this patch, mount_opt is not consistent during a transaction. btrfs_parse_options() can race with transaction. Now each transaction will keep a copy of fs_info->mount_opt upon creation, and new btrfs_test_trans_opt() macro is introduced to get the mount_opt in the transaction. Signed-off-b

[PATCH v4 2/8] btrfs: Make btrfs_parse_options() parse mount option in a atomic way

2015-01-28 Thread Qu Wenruo
Current btrfs_parse_options() is not atomic, which can set and clear a bit, especially for nospace_cache case. For example, if a fs is mounted with nospace_cache, btrfs_parse_options() will set SPACE_CACHE bit first(since cache_generation is non-zeo) and clear the SPACE_CACHE bit due to nospace_ca

[PATCH v4 4/8] btrfs: Use btrfs_test_trans_opt() to handle INODE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h| 1 - fs/btrfs/disk-io.c | 4 fs/btrfs/free-space-cache.c |

[PATCH v4 5/8] btrfs: Use btrfs_test_trans_opt() to handle INODE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h| 1 - fs/btrfs/disk-io.c | 4 fs/btrfs/free-space-cache.c |

[PATCH v6] btrfs: add regression test for remount with thread_pool resized

2015-01-28 Thread Xing Gu
Regression test for a btrfs issue of resizing 'thread_pool' when remount the fs. This regression was introduced by the following linux kernel commit: btrfs: Added btrfs_workqueue_struct implemented ordered execution based on kernel workqueue 08a9ff3264181986d1d692a4e6fce3669700c9f8 And

[PATCH v4 3/8] btrfs: Use btrfs_test_trans_opt() to handle SPACE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/transaction.c | 7 --- 2 files changed, 5 insertions(+), 4 d

[PATCH v4 1/8] Revert "btrfs: add support for processing pending changes" related commits

2015-01-28 Thread Qu Wenruo
This reverts commit 572d9ab7845 ~ a6f69dc8018. This pending commits patches introduce deadlock with freeze, and fix for it will introduce extra checks on freeze and read only case. For mount option change, later patches will introduce per-transaction mount option to keep mount options consistent

[PATCH v4 5/8] btrfs: Make btrfs_parse_options() parse mount option in a atomic way

2015-01-28 Thread Qu Wenruo
Current btrfs_parse_options() is not atomic, which can set and clear a bit, especially for nospace_cache case. For example, if a fs is mounted with nospace_cache, btrfs_parse_options() will set SPACE_CACHE bit first(since cache_generation is non-zeo) and clear the SPACE_CACHE bit due to nospace_ca

[PATCH v4 8/8] btrfs: Use mnt_want_write() to protect sysfs feature change.

2015-01-28 Thread Qu Wenruo
Just like label change, use mnt_want_write() to do a correct protection. Signed-off-by: Qu Wenruo --- fs/btrfs/sysfs.c | 25 - 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index cdfa6b9..71e50ba 100644 --- a/fs/btrfs/s

[PATCH v4 0/8] Fix freeze/sysfs deadlock in better method.

2015-01-28 Thread Qu Wenruo
Patchset to solve the previous found freeze/sysfs deadlock. Unlike previous pending_changes, which uses transaction commits to ensure mount option doesn't change during transaction. This idea is great in concept, but will introduce extra and somewhat duplicated ro/freeze check, and the original sy

[PATCH v4 7/8] btrfs: Use mnt_want_write() to protect label change.

2015-01-28 Thread Qu Wenruo
Use the new vfs API get_vfsmount_sb() to get a vfsmount and use it to do the write protection of the label change transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/sysfs.c | 24 +++- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/s

[PATCH v4 3/8] btrfs: Introduce per-transaction mount_opt to keep mount option consistent during transaction.

2015-01-28 Thread Qu Wenruo
Before this patch, mount_opt is not consistent during a transaction. btrfs_parse_options() can race with transaction. Now each transaction will keep a copy of fs_info->mount_opt upon creation, and new btrfs_test_trans_opt() macro is introduced to get the mount_opt in the transaction. Signed-off-b

[PATCH v4 4/8] btrfs: Use btrfs_test_trans_opt() to handle SPACE_CACHE if it's under transaction protect.

2015-01-28 Thread Qu Wenruo
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under transaction protection. This will ensure SPACE_CACHE bit is consistent during transaction. Signed-off-by: Qu Wenruo --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/transaction.c | 7 --- 2 files changed, 5 insertions(+), 4 d

[PATCH v4 6/8] vfs: Add get_vfsmount_sb() function to get vfsmount from a given sb.

2015-01-28 Thread Qu Wenruo
There are sysfs interfaces in some fs, only btrfs yet, which will modify on-disk data. Unlike normal file operation routine we can use mnt_want_write_file() to protect the operation, change through sysfs won't to be binded to any file in the filesystem. So we can only extract the first vfsmount of

Re: [PATCH 0/4] Better btrfsck tree corruption report and automatic csum tree rebuild.

2015-01-28 Thread Qu Wenruo
Original Message Subject: Re: [PATCH 0/4] Better btrfsck tree corruption report and automatic csum tree rebuild. From: David Sterba To: Qu Wenruo Date: 2015年01月29日 02:27 On Tue, Jan 13, 2015 at 10:04:43AM +0800, Qu Wenruo wrote: Although btrfsck can rebuild the csum tree,

Re: [PATCH RFC v3 1/5] Revert "btrfs: add support for processing pending changes" related commits

2015-01-28 Thread Qu Wenruo
Original Message Subject: Re: [PATCH RFC v3 1/5] Revert "btrfs: add support for processing pending changes" related commits From: David Sterba To: Qu Wenruo Date: 2015年01月28日 21:25 On Mon, Jan 26, 2015 at 02:05:18PM +0800, Qu Wenruo wrote: Original Message ---

Re: Am I missing partiy/redundancy - WARNING: RAID56 detected, not implemented

2015-01-28 Thread David Sterba
On Tue, Jan 27, 2015 at 02:21:24AM -0700, Jeff Smith wrote: > I am just getting started with btrfs and wanted to test raid5. I set > it up and thought it was setup correctly but when I run " btrfs fi > usage /mnt/btrdata" I get the message "WARNING: RAID56 detected, not > implemented". Does thi

Re: btrfs-prog: improve build-system by autoconf

2015-01-28 Thread David Sterba
On Fri, Jan 16, 2015 at 04:26:13PM +0100, Karel Zak wrote: > On Wed, Dec 17, 2014 at 03:07:26PM +0100, David Sterba wrote: > > On Fri, Dec 12, 2014 at 01:35:14PM +0100, Karel Zak wrote: > > > This is first step to make btrfs-progs build system more conventional > > > for userspace users and develop

Re: [PATCH 1/1] btrfs: Align EOF length to block in extent_same

2015-01-28 Thread David Sterba
On Mon, Jan 26, 2015 at 06:05:51PM +, Matt Robinson wrote: > It is not currently possible to deduplicate the last block of files > whose size is not a multiple of the block size, as the btrfs_extent_same > ioctl returns -EINVAL if offset + size is greater than the file size or > is not aligned

Re: [PATCH v2 02/10] btrfs-progs: Add support to suppress tree block csum error output.

2015-01-28 Thread David Sterba
On Mon, Jan 19, 2015 at 02:45:04PM +0800, Qu Wenruo wrote: > Add new open ctree flag OPEN_CTREE_SUPPRESS_ERROR to suppress tree block > csum error output. > > @@ -996,6 +996,7 @@ struct btrfs_fs_info { > unsigned int on_restoring:1; > unsigned int is_chunk_recover:1; > unsigned i

Linux documentation: btrfs.txt

2015-01-28 Thread Pavel Volkov
In my 3.18.3 sources the file at Documentation/filesystems/btrfs.txt says: "Btrfs is under heavy development, and is not suitable for any uses other than benchmarking and review. The Btrfs disk format is not yet finalized." btrfs is no longer marked experimental in kernel and the file format is

Re: [PATCH 0/4] Better btrfsck tree corruption report and automatic csum tree rebuild.

2015-01-28 Thread David Sterba
On Tue, Jan 13, 2015 at 10:04:43AM +0800, Qu Wenruo wrote: > Although btrfsck can rebuild the csum tree, but has the following > problems for end users or sysadmins who is not familiar with btrfs. > 1) No brief info on which tree is corrupted. >In fact, after extent and chunk tree check, we ite

Re: [PATCH 1/1] btrfs: Align EOF length to block in extent_same

2015-01-28 Thread Matt Robinson
On 28 January 2015 at 12:55, David Sterba wrote: > On Mon, Jan 26, 2015 at 06:05:51PM +, Matt Robinson wrote: >> It is not currently possible to deduplicate the last block of files >> whose size is not a multiple of the block size, as the btrfs_extent_same >> ioctl returns -EINVAL if offset +