Re: [PATCH v2] fstest: btrfs: test single 4k extent after subpagesize buffered writes

2016-03-21 Thread Eryu Guan
On Thu, Mar 17, 2016 at 03:56:38PM -0700, Liu Bo wrote: > This is to test if COW enabled btrfs can end up with single 4k extents > when doing subpagesize buffered writes. What happens if btrfs is mounted with "nodatacow" option? Does it need to _notrun if cow is disabled? > > The patch to fix th

Re: [PATCH] fstests: check for general protection fault traces in _check_dmesg

2016-03-21 Thread Eryu Guan
On Mon, Mar 21, 2016 at 07:23:38PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > While writing a test case for overlayfs I noticed that _check_dmesg() > did not detect general protection fault traces like the following: > > [15277.026108] run fstests overlay/002 at 2016-03-21 18:42

Re: [PATCH v2] fstests: test fsync on overlayfs merged directory

2016-03-21 Thread Eryu Guan
On Mon, Mar 21, 2016 at 05:50:25PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that calling fsync against a file using the merged directory does > not result in a crash nor fails unexpectedly. > > This is motivated by a change in overlayfs that resulted in a crash > (invalid

Re: [PATCH v8 15/27] btrfs: dedupe: Add ioctl for inband dedupelication

2016-03-21 Thread kbuild test robot
Hi Wang, [auto build test ERROR on btrfs/next] [also build test ERROR on next-20160321] [cannot apply to v4.5] [if your patch is applied to the wrong git tree, please drop us a note to help improving the system] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/Btrfs-Add-inband-write

Re: [GIT PULL] Btrfs

2016-03-21 Thread Linus Torvalds
On Mon, Mar 21, 2016 at 7:24 PM, Chris Mason wrote: > > Hmmm, rereading my answer I realized I didn't actually answer. I really > think this is fixed. I left the warning only because I originally > expected something much more exotic. Ok. It's more that you said the top commit fixes a problem,

Re: [PATCH v8 15/27] btrfs: dedupe: Add ioctl for inband dedupelication

2016-03-21 Thread kbuild test robot
Hi Wang, [auto build test ERROR on btrfs/next] [also build test ERROR on next-20160321] [cannot apply to v4.5] [if your patch is applied to the wrong git tree, please drop us a note to help improving the system] url: https://github.com/0day-ci/linux/commits/Qu-Wenruo/Btrfs-Add-inband-write

Re: [GIT PULL] Btrfs

2016-03-21 Thread Chris Mason
On Mon, Mar 21, 2016 at 10:15:33PM -0400, Chris Mason wrote: > On Mon, Mar 21, 2016 at 06:16:54PM -0700, Linus Torvalds wrote: > > On Mon, Mar 21, 2016 at 5:24 PM, Chris Mason wrote: > > > > > > I waited an extra day to send this one out because I hit a crash late > > > last week with CONFIG_DEBUG

Re: [GIT PULL] Btrfs

2016-03-21 Thread Chris Mason
On Mon, Mar 21, 2016 at 06:16:54PM -0700, Linus Torvalds wrote: > On Mon, Mar 21, 2016 at 5:24 PM, Chris Mason wrote: > > > > I waited an extra day to send this one out because I hit a crash late > > last week with CONFIG_DEBUG_PAGEALLOC enabled (fixed in the top commit). > > Hmm. If that commit

[PATCH v8 09/27] btrfs: dedupe: Inband in-memory only de-duplication implement

2016-03-21 Thread Qu Wenruo
Core implement for inband de-duplication. It reuse the async_cow_start() facility to do the calculate dedupe hash. And use dedupe hash to do inband de-duplication at extent level. The work flow is as below: 1) Run delalloc range for an inode 2) Calculate hash for the delalloc range at the unit of

[PATCH v8 13/27] btrfs: dedupe: Add support to delete hash for on-disk backend

2016-03-21 Thread Qu Wenruo
Now on-disk backend can delete hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedupe.c | 100 ++ 1 file changed, 100 insertions(+) diff --git a/fs/btrfs/dedupe.c b/fs/btrfs/dedupe.c index f73a4c7..c38137e 100644

[PATCH v8 00/27][For 4.7] Btrfs: Add inband (write time) de-duplication framework

2016-03-21 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/linux.git wang_dedupe_20160322 This updated version of inband de-duplication has the following features: 1) ONE unified dedup framework. Most of its code is hidden quietly in dedup.c and export the minimal interfaces fo

[PATCH v8 25/27] btrfs: dedupe: Add support for compression and dedpue

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang The basic idea is also calculate hash before compression, and add needed members for dedupe to record compressed file extent. Since dedupe support dedupe_bs larger than 128K, which is the up limit of compression file extent, in that case we will skip dedupe and prefer compre

[PATCH v8 12/27] btrfs: dedupe: Add support for on-disk hash search

2016-03-21 Thread Qu Wenruo
Now on-disk backend should be able to search hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedupe.c | 133 +++--- fs/btrfs/dedupe.h | 1 + 2 files changed, 118 insertions(+), 16 deletions(-) diff --git a/fs/btrfs

[PATCH v8 14/27] btrfs: dedupe: Add support for adding hash for on-disk backend

2016-03-21 Thread Qu Wenruo
Now on-disk backend can add hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedupe.c | 83 +++ 1 file changed, 83 insertions(+) diff --git a/fs/btrfs/dedupe.c b/fs/btrfs/dedupe.c index c38137e..6a80afc 100644 ---

[PATCH v8 16/27] btrfs: dedupe: add an inode nodedupe flag

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce BTRFS_INODE_NODEDUP flag, then we can explicitly disable online data dedupelication for specified files. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h | 1 + fs/btrfs/ioctl.c | 6 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/

[PATCH v8 10/27] btrfs: dedupe: Add basic tree structure for on-disk dedupe method

2016-03-21 Thread Qu Wenruo
Introduce a new tree, dedupe tree to record on-disk dedupe hash. As a persist hash storage instead of in-memeory only implement. Unlike Liu Bo's implement, in this version we won't do hack for bytenr -> hash search, but add a new type, DEDUP_BYTENR_ITEM for such search case, just like in-memory ba

[PATCH v8 22/27] btrfs: dedupe: Fix metadata balance error when dedupe is enabled

2016-03-21 Thread Qu Wenruo
A missing branch in btrfs_get_fs_root() is making dedupe_root read from disk, and REF_COWS bit set. This makes btrfs balance treating dedupe_root as fs root, and reusing the old dedupe root bytenr to drop tree ref, causing the following kernel warning after metadata balancing: BTRFS error (device

[PATCH v8 26/27] btrfs: relocation: Enhance error handling to avoid BUG_ON

2016-03-21 Thread Qu Wenruo
Since the introduce of btrfs dedupe tree, it's possible that balance can race with dedupe disabling. When this happens, dedupe_enabled will make btrfs_get_fs_root() return PTR_ERR(-ENOENT). But due to a bug in error handling branch, when this happens backref_cache->nr_nodes is increased but the no

[PATCH v8 21/27] btrfs: Fix a memory leak in inband dedupe hash

2016-03-21 Thread Qu Wenruo
We allocate a dedupe hash into async_extent, but forget to free it. Fix it by freeing the hash before freeing async_extent. Reported-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/inode.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index

[PATCH v8 15/27] btrfs: dedupe: Add ioctl for inband dedupelication

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Add ioctl interface for inband dedupelication, which includes: 1) enable 2) disable 3) status We will later add ioctl to disable inband dedupe for given file/dir. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/dedupe.c | 48 +

[PATCH v8 11/27] btrfs: dedupe: Introduce interfaces to resume and cleanup dedupe info

2016-03-21 Thread Qu Wenruo
Since we will introduce a new on-disk based dedupe method, introduce new interfaces to resume previous dedupe setup. And since we introduce a new tree for status, also add disable handler for it. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedupe.c | 269 ++

[PATCH v8 08/27] btrfs: ordered-extent: Add support for dedupe

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Add ordered-extent support for dedupe. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/ordered-data.c | 44 +

[PATCH v8 20/27] btrfs: dedupe: Fix a bug when running inband dedupe with balance

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang When running inband dedupe with balance, it's possible that inband dedupe still increase ref on extents which are in RO chunk. This may cause either find_data_references() gives warning, or make run_delayed_refs() return -EIO and cause trans abort. The cause is, normal dedu

[PATCH v8 06/27] btrfs: dedupe: Introduce function to search for an existing hash

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the btrfs_dedupe_search() interface. Signed-

[PATCH v8 02/27] btrfs: dedupe: Introduce function to initialize dedupe info

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Add generic function to initialize dedupe info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/Makefile | 2 +- fs/btrfs/dedupe.c | 97 +++ fs/btrfs/dedupe.h | 16 +++-- 3 files changed, 112 ins

[PATCH v8 24/27] btrfs: dedupe: Preparation for compress-dedupe co-work

2016-03-21 Thread Qu Wenruo
For dedupe to work with compression, new members recording compression algorithm and on-disk extent length are needed. Add them for later compress-dedupe co-work. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h| 11 - fs/btrfs/dedupe.c | 64 ++

[PATCH v8 04/27] btrfs: dedupe: Introduce function to remove hash from in-memory tree

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedupe tree. And implement btrfs_dedupe_del() and btrfs_dedup_destroy() interfaces. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/dedupe.c | 105 +

[PATCH v8 27/27] btrfs: dedupe: Fix a space cache delalloc bytes underflow bug

2016-03-21 Thread Qu Wenruo
Dedupe has a bug that underflow block_group_cache->delalloc_bytes, makes it unable to return to 0. This will cause free space cache for that block group never written to disk. And cause the following kernel message at umount: BTRFS info (device vdc): The free space cache file (1485570048) is inval

[PATCH v8 07/27] btrfs: dedupe: Implement btrfs_dedupe_calc_hash interface

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Unlike in-memory or on-disk dedupe method, only SHA256 hash method is supported yet, so implement btrfs_dedupe_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/dedupe.c | 49 +++

[PATCH v8 01/27] btrfs: dedupe: Introduce dedupe framework and its header

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce the header for btrfs online(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedupe methods and 1 dedupe hash. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h

[PATCH v8 18/27] btrfs: dedupe: add per-file online dedupe control

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce inode_need_dedupe() to implement per-file online dedupe control. Signed-off-by: Wang Xiaoguang --- fs/btrfs/inode.c | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 13ae366..979811c 1006

[PATCH v8 03/27] btrfs: dedupe: Introduce function to add hash into in-memory tree

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedupe_add() interface. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/dedupe.c | 162 ++ 1 f

[PATCH v8 05/27] btrfs: delayed-ref: Add support for increasing data ref under spinlock

2016-03-21 Thread Qu Wenruo
For in-band dedupe, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Signed-off-by: Qu Wenruo --- fs/btrfs/delayed-ref.c | 30 +++--- fs/btrfs/delayed

[PATCH v8 19/27] btrfs: try more times to alloc metadata reserve space

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang In btrfs_delalloc_reserve_metadata(), the number of metadata bytes we try to reserve is calculated by the difference between outstanding_extents and reserved_extents. When reserve_metadata_bytes() fails to reserve desited metadata space, it has already done some reclaim work

[PATCH v8 17/27] btrfs: dedupe: add a property handler for online dedupe

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang We use btrfs extended attribute "btrfs.dedupe" to record per-file online dedupe status, so add a dedupe property handler. Signed-off-by: Wang Xiaoguang --- fs/btrfs/props.c | 41 + 1 file changed, 41 insertions(+) diff --git a/fs/bt

[PATCH v8 23/27] btrfs: dedupe: Avoid submit IO for hash hit extent

2016-03-21 Thread Qu Wenruo
From: Wang Xiaoguang Before this patch, even for duplicated extent, it will still go through page write, meaning we didn't skip IO for them. Although such write will be skipped by block level, as block level will only select the last submitted write request to the same bytenr. This patch will m

Re: [GIT PULL] Btrfs

2016-03-21 Thread Linus Torvalds
On Mon, Mar 21, 2016 at 5:24 PM, Chris Mason wrote: > > I waited an extra day to send this one out because I hit a crash late > last week with CONFIG_DEBUG_PAGEALLOC enabled (fixed in the top commit). Hmm. If that commit helps, it will spit out a warning. So is it actually fixed, or just hacked

[GIT PULL] Btrfs

2016-03-21 Thread Chris Mason
Hi Linus, I waited an extra day to send this one out because I hit a crash late last week with CONFIG_DEBUG_PAGEALLOC enabled (fixed in the top commit). Please pull my my for-linus-4.6 branch: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-linus-4.6 We have a good siz

Re: Btrfs: fix up read_tree_block to return proper error

2016-03-21 Thread Liu Bo
Hi Dan, On Fri, Mar 18, 2016 at 08:37:13AM +0300, Dan Carpenter wrote: > Hello Liu Bo, > > The patch 64c043de466d: "Btrfs: fix up read_tree_block to return > proper error" from May 25, 2015, leads to the following static > checker warning: > > fs/btrfs/relocation.c:1858 replace_path() >

Re: [PATCH] Add support for RENAME_{EXCHANGE,WHITEOUT}

2016-03-21 Thread Dan Fuhry
On Mon, Mar 21, 2016 at 1:49 PM, David Sterba wrote: > > On Thu, Jan 28, 2016 at 10:52:28AM -0500, Dan Fuhry wrote: > > Add support for the RENAME_WHITEOUT and RENAME_EXCHANGE flags in > > renameat2(). This brings us pretty close to having btrfs ready to be > > an upper layer for overlayfs. (The l

[PATCH] Btrfs: fix invalid reference in replace_path

2016-03-21 Thread Liu Bo
Dan Carpenter's static checker has found this error, it's introduced by commit 64c043de466d ("Btrfs: fix up read_tree_block to return proper error") It's really supposed to 'break' the loop on error like others. Cc: Dan Carpenter Reported-by: Dan Carpenter Signed-off-by: Liu Bo --- fs/btrfs/

Re: [PATCH] Btrfs: fix performance regression of writing to prealloc/nocow file

2016-03-21 Thread Liu Bo
Hi, On Fri, Mar 18, 2016 at 05:44:03PM +0500, Roman Mamedov wrote: > On Thu, 17 Mar 2016 15:16:15 -0700 > Liu Bo wrote: > > > For nocow/prealloc files, we try our best to not allocate space, however, > > this ends up a huge performance regression since it's expensive to check > > if data is shar

FOR YOUR INFORMATION DEAR BENEFICIARY,

2016-03-21 Thread David Kedogo
FOR YOUR INFORMATION DEAR BENEFICIARY, Your Over-due ATM Card payment of $1.5MUSD by the UN Office have deposited to the ATM MASTER CARD OFFICE. All you have to do now is to contact the Office Manager Dr. peter Akupa Boni at: (peterakupa...@gmail.com) and Phone number: +229 61 31 07 78 , he will

[PULL] Btrfs, misc for 4.6

2016-03-21 Thread David Sterba
Hi, a few low-risk patches that appeared after I sent my main pull for 4.6. I think they still apply to 4.6, the freezer patches help the livepatch to clean up the mis-use of the freezer API, the rest makes sure we don't miss an error during writing checksums. I've removed Liu Bo's patch fixing t

Re: [PATCH 2/2] btrfs: do not write corrupted metadata blocks to disk

2016-03-21 Thread Filipe Manana
On Thu, Mar 10, 2016 at 11:10 AM, Alex Lyakas wrote: > csum_dirty_buffer was issuing a warning in case the extent buffer > did not look alright, but was still returning success. > Let's return error in this case, and also add an additional sanity > check on the extent buffer header. > The caller u

Re: [PATCH 1/2] btrfs: csum_tree_block: return proper errno value

2016-03-21 Thread Filipe Manana
On Thu, Mar 10, 2016 at 11:09 AM, Alex Lyakas wrote: > Signed-off-by: Alex Lyakas Reviewed-by: Filipe Manana > --- > fs/btrfs/disk-io.c | 13 + > 1 file changed, 5 insertions(+), 8 deletions(-) > > diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c > index 4545e2e..4420ab2 100644

Re: [PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread David Sterba
On Mon, Mar 21, 2016 at 02:08:00PM -0400, Chris Mason wrote: > On Mon, Mar 21, 2016 at 06:58:08PM +0100, David Sterba wrote: > > On Mon, Mar 21, 2016 at 01:51:07PM -0400, Chris Mason wrote: > > > On Mon, Mar 21, 2016 at 05:38:44PM +, fdman...@kernel.org wrote: > > > > From: Filipe Manana > > >

[PATCH] fstests: check for general protection fault traces in _check_dmesg

2016-03-21 Thread fdmanana
From: Filipe Manana While writing a test case for overlayfs I noticed that _check_dmesg() did not detect general protection fault traces like the following: [15277.026108] run fstests overlay/002 at 2016-03-21 18:42:27 [15277.933179] general protection fault: [#11] PREEMPT SMP DEBUG_PAGEALL

Re: [PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread Chris Mason
On Mon, Mar 21, 2016 at 06:58:08PM +0100, David Sterba wrote: > On Mon, Mar 21, 2016 at 01:51:07PM -0400, Chris Mason wrote: > > On Mon, Mar 21, 2016 at 05:38:44PM +, fdman...@kernel.org wrote: > > > From: Filipe Manana > > > > > > If the lower or upper directory of an overlayfs mount belong

Re: [PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread David Sterba
On Mon, Mar 21, 2016 at 01:51:07PM -0400, Chris Mason wrote: > On Mon, Mar 21, 2016 at 05:38:44PM +, fdman...@kernel.org wrote: > > From: Filipe Manana > > > > If the lower or upper directory of an overlayfs mount belong to a btrfs > > file system and we fsync the file through the overlayfs'

Re: [PATCH] btrfs-progs: fix fi du so it works in more cases

2016-03-21 Thread Austin S. Hemmelgarn
On 2016-03-21 13:40, David Sterba wrote: On Mon, Mar 21, 2016 at 08:23:11AM -0400, Austin S. Hemmelgarn wrote: Currently, btrfs fi du uses open_file_or_dir(), which tries to open it's argument with o_RDWR. Because of POSIX semantics, this fails for non-root users when the file is read-only or i

[PATCH] fstests: test fsync on overlayfs merged directory

2016-03-21 Thread fdmanana
From: Filipe Manana Test that calling fsync against a file using the merged directory does not result in a crash nor fails unexpectedly. This is motivated by a change in overlayfs that resulted in a crash (invalid memory access) when the lower or upper directory belonged to a btrfs file system.

Re: [PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread Filipe Manana
On Mon, Mar 21, 2016 at 5:51 PM, Chris Mason wrote: > On Mon, Mar 21, 2016 at 05:38:44PM +, fdman...@kernel.org wrote: >> From: Filipe Manana >> >> If the lower or upper directory of an overlayfs mount belong to a btrfs >> file system and we fsync the file through the overlayfs' merged direct

Re: confusing mountinfo output when bind-mounting files

2016-03-21 Thread Austin S. Hemmelgarn
On 2016-03-21 13:29, Tycho Andersen wrote: On Mon, Mar 21, 2016 at 11:22:06AM -0600, Chris Murphy wrote: On Mon, Mar 21, 2016 at 9:21 AM, Tycho Andersen wrote: Hi all, I'm seeing some strange behavior when bind mounting files from a btrfs subvolume. Consider the output below: root@criu2:/tmp

Re: [PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread Chris Mason
On Mon, Mar 21, 2016 at 05:38:44PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > If the lower or upper directory of an overlayfs mount belong to a btrfs > file system and we fsync the file through the overlayfs' merged directory > we ended up accessing an inode that didn't belong to

[PATCH v2] fstests: test fsync on overlayfs merged directory

2016-03-21 Thread fdmanana
From: Filipe Manana Test that calling fsync against a file using the merged directory does not result in a crash nor fails unexpectedly. This is motivated by a change in overlayfs that resulted in a crash (invalid memory access) when the lower or upper directory belonged to a btrfs file system.

Re: [PATCH] Add support for RENAME_{EXCHANGE,WHITEOUT}

2016-03-21 Thread David Sterba
On Thu, Jan 28, 2016 at 10:52:28AM -0500, Dan Fuhry wrote: > Add support for the RENAME_WHITEOUT and RENAME_EXCHANGE flags in > renameat2(). This brings us pretty close to having btrfs ready to be > an upper layer for overlayfs. (The last remaining issue is in > btrfs_sync_file, which I'm looking a

Re: [PATCH] btrfs-progs: fix fi du so it works in more cases

2016-03-21 Thread David Sterba
On Mon, Mar 21, 2016 at 08:23:11AM -0400, Austin S. Hemmelgarn wrote: > Currently, btrfs fi du uses open_file_or_dir(), which tries to open > it's argument with o_RDWR. Because of POSIX semantics, this fails for > non-root users when the file is read-only or is an executable that > is being run cu

[PATCH] Btrfs: fix crash/invalid memory access on fsync when using overlayfs

2016-03-21 Thread fdmanana
From: Filipe Manana If the lower or upper directory of an overlayfs mount belong to a btrfs file system and we fsync the file through the overlayfs' merged directory we ended up accessing an inode that didn't belong to btrfs as if it were a btrfs inode at btrfs_sync_file() resulting in a crash li

Re: [PATCH 1/2] btrfs-progs: copy btrfs-calc-size to inspect-internal tree-stats

2016-03-21 Thread David Sterba
On Fri, Mar 11, 2016 at 12:49:45PM +0100, Alexander Fougner wrote: > Signed-off-by: Alexander Fougner Both applied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.or

Re: confusing mountinfo output when bind-mounting files

2016-03-21 Thread Tycho Andersen
On Mon, Mar 21, 2016 at 11:22:06AM -0600, Chris Murphy wrote: > On Mon, Mar 21, 2016 at 9:21 AM, Tycho Andersen > wrote: > > Hi all, > > > > I'm seeing some strange behavior when bind mounting files from a btrfs > > subvolume. Consider the output below: > > > > root@criu2:/tmp# mount -o loop /tmp/

Re: confusing mountinfo output when bind-mounting files

2016-03-21 Thread Chris Murphy
On Mon, Mar 21, 2016 at 9:21 AM, Tycho Andersen wrote: > Hi all, > > I'm seeing some strange behavior when bind mounting files from a btrfs > subvolume. Consider the output below: > > root@criu2:/tmp# mount -o loop /tmp/tester.btrfs /tmp/dir1 > root@criu2:/tmp# touch dir1/file > root@criu2:/tmp# s

Re: overlay file to test btrfs repairs

2016-03-21 Thread Chris Murphy
On Mon, Mar 21, 2016 at 5:22 AM, Austin S. Hemmelgarn wrote: > On 2016-03-21 05:55, Duncan wrote: >> >> Chris Murphy posted on Sun, 20 Mar 2016 21:43:52 -0600 as excerpted: >> >>> Hi folks, >>> >>> So I just ran into this: >>> https://raid.wiki.kernel.org/index.php/ >> >> Recovering_a_failed_softw

Re: [PATCH] btrfs: make sure we stay inside the bvec during __btrfs_lookup_bio_sums

2016-03-21 Thread Chandan Rajendra
On Monday 21 Mar 2016 11:00:14 Chris Mason wrote: > Hi everyone, > > I realized last week that CONFIG_DEBUG_PAGEALLOC had dropped out of my > config, and hit a crash inside __btrfs_lookup_bio_sums once I enabled it > again. It's hard for this bug to cause problems because Chandan's inner > loop i

[PATCH V3] _btrfs_stress_subvolume: Fix race condition by making 'subvolume stress' task to exit gracefully

2016-03-21 Thread Chandan Rajendra
The following scenario can occur when running btrfs/066, Task ATask B Task C run_test() - Execute _btrfs_stress_subvolume() in a background shell. _btrfs_stress_subvolme()

confusing mountinfo output when bind-mounting files

2016-03-21 Thread Tycho Andersen
Hi all, I'm seeing some strange behavior when bind mounting files from a btrfs subvolume. Consider the output below: root@criu2:/tmp# mount -o loop /tmp/tester.btrfs /tmp/dir1 root@criu2:/tmp# touch dir1/file root@criu2:/tmp# sudo mount --bind dir1/file dir2/file root@criu2:/tmp# grep "/tmp/dir"

Re: [PATCH] btrfs-progs: fix fi du so it works in more cases

2016-03-21 Thread Martin Volf
Works for me, thanks. Martin Volf On Mon, Mar 21, 2016 at 1:23 PM, Austin S. Hemmelgarn wrote: > Currently, btrfs fi du uses open_file_or_dir(), which tries to open > it's argument with o_RDWR. Because of POSIX semantics, this fails for > non-root users when the file is read-only or is an execu

[PATCH] btrfs: make sure we stay inside the bvec during __btrfs_lookup_bio_sums

2016-03-21 Thread Chris Mason
Hi everyone, I realized last week that CONFIG_DEBUG_PAGEALLOC had dropped out of my config, and hit a crash inside __btrfs_lookup_bio_sums once I enabled it again. It's hard for this bug to cause problems because Chandan's inner loop is always done at the same time the outer loop is done. Withou

[PATCH] btrfs-progs: build: fix static standalone utilities

2016-03-21 Thread Noah Massey
commit b5e7979 "btrfs-progs: build: extend per-binary objects" allows the standalone utilities to link against object files shared with the main binary. However, the btrfs-*.static targets need to be adjusted to build against the static versions of the common files. Signed-off-by: Noah Massey ---

Re: [PATCH V2] _btrfs_stress_subvolume: Fix race condition by making 'subvolume stress' task to exit gracefully

2016-03-21 Thread Chandan Rajendra
On Monday 21 Mar 2016 20:07:06 Eryu Guan wrote: > > diff --git a/common/rc b/common/rc > > index 16f5a43..7d971ea 100644 > > --- a/common/rc > > +++ b/common/rc > > @@ -3280,9 +3280,10 @@ _btrfs_stress_subvolume() > > > > local btrfs_mnt=$2 > > local subvol_name=$3 > > local subvol_mnt

Re: btrfs fi du ERROR

2016-03-21 Thread Austin S. Hemmelgarn
On 2016-03-21 02:37, Martin Volf wrote: Hello, I have just tried the new "btrfs fi du" command from btrfs-progs 4.5 on 4.4.6 linux kernel, and it gave me: # btrfs fi du /bin Total Exclusive Set shared Filename (many lines of output for individual files, probably OK) ... ERROR: cannot

[PATCH] btrfs-progs: fix fi du so it works in more cases

2016-03-21 Thread Austin S. Hemmelgarn
Currently, btrfs fi du uses open_file_or_dir(), which tries to open it's argument with o_RDWR. Because of POSIX semantics, this fails for non-root users when the file is read-only or is an executable that is being run currently, or for all users (including root) when the filesystem is read-only.

Re: [PATCH V2] _btrfs_stress_subvolume: Fix race condition by making 'subvolume stress' task to exit gracefully

2016-03-21 Thread Eryu Guan
On Mon, Mar 21, 2016 at 01:09:52PM +0530, Chandan Rajendra wrote: > The following scenario can occur when running btrfs/066, > > Task ATask B Task C > > run_test() > - Execute _btrfs_stress_subvolume() > in a background shell. >

Re: overlay file to test btrfs repairs

2016-03-21 Thread Austin S. Hemmelgarn
On 2016-03-21 05:55, Duncan wrote: Chris Murphy posted on Sun, 20 Mar 2016 21:43:52 -0600 as excerpted: Hi folks, So I just ran into this: https://raid.wiki.kernel.org/index.php/ Recovering_a_failed_software_RAID#Making_the_harddisks_read- only_using_an_overlay_file [That's a single link, wr

Re: btrfs fi du ERROR

2016-03-21 Thread Austin S. Hemmelgarn
On 2016-03-21 02:37, Martin Volf wrote: Hello, I have just tried the new "btrfs fi du" command from btrfs-progs 4.5 on 4.4.6 linux kernel, and it gave me: # btrfs fi du /bin Total Exclusive Set shared Filename (many lines of output for individual files, probably OK) ... ERROR: cannot

Re: overlay file to test btrfs repairs

2016-03-21 Thread Duncan
Chris Murphy posted on Sun, 20 Mar 2016 21:43:52 -0600 as excerpted: > Hi folks, > > So I just ran into this: > https://raid.wiki.kernel.org/index.php/ Recovering_a_failed_software_RAID#Making_the_harddisks_read- only_using_an_overlay_file [That's a single link, wrapped by my client.] > This i

Re: Btrfsck memory usage reduce idea

2016-03-21 Thread Duncan
Qu Wenruo posted on Mon, 21 Mar 2016 10:15:55 +0800 as excerpted: > The point that I didn't want to keep the current behavior is, the old > one is just OK or OOM, no one would know if it will OOM until it > happens. > > But the new one would be much flex than current behavior. > As it fully uses

Re: [PATCH 1/1] Btrfs: Code Cleanup

2016-03-21 Thread Anand Jain
Hi Flex, diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 366b335..5c16f04 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -2325,7 +2325,10 @@ int btrfs_init_new_device(struct btrfs_root *root, char *device_path) if (seeding_dev) { sb->s_flags &=

[PATCH V2] _btrfs_stress_subvolume: Fix race condition by making 'subvolume stress' task to exit gracefully

2016-03-21 Thread Chandan Rajendra
The following scenario can occur when running btrfs/066, Task ATask B Task C run_test() - Execute _btrfs_stress_subvolume() in a background shell. _btrfs_stress_subvolme()

Re: [PATCH v2] btrfs-progs: subvolume functions reorg

2016-03-21 Thread Anand Jain
On 03/18/2016 09:31 PM, David Sterba wrote: On Fri, Mar 18, 2016 at 04:54:24PM +0800, Anand Jain wrote: On 03/17/2016 10:19 PM, David Sterba wrote: On Wed, Mar 16, 2016 at 10:14:33PM +0800, Anand Jain wrote: Makes few subvol related functions usable outside of subvol command set. This is

[PATCH] fstests: _fail the tests if _scratch_mount failed to avoid fully filling root fs

2016-03-21 Thread Eryu Guan
btrfs failed to mount small fs on ppc64 host with error ENOSPC, even creating such small fs succeeded, then generic/027 consumed all free space on root fs not on SCRATCH_DEV and test harness cannot create tmp files and continue other tests. Though I think it's a btrfs bug, it's still worth prevent

[PATCH 5/6] btrfs-progs: create get_subvol_info()

2016-03-21 Thread Anand Jain
get_subvol_info() is useful as we are adding more features around subvolume. This function was inline with the function cmd_subvol_show(). Signed-off-by: Anand Jain --- cmds-subvolume.c | 80 ++-- utils.c | 73 +

[PATCH 4/6] btrfs-progs: move get_subvol_name() to utils.c

2016-03-21 Thread Anand Jain
Signed-off-by: Anand Jain --- cmds-send.c | 11 --- utils.c | 12 utils.h | 1 + 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/cmds-send.c b/cmds-send.c index 3e34d75bb834..7605ed94cd64 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -335,17 +335,6 @

[PATCH 3/6] btrfs-progs: remove duplicate function __is_subvol()

2016-03-21 Thread Anand Jain
The function test_issubvolume() provides the same check, and has better logic. Signed-off-by: Anand Jain --- utils.c | 21 + 1 file changed, 1 insertion(+), 20 deletions(-) diff --git a/utils.c b/utils.c index 046ddf8eef19..110a4badb764 100644 --- a/utils.c +++ b/utils.c @@

[PATCH 0/6 v3] btrfs-progs: subvolume functions reorg

2016-03-21 Thread Anand Jain
We need subvolume helper functions easily accessible for features around subvolume. This patch set is just a cleanup of subvolume functions. This is tested fine with fstests group subvol and subvol hand tests. v3: Separate changes into 6 commits, Drops the idea of creating subvolume.c, instea

[PATCH 6/6] btrfs-progs: rename get_subvol_name() to subvol_minus_mnt()

2016-03-21 Thread Anand Jain
Signed-off-by: Anand Jain --- cmds-send.c | 4 ++-- utils.c | 4 ++-- utils.h | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/cmds-send.c b/cmds-send.c index 7605ed94cd64..339475927837 100644 --- a/cmds-send.c +++ b/cmds-send.c @@ -455,7 +455,7 @@ int cmd_send(int a

[PATCH 1/6] btrfs-progs: spatial rearrange subvolume functions together

2016-03-21 Thread Anand Jain
Signed-off-by: Anand Jain --- utils.c | 24 +--- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/utils.c b/utils.c index c0c564e2f69b..fb549ee923c7 100644 --- a/utils.c +++ b/utils.c @@ -2784,17 +2784,6 @@ int test_minimum_size(const char *file, u32 nodesize)

[PATCH 2/6] btrfs-progs: move test_issubvolume() to utils.c

2016-03-21 Thread Anand Jain
Signed-off-by: Anand Jain --- cmds-subvolume.c | 27 --- utils.c | 27 +++ utils.h | 1 + 3 files changed, 28 insertions(+), 27 deletions(-) diff --git a/cmds-subvolume.c b/cmds-subvolume.c index 3953d7c060a2..f62ab9584ce7 100644