Re: BTRFS critical: corrupt leaf, slot offset bad; then read-only

2017-02-21 Thread Lukas Tribus
Upgrading to 4.8, the FS no longer causes a kernel calltrace and does not go read-only. It only shows the "corrupt leaf, slot offset bad" message. A scrub completed without errors on 3 devices, while it was aborted on 2 devices. Not sure why it was aborted, since there is no error message in d

Re: [PATCH 1/4] btrfs: let writepage_end_io_hook return void

2017-02-21 Thread Liu Bo
On Mon, Feb 20, 2017 at 07:31:24PM +0100, David Sterba wrote: > There's no error path in any of the instances, always return 0. Reviewed-by: Liu Bo Thanks, -liubo > > Signed-off-by: David Sterba > --- > fs/btrfs/extent_io.c | 9 +++-- > fs/btrfs/extent_io.h | 2 +- > fs/btrfs/inode.c

Re: [PATCH 3/3] btrfs: do proper error handling in btrfs_insert_xattr_item

2017-02-21 Thread Liu Bo
On Mon, Feb 20, 2017 at 07:25:06PM +0100, David Sterba wrote: > The space check in btrfs_insert_xattr_item is duplicated in it's caller > (do_setxattr) so we won't hit the BUG_ON. Continuing without any check > could be disasterous so turn it to a proper error handling. > > Signed-off-by: David St

Re: [PATCH 2/3] btrfs: handle allocation error in update_dev_stat_item

2017-02-21 Thread Liu Bo
On Mon, Feb 20, 2017 at 07:25:04PM +0100, David Sterba wrote: > Signed-off-by: David Sterba > --- > fs/btrfs/volumes.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c > index 1fac98728814..64d6665f6eda 100644 > --- a/fs/btrfs/vol

Re: [PATCH 1/3] btrfs: remove BUG_ON from __tree_mod_log_insert

2017-02-21 Thread Liu Bo
On Mon, Feb 20, 2017 at 07:25:01PM +0100, David Sterba wrote: > All callers dereference the 'tm' parameter before it gets to this > function, the NULL check does not make much sense here. Reviewed-by: Liu Bo Thanks, -liubo > > Signed-off-by: David Sterba > --- > fs/btrfs/ctree.c | 2 -- > 1

[PATCH v3 2/2] btrfs: Handle delalloc error correctly to avoid ordered extent deadlock

2017-02-21 Thread Qu Wenruo
If run btrfs/125 with nospace_cache or space_cache=v2 mount option, btrfs will block with the following backtrace: Call Trace: __schedule+0x2d4/0xae0 schedule+0x3d/0x90 btrfs_start_ordered_extent+0x160/0x200 [btrfs] ? wake_atomic_t_function+0x60/0x60 btrfs_run_ordered_extent_work+0x25/0x40 [b

[PATCH v3 1/2] btrfs: ordered-extent: Introduce new bit to skip releasing metadata

2017-02-21 Thread Qu Wenruo
Introduce a new bit, BTRFS_ORDERED_SKIP_META for ordered extent to allow btrfs_finish_ordered_io() to skip releasing metadata. There are two sources for fill_delalloc() to release metadata: 1) extent_clear_unlock_delalloc() When EXTENT_DO_ACCOUNTING control bit is going to be cleared, we will

Re: [PATCH v2] btrfs: Handle delalloc error correctly to avoid deadlock

2017-02-21 Thread Qu Wenruo
At 02/22/2017 09:14 AM, Liu Bo wrote: On Tue, Feb 21, 2017 at 04:06:59PM +0800, Qu Wenruo wrote: If run btrfs/125 with nospace_cache or space_cache=v2 mount option, btrfs will block with the following backtrace: Call Trace: __schedule+0x2d4/0xae0 schedule+0x3d/0x90 btrfs_start_ordered_exte

Re: [PATCH v2] btrfs: Handle delalloc error correctly to avoid deadlock

2017-02-21 Thread Liu Bo
On Tue, Feb 21, 2017 at 04:06:59PM +0800, Qu Wenruo wrote: > If run btrfs/125 with nospace_cache or space_cache=v2 mount option, > btrfs will block with the following backtrace: > > Call Trace: > __schedule+0x2d4/0xae0 > schedule+0x3d/0x90 > btrfs_start_ordered_extent+0x160/0x200 [btrfs] > ? w

Re: [PATCH] Btrfs: try harder to migrate items to left sibling before splitting a leaf

2017-02-21 Thread Liu Bo
On Sun, Feb 19, 2017 at 08:56:39PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Before attempting to split a leaf we try to migrate items from the leaf to > its right and left siblings. We start by trying to move items into the > rigth sibling and, if the new item is meant to be in

[PATCH 1/1] btrfs-progs: docs: fix many typos, plus three edits for clarity

2017-02-21 Thread Nicholas D Steeves
Signed-off-by: Nicholas D Steeves --- Documentation/btrfs-balance.asciidoc | 2 +- Documentation/btrfs-check.asciidoc| 8 Documentation/btrfs-device.asciidoc | 6 +++--- Documentation/btrfs-filesystem.asciidoc | 6 +++--- Documentation/btrfs-inspe

[PATCH] btrfs-progs: docs: fix many typos, plus three edits for clarit

2017-02-21 Thread Nicholas D Steeves
Hi David, Please see attached a reasonably thorough patch for all the typos I could find in btrfs-progs documentation. The three edits are very minor and look larger than they are because I had to reflow the paragraphs. I'm confident in the quality of the work, with the exception of the followin

Re: FS gives kernel UPS on attempt to create snapshot and after running balance it's unmountable.

2017-02-21 Thread Tomasz Kusmierz
Anyone ? On 18 Feb 2017, at 16:44, Tomasz Kusmierz wrote: So Qu, currently my situation is that: I've tried to go btrfs scan --repair, and it did relair some stuff is qgroup's ... then tried to mont it and, surprise surpeire system locked out in 20 seconds. Reboot, again scan --repair = a lot

[PATCH] ioctl_getfsmap.2: document the GETFSMAP ioctl

2017-02-21 Thread Darrick J. Wong
Document the new GETFSMAP ioctl that returns the physical layout of a (disk-based) filesystem. This time around the fs-specific parts have been moved to a separate section; I'll move move them into separate xfsprogs/e2fsprogs manpages when we get closer to landing the ioctl. Signed-off-by: Darric

[PATCH] Btrfs: use the correct type when creating cow dio extent

2017-02-21 Thread Liu Bo
'BTRFS_ORDERED_REGULAR' was introduced for the cow case in patch 'Btrfs: specify a new ordered extent type for create_io_em', but it missed the directIO cow case. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/inode.c b/fs/

[PATCH] Btrfs: fix deadlock between dedup on same file and starting writeback

2017-02-21 Thread fdmanana
From: Filipe Manana If we are deduping two ranges of the same file we need to make sure that we lock all pages in ascending order, that is, lock first the pages from the range with lower offset and then the pages from the other range, as otherwise we can deadlock with a concurrent task that is st

BTRFS critical: corrupt leaf, slot offset bad; then read-only

2017-02-21 Thread Lukas Tribus
Hi list! I have btrfs pool consisting of 5x 2,72 TiB LUKS (dm-crypt) partitions in RAID1, mounted on Linux 4.4 with btrfs-progs 4.4. I never had any crashes or power loss here, but recently about every 60 - 120 minutes (while in use) btrfs detects corruptions, aborts the transaction and drop

Re: Large no.of extents for a single small file with a fresh RAID5 setup

2017-02-21 Thread Lakshmipathi.G
> > >I'm using Ubuntu(16.04) desktop version (not server) running Xorg and > >others. May be its possible its > >flushing data to disk at constant time. If you like to give it a try again > >with servers, below is the > >exact script along with its timing & multiextent output. > > Archlinux her

Re: [PATCH] btrfs-progs: RAID5:Inject data stripe corruption and verify scrub fixes it.

2017-02-21 Thread Lakshmipathi.G
> > > >Looked into patch description: > > > >After scrubbing dev3 only: > >0xcdcd (Good) | 0xcdcd | 0xcdcd (Bad) > >(D1) (D2)(P) > > > >So the Parity stripe (P) always get replaced by exact content of D1/D2 > >(data-stripe) > >or by random data? > > Neithe

Re: Large no.of extents for a single small file with a fresh RAID5 setup

2017-02-21 Thread Lakshmipathi.G
> >>How did you create the 2m file? > > > >Yes,this is the problem. script invokes 'dd' with 'notrunc' for each > >byte. If the file is 1kb, 'dd' invoked 1024 times with notrunc. > >This seems to be creating the issue. > > IIRC that's not the direct cause though. > > Since kernel is using delayed

[PATCH v3 01/12] btrfs-progs: lowmem check: Fix several bugs related to afterward search

2017-02-21 Thread Qu Wenruo
Since btrfs_search_slot() can points to the slot which is beyond the leaves' capacity, we should pay extra attention when doing afterward search. While for lowmem check, several places uses afterward search: 1) Block group item used space check 2) Device item used space check 3) Data extent backre

[PATCH v3 11/12] btrfs-progs: fsck: Fix lowmem mode override to allow it skip repair work

2017-02-21 Thread Qu Wenruo
From: Lu Fengqi Current common.local doesn't handle lowmem mode well. It passes "--mode=lowmem" alone with "--repair", making it unable to check lowmem mode. It's caused by the following bugs: 1) Wrong variable in test/common.local We should check TEST_ARGS_CHECK, not TEST_CHECK, which is no

[PATCH v3 12/12] btrfs-progs: cmds-check.c: walk_down_tree_v2 break cause of leaf process

2017-02-21 Thread Qu Wenruo
From: Su Yue In lowmem mode, 'walk_down_tree_v2' returns negative values wheather the error is fatal or not. It causes the loop where 'walk_down_tree_v2' is to break even the error is tolerated and then subsequent nodes process will be skipped. Fix it by redefining meanings of values 'walk_down_

[PATCH v3 08/12] btrfs-progs: fsck-test: Add test image for lowmem mode block group false alert

2017-02-21 Thread Qu Wenruo
Add a minimal image which can reproduce the block group used space false alert for lowmem mode fsck. Reported-by: Christoph Anton Mitterer Signed-off-by: Qu Wenruo --- .../block_group_item_false_alert.raw.xz | Bin 0 -> 47792 bytes tests/fsck-tests/020-extent-ref-cases/test.sh

[PATCH v3 10/12] btrfs-progs: fsck-test: Add new test case for file extent false alerts

2017-02-21 Thread Qu Wenruo
Lowmem mode exposed several false alerts, all related to file extents check. 1) Partly written prealloc extent Cause lowmem mode to report missing csum or prealloc extent should not have csum 2) Compressed inline extent Cause lowmem mode to find mismatch on inline len and item len. Wh

[PATCH v3 04/12] btrfs-progs: lowmem check: Fix extent item size false alert

2017-02-21 Thread Qu Wenruo
If one extent item has no inline ref, btrfs lowmem mode check can give false alert without outputting any error message. The problem is lowmem mode always assume that extent item has inline refs, and when it encounters such case it flags the extent item has wrong size, but doesn't output the error

[PATCH v3 05/12] btrfs-progs: lowmem check: Fix false alert on inline compressed extent

2017-02-21 Thread Qu Wenruo
Old lowmem check doesn't check if the inline extent is compressed and always check extent numbytes against inline item size. And when it finds the extent numbytes mismatch with inline item size it doesn't output any error message, just return error silently, making it quite hard to debug. Fix it

[PATCH v3 00/12] lowmem mode fixes

2017-02-21 Thread Qu Wenruo
Patches can be fetch from github: https://github.com/adam900710/btrfs-progs/tree/lowmem_fixes Thanks for reports from Chris Murphy and Christoph Anton Mitterer, several new bugs are exposed for lowmem mode fsck. Special thank to Christoph, who did rounds of test during the patch development. The

[PATCH v3 03/12] btrfs-progs: lowmem check: Fix false alert in checking data extent csums

2017-02-21 Thread Qu Wenruo
Btrfs lowmem check can report false csum error like: ERROR: root 5 EXTENT_DATA[257 0] datasum missing ERROR: root 5 EXTENT_DATA[257 4096] prealloc shouldn't have datasum This is because lowmem check code always compare the found csum size with the whole extent which data extents points to. Normal

[PATCH v3 06/12] btrfs-progs: lowmem check: Fix silent error if first inode item missing

2017-02-21 Thread Qu Wenruo
If first inode item is missing, lowmem check will detect it but not output any error message. Add error message for it. Signed-off-by: Qu Wenruo --- cmds-check.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/cmds-check.c b/cmds-check.c index cf5a08ce..7d273623 100644 --- a/cmds-check.c

[PATCH v3 07/12] btrfs-progs: tests: Move fsck-tests/015 to fuzz tests

2017-02-21 Thread Qu Wenruo
The test case fsck-tests/015-check-bad-memory-access can't be repair by btrfs check, and it's a fortunate bug makes original mode to forget the error code from extent tree, making original mode pass it. So fuzz-tests is more suitable for it. Signed-off-by: Qu Wenruo --- .../images}/bko-97171-bt

[PATCH v3 02/12] btrfs-progs: check: Output verbose error when fsck found a bug in any tree

2017-02-21 Thread Qu Wenruo
Although we output error like "errors found in extent allocation tree or chunk allocation", but we lacks such output for other trees, but leaving the final "found error is %d" to catch the last return value(and sometime it's cleared) This patch adds extra error message for top level error path, an

[PATCH v3 09/12] btrfs-progs: fsck-test: Make 013 compatible with lowmem mode

2017-02-21 Thread Qu Wenruo
fsck-tests/013-extent-tree-rebuild uses "--init-extent-tree", which implies "--repair". But the test script doesn't specify "--repair" for lowmem mode test to detect it. Add it so lowmem mode test can be happy with it. Signed-off-by: Qu Wenruo --- tests/fsck-tests/013-extent-tree-rebuild/test.

Re: mount troubles after crash

2017-02-21 Thread Patrick Schmid
Hi Qu A known bug when qgroup is enabled. Fixed in v4.10-rcs. Please mount using v4.10 kernels and it will mount without problem. I have rebootet with the v4.10 kernel and everything works again! Thank you very much, you save me a lot of work! Regards Patrick -- Patrick Schmid support:

[PATCH v2] btrfs: Handle delalloc error correctly to avoid deadlock

2017-02-21 Thread Qu Wenruo
If run btrfs/125 with nospace_cache or space_cache=v2 mount option, btrfs will block with the following backtrace: Call Trace: __schedule+0x2d4/0xae0 schedule+0x3d/0x90 btrfs_start_ordered_extent+0x160/0x200 [btrfs] ? wake_atomic_t_function+0x60/0x60 btrfs_run_ordered_extent_work+0x25/0x40 [b

[PATCH v2] btrfs: Handle delalloc error correctly to avoid deadlock

2017-02-21 Thread Qu Wenruo
If run btrfs/125 with nospace_cache or space_cache=v2 mount option, btrfs will block with the following backtrace: Call Trace: __schedule+0x2d4/0xae0 schedule+0x3d/0x90 btrfs_start_ordered_extent+0x160/0x200 [btrfs] ? wake_atomic_t_function+0x60/0x60 btrfs_run_ordered_extent_work+0x25/0x40 [b