[PATCH] fstests: speedup generic/027 for new version of btrfs

2015-11-24 Thread Zhaolei
From: Zhao Lei New version of btrfs create non-mixed blockgroups in all case. For generic/027, the filesystem in test is convert from mixed-blockgroup to non-mixed blockgroup. And test time is changed from 400s -> 2700s in my node. To test btrfs with all mountoptions, this testitem need about 7

[PATCH] fstests: Fix generic/102 fail for btrfs

2015-12-03 Thread Zhaolei
From: Zhao Lei generic/102 sometimes fails in newest btrfs toolchain, because it use non-mixed mode in default, which request more space for metadata, and no space for data writing. This patch force mixed mode for btrfs in generic/102. Signed-off-by: Zhao Lei --- tests/generic/102 | 2 ++ 1 f

[PATCH] btrfs: wait for delayed iputs on no space

2015-03-27 Thread Zhaolei
From: Zhao Lei This is another fix of no_space case. All patchs for fix no_space bug are available at fix_no_space branch on: git://github.com/zhaoleidd/btrfs Any suggestions are welcome. Zhao Lei (1): btrfs: wait for delayed iputs on no space fs/btrfs/extent-tree.c | 3 +++ 1 file chang

[PATCH] btrfs: wait for delayed iputs on no space

2015-03-27 Thread Zhaolei
From: Zhao Lei btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f "$DEV" # mount -o nodatacow "$DEV" "$MNT" # for ((i = 0; i < 100; i++)); do dd if=/dev/zero of="$MNT"/file0

[PATCH 0/9] btrfs: Fix no_space on dd and rm loop

2015-04-03 Thread Zhaolei
n v4.0-rc5 and no-problem on top of this patchset. I'll add xfstests for this case later. This is available at fix_no_space branch on my tree: git://github.com/zhaoleidd/btrfs.git It is also included in integration-for-chris branch in above tree. Thanks Zhaolei Zhao Lei (9): btrfs: f

[PATCH 9/9] btrfs: cleanup unused alloc_chunk varible

2015-04-03 Thread Zhaolei
From: Zhao Lei Remove int alloc_chunk in btrfs_check_data_free_space() for not necessary. Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 05747d2..b83060f 100644 ---

[PATCH 8/9] btrfs: wait for delayed iputs on no space

2015-04-03 Thread Zhaolei
From: Zhao Lei btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f "$DEV" # mount -o nodatacow "$DEV" "$MNT" # for ((i = 0; i < 100; i++)); do dd if=/dev/zero of="$MNT"/file0

[PATCH 2/9] btrfs: Fix tail space processing in find_free_dev_extent()

2015-04-03 Thread Zhaolei
From: Zhao Lei It is another reason for NO_SPACE case. When we found enough free space in loop and saved them to max_hole_start/size before, and tail space contains pending extent, origional innocent max_hole_start/size are reset in retry. As a result, find_free_dev_extent() returns less space

[PATCH 6/9] btrfs: Fix NO_SPACE bug caused by delayed-iput

2015-04-03 Thread Zhaolei
From: Zhao Lei Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call btrfs_run_delayed_iputs() in end

[PATCH 3/9] btrfs: Adjust commit-transaction condition to avoid NO_SPACE more

2015-04-03 Thread Zhaolei
From: Zhao Lei If we have any chance to make a successful write, we should not give up. This patch adjust commit-transaction condition from: pinned >= wanted to left + pinned >= wanted Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(

[PATCH 5/9] btrfs: add WARN_ON() to check is space_info op current

2015-04-03 Thread Zhaolei
From: Zhao Lei space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1->v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 10 ++ 1 file chan

[PATCH 7/9] btrfs: Support busy loop of write and delete

2015-04-03 Thread Zhaolei
From: Zhao Lei Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We already have commit_transac

[PATCH 4/9] btrfs: Set relative data on clear btrfs_block_group_cache->pinned

2015-04-03 Thread Zhaolei
From: Zhao Lei Bug1: space_info->bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache->pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info->bytes_readonly. Then in btrfs_remove

[PATCH 1/9] btrfs: fix condition of commit transaction

2015-04-03 Thread Zhaolei
From: Zhao Lei Old code bypass commit transaction when we don't have enough pinned space, but another case is there exist freed bgs in current transction, it have possibility to make alloc_chunk success. This patch modify the condition to: if (have_free_bg || have_pinned_space) commit_transactio

[PATCH 8/9] btrfs: wait for delayed iputs on no space

2015-04-08 Thread Zhaolei
From: Zhao Lei btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f "$DEV" # mount -o nodatacow "$DEV" "$MNT" # for ((i = 0; i < 100; i++)); do dd if=/dev/zero of="$MNT"/file0

[PATCH 3/9] btrfs: Adjust commit-transaction condition to avoid NO_SPACE more

2015-04-08 Thread Zhaolei
From: Zhao Lei If we have any chance to make a successful write, we should not give up. This patch adjust commit-transaction condition from: pinned >= wanted to left + pinned >= wanted Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(

[PATCH 7/9] btrfs: Support busy loop of write and delete

2015-04-08 Thread Zhaolei
From: Zhao Lei Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We already have commit_transac

[PATCH 4/9] btrfs: Set relative data on clear btrfs_block_group_cache->pinned

2015-04-08 Thread Zhaolei
From: Zhao Lei Bug1: space_info->bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache->pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info->bytes_readonly. Then in btrfs_remove

[PATCH v2 0/9] btrfs: Fix no_space on dd and rm loop

2015-04-08 Thread Zhaolei
xfstests for this case later. This is available at fix_no_space branch on my tree: git://github.com/zhaoleidd/btrfs.git It is also included in integration-for-chris branch in above tree. Thanks Zhaolei Zhao Lei (9): btrfs: fix condition of commit transaction btrfs: Fix tail space proces

[PATCH 9/9] btrfs: cleanup unused alloc_chunk varible

2015-04-08 Thread Zhaolei
From: Zhao Lei Remove int alloc_chunk in btrfs_check_data_free_space() for not necessary. Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index d5ec383..b009987 100644 ---

[PATCH 5/9] btrfs: add WARN_ON() to check is space_info op current

2015-04-08 Thread Zhaolei
From: Zhao Lei space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1->v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba Signed-off-by: Zhao Lei --- fs/btrfs/extent-tree.c | 10 ++ 1 file chan

[PATCH 6/9] btrfs: Fix NO_SPACE bug caused by delayed-iput

2015-04-08 Thread Zhaolei
From: Zhao Lei Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call btrfs_run_delayed_iputs() in end

[PATCH 2/9] btrfs: Fix tail space processing in find_free_dev_extent()

2015-04-08 Thread Zhaolei
From: Zhao Lei It is another reason for NO_SPACE case. When we found enough free space in loop and saved them to max_hole_start/size before, and tail space contains pending extent, origional innocent max_hole_start/size are reset in retry. As a result, find_free_dev_extent() returns less space

[PATCH 1/9] btrfs: fix condition of commit transaction

2015-04-08 Thread Zhaolei
From: Zhao Lei Old code bypass commit transaction when we don't have enough pinned space, but another case is there exist freed bgs in current transction, it have possibility to make alloc_chunk success. This patch modify the condition to: if (have_free_bg || have_pinned_space) commit_transactio

[PATCH 1/2] btrfs: Use ref_cnt for set_block_group_ro()

2015-05-21 Thread Zhaolei
From: Zhao Lei More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename set_block_group_ro/set_block_gr

[PATCH 2/2] btrfs: Fix xfstests btrfs/070

2015-05-21 Thread Zhaolei
From: Zhao Lei xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found by scrub. Actually, it have

[PATCH 4/4] btrfs: Fix data checksum error cause by replace with io-load.

2015-05-29 Thread Zhaolei
From: Zhao Lei xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found by scrub. Actually, i

[PATCH 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

2015-05-29 Thread Zhaolei
From: Zhao Lei It can reduce current duplicated code which is similar to scrub_blocked_if_needed() but can not call it because little different. It also used by my next patch which is in same case. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 11 ++- 1 file changed, 10 insertions(+),

[PATCH 1/4] btrfs: Use ref_cnt for set_block_group_ro()

2015-05-29 Thread Zhaolei
From: Zhao Lei More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename set_block_group_ro/set_block_gr

[PATCH 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

2015-05-29 Thread Zhaolei
From: Zhao Lei Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index a3d1546..8da3459 100644

[PATCH] btrfs: Fix lockdep warning of wr_ctx->wr_lock in scrub_free_wr_ctx()

2015-06-02 Thread Zhaolei
From: Zhao Lei lockdep report following warning in test: [25176.843958] = [25176.844519] [ INFO: inconsistent lock state ] [25176.845047] 4.1.0-rc3 #22 Tainted: GW [25176.845591] - [25176.846153] inconsistent {SOFTIRQ-ON

[PATCH v2] btrfs: Fix lockdep warning of wr_ctx->wr_lock in scrub_free_wr_ctx()

2015-06-04 Thread Zhaolei
From: Zhao Lei lockdep report following warning in test: [25176.843958] = [25176.844519] [ INFO: inconsistent lock state ] [25176.845047] 4.1.0-rc3 #22 Tainted: GW [25176.845591] - [25176.846153] inconsistent {SOFTIRQ-ON

[PATCH] btrfs: cleanup noused initialization of dev in btrfs_end_bio()

2015-06-08 Thread Zhaolei
From: Zhao Lei It is introduced by: c404e0dc2c843b154f9a36c3aec10d0a715d88eb Btrfs: fix use-after-free in the finishing procedure of the device replace But seems no relationship with that bug, this patch revirt these code block for cleanup. Signed-off-by: Zhao Lei --- fs/btrfs/volumes.c | 2

[PATCH] btrfs: Update out-of-date "skip parity stripe" comment

2015-06-10 Thread Zhaolei
From: Zhao Lei Because btrfs support scrub raid56 parity stripe now. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index a13f91a..5ee5630 100644 --- a/fs/btrfs/scrub.c +++ b/fs/btrfs/scrub.c

[PATCH] btrfs: [RFC] Don't use workqueue for raid56 parity scrub

2015-06-12 Thread Zhaolei
From: Zhao Lei The code in workqueue only do fast initialization and submit a bio in end, plus, it will not called in interrupe context, no need to queue a work for this type of work. Call it directly will make code simple, easy to debug and reduce potential problem. Signed-off-by: Zhao Lei --

[PATCH] btrfs: add error handling for scrub_workers_get()

2015-06-12 Thread Zhaolei
From: Zhao Lei Although it is a rare case, we'd better free previous allocated memory on error. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 31 --- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index ab58115..

[PATCH v3 0/4] btrfs: Fix data checksum error cause by replace with io-load

2015-07-07 Thread Zhaolei
From: Zhao Lei This patchset is used to fix data checksum error cause by replace with io-load. It cause xfstests btrfs/070 and btrfs/071 sometimes failed. See description in [PATCH 4/4] for detail. Changelog v2->v3: 1: Fix a typo(caused in rebase) which make xfstests failed in btrfs/073 and

[PATCH v3 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

2015-07-07 Thread Zhaolei
From: Zhao Lei It can reduce current duplicated code which is similar to scrub_blocked_if_needed() but can not call it because little different. It also used by my next patch which is in same case. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 11 ++- 1 file changed, 10 insertions(+),

[PATCH v3 1/4] btrfs: Use ref_cnt for set_block_group_ro()

2015-07-07 Thread Zhaolei
From: Zhao Lei More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename set_block_group_ro/set_block_gr

[PATCH v3 4/4] btrfs: Fix data checksum error cause by replace with io-load.

2015-07-07 Thread Zhaolei
From: Zhao Lei xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found by scrub. Actually, i

[PATCH v3 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

2015-07-07 Thread Zhaolei
From: Zhao Lei Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index cbfb8c7..a882a34 100644

[PATCH] btrfs: Remove noused chunk_tree and chunk_objectid from scrub_enumerate_chunks() and scrub_chunk()

2015-07-09 Thread Zhaolei
From: Zhao Lei These variables are not used from introduced version , remove them. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 9 ++--- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index eb35176..f552937 100644 --- a/fs/btrfs/scrub.c

[PATCH] btrfs: Fix lockdep warning of btrfs_run_delayed_iputs()

2015-07-14 Thread Zhaolei
From: Zhao Lei Liu Bo reported a lockdep warning of delayed_iput_sem in xfstests generic/241: [ 2061.345955] = [ 2061.346027] [ INFO: possible recursive locking detected ] [ 2061.346027] 4.1.0+ #268 Tainted: GW [ 2061.346027] --

[PATCH] btrfs: Avoid NULL pointer dereference of free_extent_buffer when read_tree_block() fail

2015-07-15 Thread Zhaolei
From: Zhao Lei When read_tree_block() failed, we can see following dmesg: [ 134.371389] BUG: unable to handle kernel NULL pointer dereference at 0063 [ 134.372236] IP: [] free_extent_buffer+0x21/0x90 [ 134.372236] PGD 0 [ 134.372236] Oops: [#1] SMP [ 134.372236] Module

[PATCH] btrfs: Show detail information when mount failed on missing devices

2015-07-16 Thread Zhaolei
From: Zhao Lei When mount failed because missing device, we can see following dmesg: [ 1060.267743] BTRFS: too many missing devices, writeable mount is not allowed [ 1060.273158] BTRFS: open_ctree failed This patch add missing_device_number and tolerated_missing_device_number to above output,

[PATCH] btrfs: Cleanup for btrfs_calc_num_tolerated_disk_barrier_failures()

2015-07-16 Thread Zhaolei
From: Zhao Lei 1: Use ARRAY_SIZE(types) to replace a static-value variant: int num_types = 4; 2: Use 'continue' on condition to reduce one level tab if (!XXX) { code; ... } -> if (XXX) continue; code; ... 3: Put setting 'num_tolerated_disk_barrier_failu

[PATCH] btrfs: Add raid56 support for updating num_tolerated_disk_barrier_failures in btrfs_balance()

2015-07-16 Thread Zhaolei
From: Zhao Lei Code for updating fs_info->num_tolerated_disk_barrier_failures in btrfs_balance() lacks raid56 support. Reason: Above code was wroten in 2012-08-01, together with btrfs_calc_num_tolerated_disk_barrier_failures()'s first version. Then, btrfs_calc_num_tolerated_disk_barrier_fail

[PATCH] [RFC] btrfs: Avoid using single-type chunk on degree mode

2015-07-17 Thread Zhaolei
From: Zhao Lei We can get mount-fail in following operation: # mkfs a raid1 filesystem mkfs.btrfs -f -d raid1 -m raid1 /dev/vdd /dev/vde # destroy a disk dd if=/dev/zero of=/dev/vde bs=1M count=1 # do some fs operation on degraded mode mount -o degraded /dev/vdd /mnt/test touch /m

[PATCH v2] btrfs: Add raid56 support for updating num_tolerated_disk_barrier_failures in btrfs_balance()

2015-07-20 Thread Zhaolei
From: Zhao Lei Code for updating fs_info->num_tolerated_disk_barrier_failures in btrfs_balance() lacks raid56 support. Reason: Above code was wroten in 2012-08-01, together with btrfs_calc_num_tolerated_disk_barrier_failures()'s first version. Then, btrfs_calc_num_tolerated_disk_barrier_fail

[PATCH] btrfs: Show detail information when mount failed on missing devices

2015-07-20 Thread Zhaolei
From: Zhao Lei When mount failed because missing device, we can see following dmesg: [ 1060.267743] BTRFS: too many missing devices, writeable mount is not allowed [ 1060.273158] BTRFS: open_ctree failed This patch add missing_device_number and tolerated_missing_device_number to above output,

[PATCH 1/2] btrfs: Check cancel and pause in interval of scrub operation

2015-07-20 Thread Zhaolei
From: Zhao Lei Old code checking cancel and pause request inside scrub stripe operation, like: loop() { if (parity) { scrub_parity_stripe(); continue; } check_cancel_and_pause() scrub_normal_stripe(); } Reason is when introduce raid56 stripe scrub, new code is i

[PATCH 2/2] btrfs: Free checksum list on scrub_extent() fail

2015-07-20 Thread Zhaolei
From: Zhao Lei When scrub_extent() failed, we need to free previois created checksum list. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index f8551b9..24720f6 100644 --- a/fs/btrfs/s

[PATCH] btrfs: Fix calculate typo caused by ambiguous meaning of logic_end

2015-07-21 Thread Zhaolei
From: Zhao Lei For example, in scrub_raid56_parity(), following lines are used to judge is all data processed: place1: if (key.objectid > logic_end) ... place2: if (logic_start >= logic_end) ... ... (place2 is typo, is should be ">", it is copied from other place, where logic_end's meaning

[PATCH 1/2] btrfs: Load only necessary csums into list in scrub

2015-07-21 Thread Zhaolei
From: Zhao Lei We need not load csum of whole strip in scrub because strip is trimed before use, it is to say, what we really need to calculate csum is data between [extent_logical, extent_len). This patch changed to use above segment for btrfs_lookup_csums_range() in scrub_stripe() Signed-off-

[PATCH 2/2] btrfs: Bypass unrelated items before accessing its contents in scrub

2015-07-21 Thread Zhaolei
From: Zhao Lei When we access extent_root in scrub_stripe() and scrub_raid56_parity(), we need bypass unrelated tree item firstly before using its contents to do other condition. It is not a bug fix, only making code sequence in logic. Signed-off-by: Zhao Lei --- fs/btrfs/scrub.c | 16 +++

[PATCH] btrfs-progs: Accurate errormsg for resize operation on no-enouth-free-space case

2015-07-22 Thread Zhaolei
From: Zhao Lei btrfs progs output following error message when doing resize on no-enouth-free-space case: # btrfs filesystem resize +10g /mnt/btrfs_5gb Resize '/mnt/btrfs_5gb' of '+10g' ERROR: unable to resize '/mnt/btrfs_5gb' - File too large # It is not a good description for users, and th

[PATCH] btrfs: Fix scrub panic when leaf accross stripes

2015-07-22 Thread Zhaolei
From: Zhao Lei Scrub panic in following operation: mkfs.ext4 /dev/vdh btrfs-convert /dev/vdh mount /dev/vdh /mnt/tmp1 btrfs scrub start -B /dev/vdh (panic) Reason: 1: In some case, leaf created by btrfs-convert was splited into 2 strips. 2: Scrub bypassed part of above wrong l

[PATCH 5/5] btrfs-progs: Set info->periodic.timer_fd to 0 in init-fail

2015-07-24 Thread Zhaolei
From: Zhao Lei In current code, (info->periodic.timer_fd == 0) means it is not valid, as: if (info->periodic.timer_fd) { close(info->periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by: Zhao Lei --- task-ut

[PATCH 1/5] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

2015-07-24 Thread Zhaolei
From: Zhao Lei We can reproduce this bug by a simple script: DEV=/dev/vdh for ((i = 0; i < 100; i++)); do echo "loop $i" mkfs.ext4 "$DEV" &>/dev/null || { echo mkfs fail; break; } btrfs-convert "$DEV" || break done Result is like: loop 0 ... loop 1 ... loop 3

[PATCH 2/5] btrfs-progs: Move close timer handle code to line after sub process exit

2015-07-24 Thread Zhaolei
From: Zhao Lei The timer handle have possibility in using by sub thread, better to close it after sub process exit. Signed-off-by: Zhao Lei --- task-utils.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/task-utils.c b/task-utils.c index 0390a69..17fd573 100644 --- a

[PATCH 4/5] btrfs-progs: resst info->periodic.timer_fd's value after free

2015-07-24 Thread Zhaolei
From: Zhao Lei task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info->id for safe. Signed-off-by: Zhao Lei --- task-utils.c | 6 +- 1 file changed, 5 insertions(+)

[PATCH 3/5] btrfs-progs: Remove cleanup-timer code for btrfs-convert

2015-07-24 Thread Zhaolei
From: Zhao Lei No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei --- btrfs-convert.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --gi

[PATCH v2 0/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

2015-07-26 Thread Zhaolei
From: Zhao Lei It is v2 patchset of: Fix wrong address accessing by subthread in btrfs-convert. Changelog: 1: Move bug test script from patch description into a single script in test/misc-tests, suggested-by: David Sterba 2: Move code for creating loop device to common [PATCH 6/7] Zhao

[PATCH v2 3/7] btrfs-progs: Remove cleanup-timer code for btrfs-convert

2015-07-26 Thread Zhaolei
From: Zhao Lei No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei --- btrfs-convert.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --gi

[PATCH v2 4/7] btrfs-progs: resst info->periodic.timer_fd's value after free

2015-07-26 Thread Zhaolei
From: Zhao Lei task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info->id for safe. Signed-off-by: Zhao Lei --- task-utils.c | 6 +- 1 file changed, 5 insertions(+)

[PATCH v2 1/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

2015-07-26 Thread Zhaolei
From: Zhao Lei btrfs-convert sometimes show 'Assertion failed' in converting a nearly blank file system, as: create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating btrfs metadata. creating ext2fs image f

[PATCH v2 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert

2015-07-26 Thread Zhaolei
From: Zhao Lei Current code of btrfs-convert have a bug of thread conflict, which caused invalid memory accessing between threads, and make program panic. This patch add a test item for above bug, as: # ./misc-tests.sh [TEST] 001-btrfstune-features [TEST] 002-uuid-rewrite [TEST]

[PATCH v2 5/7] btrfs-progs: Set info->periodic.timer_fd to 0 in init-fail

2015-07-26 Thread Zhaolei
From: Zhao Lei In current code, (info->periodic.timer_fd == 0) means it is not valid, as: if (info->periodic.timer_fd) { close(info->periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by: Zhao Lei --- task-ut

[PATCH v2 2/7] btrfs-progs: Move close timer handle code to line after sub process exit

2015-07-26 Thread Zhaolei
From: Zhao Lei The timer handle have possibility in using by sub thread, better to close it after sub process exit. Signed-off-by: Zhao Lei --- task-utils.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/task-utils.c b/task-utils.c index 0390a69..17fd573 100644 --- a

[PATCH v2 6/7] btrfs-progs: Move code to create loop device to common

2015-07-26 Thread Zhaolei
From: Zhao Lei This code block is common used, move it to common function will make code clean. Signed-off-by: Zhao Lei --- tests/common | 16 tests/fsck-tests/013-extent-tree-rebuild/test.sh | 11 +-- tests/misc-tests/001-btrfstune-

[PATCH] btrfs-progs-tests: Add '-o loop' to mount command line in convert-tests.sh

2015-07-26 Thread Zhaolei
From: Zhao Lei To fix following bug: # ./convert-tests.sh [TEST] ext2 4k nodesize, btrfs defaults failed: mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests/mnt # tail convert-tests-results.txt ... ### mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests

[PATCH] btrfs-progs: Show detail error message when write sb failed in write_dev_supers()

2015-07-27 Thread Zhaolei
From: Zhao Lei fsck-tests.sh failed and show following message in my node: # ./fsck-tests.sh [TEST] 001-bad-file-extent-bytenr disk-io.c:1444: write_dev_supers: Assertion `ret != BTRFS_SUPER_INFO_SIZE` failed. /root/btrfsprogs/btrfs-image(write_all_supers+0x2d2)[0x41031c] /root/bt

[PATCH v3 1/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

2015-07-27 Thread Zhaolei
From: Zhao Lei btrfs-convert sometimes show 'Assertion failed' in converting a nearly blank file system, as: create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating btrfs metadata. creating ext2fs image f

[PATCH v3 0/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

2015-07-27 Thread Zhaolei
From: Zhao Lei It is v3 patchset of: Fix wrong address accessing by subthread in btrfs-convert. Changelog v2->v3: 1: Add $TOP path prefix to btrfs-convert command in: [PATCH v2 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert To check prog in btrfs sourc

[PATCH v3 4/7] btrfs-progs: resst info->periodic.timer_fd's value after free

2015-07-27 Thread Zhaolei
From: Zhao Lei task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info->id for safe. Signed-off-by: Zhao Lei --- task-utils.c | 6 +- 1 file changed, 5 insertions(+)

[PATCH v3 2/7] btrfs-progs: Move close timer handle code to line after sub process exit

2015-07-27 Thread Zhaolei
From: Zhao Lei The timer handle have possibility in using by sub thread, better to close it after sub process exit. Signed-off-by: Zhao Lei --- task-utils.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/task-utils.c b/task-utils.c index 0390a69..17fd573 100644 --- a

[PATCH v3 6/7] btrfs-progs: Move code to create loop device to common

2015-07-27 Thread Zhaolei
From: Zhao Lei This code block is common used, move it to common function will make code clean. Signed-off-by: Zhao Lei --- tests/common | 16 tests/fsck-tests/013-extent-tree-rebuild/test.sh | 11 +-- tests/misc-tests/001-btrfstune-

[PATCH v3 5/7] btrfs-progs: Set info->periodic.timer_fd to 0 in init-fail

2015-07-27 Thread Zhaolei
From: Zhao Lei In current code, (info->periodic.timer_fd == 0) means it is not valid, as: if (info->periodic.timer_fd) { close(info->periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by: Zhao Lei --- task-ut

[PATCH v3 3/7] btrfs-progs: Remove cleanup-timer code for btrfs-convert

2015-07-27 Thread Zhaolei
From: Zhao Lei No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei --- btrfs-convert.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --gi

[PATCH v3 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert

2015-07-27 Thread Zhaolei
From: Zhao Lei Current code of btrfs-convert have a bug of thread conflict, which caused invalid memory accessing between threads, and make program panic. This patch add a test item for above bug, as: # ./misc-tests.sh [TEST] 001-btrfstune-features [TEST] 002-uuid-rewrite [TEST]

[PATCH] btrfs-progs-tests: Add -o loop to fsck-tests/012-leaf-corruption for loop device

2015-07-27 Thread Zhaolei
From: Zhao Lei To avoid following mount error in test: mount: /root/btrfs/progs/tests/fsck-tests/012-leaf-corruption/test.img is not a block device (maybe try `-o loop'?) Signed-off-by: Zhao Lei --- tests/fsck-tests/012-leaf-corruption/test.sh | 2 +- 1 file changed, 1 insertion(+), 1 dele

[PATCH 1/2] btrfs-progs-tests: Introduce init_env() to initialize common env variant

2015-07-27 Thread Zhaolei
From: Zhao Lei For example, $TEST_DIR is common used in severial tests, and have duplicated code for initialize. These duplicated code not only benifits harddisk vendor, but have inconsistent details, as: convert-tests.sh: lack of mkdir fsck-tests/012-leaf-corruption/test.sh: unnecessary mkd

[PATCH 2/2] btrfs-progs-tests: Fix mount fail of 013-extent-tree-rebuild

2015-07-27 Thread Zhaolei
From: Zhao Lei When using loop device for test, fsck-tests/013-extent-tree-rebuild failed with following error message: # ./fsck-tests.sh ... [TEST] 013-extent-tree-rebuild failed: mount /data/btrfsprogs/tests/test.img /data/btrfsprogs/tests/mnt test failed for case 013-extent-tree-

[PATCH] btrfs-progs-tests: Avoid outputting useless warning in ./fsck-tests.sh

2015-07-27 Thread Zhaolei
From: Zhao Lei 002-bad-transid outout 'transid verify failed' message in screen which is just a warning in btrfs-image in normal condition of this test. This patch move above warning into $RESULTS, to: 1: Avoid trouble screen output 2: Let user known detail if other error happened in btrfs-image

[PATCH] btrfs-progs: Increase running state's priority in stat output

2015-07-28 Thread Zhaolei
From: Zhao Lei Anthony Plack reported a output bug in maillist: title: btrfs-progs SCRUB reporting aborted but still running - minor btrfs scrub status report it was aborted but still runs to completion. # btrfs scrub status /mnt/data scrub status for f591ac13-1a69-476d-bd30-346f87a491da

[PATCH 01/15] Btrfs: fix a out-of-bound access of raid_map

2015-01-13 Thread Zhaolei
From: Zhao Lei We add the number of stripes on target devices into bbio->num_stripes if we are under device replacement, and we just sort the raid_map of those stripes that not on the target devices, so if when we need real raid_map, we need skip the stripes on the target devices. Signed-off-by:

[PATCH 00/15] Btrfs: Cleanup for raid56 scrib

2015-01-13 Thread Zhaolei
From: Zhao Lei Hi, everyone, These are cleanup patchs for raid56's scrib functions, it is based on review for new-developed raid56's scrub code. Some small typo-fix and cleanup for other functions are also included in this patchset. Thanks Zhaolei Zhao Lei (15): Btrfs: fix a ou

[PATCH 10/15] Btrfs: Avoid trustless page-level-repair in dev-replace

2015-01-13 Thread Zhaolei
From: Zhao Lei Current code of page level repair for dev-replace can only support io-error, we can't use it in checksum-fail case. We can skip this kind of repair in dev-replace just as we in scrub. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 8 1 file cha

[PATCH 02/15] Btrfs: sort raid_map before adding tgtdev stripes

2015-01-13 Thread Zhaolei
From: Zhao Lei It can avoid complex calculation of real stripes in sort, moreover, we can clean up code of sorting tgtdev_map because it will be in order initially. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 22 -- 1 file changed, 8 insertions

[PATCH 15/15] Btrfs: Introduce BTRFS_BLOCK_GROUP_RAID56_MASK to check raid56 simply

2015-01-13 Thread Zhaolei
From: Zhao Lei So we can check raid56 with: (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) instead of long: (map->type & (BTRFS_BLOCK_GROUP_RAID5 | BTRFS_BLOCK_GROUP_RAID6)) Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/ctree.h | 3 +++ fs/btrfs/inode.c | 3 +-- fs/btrfs/s

[PATCH 12/15] Btrfs: Combine per-page recover in dev-replace and scrub

2015-01-13 Thread Zhaolei
From: Zhao Lei The code are similar, combine them to make code clean and easy to maintenance. Some lost condition are also completed with benefit of this combination. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 110 ---

[PATCH 13/15] Btrfs: Simplify scrub_setup_recheck_block()'s argument

2015-01-13 Thread Zhaolei
From: Zhao Lei scrub_setup_recheck_block() have many arguments but most of them can be get from one of them, we can remove them to make code clean. Some other cleanup for that function also included in this patch. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 25 +

[PATCH 14/15] Btrfs: Include map_type in raid_bio

2015-01-13 Thread Zhaolei
From: Zhao Lei Corrent code use many kinds of "clever" way to determine operation target's raid type, as: raid_map != NULL or raid_map[MAX_NR] == RAID[56]_Q_STRIPE To make code easy to maintenance, this patch put raid type into bbio, and we can always get raid type from bbio with a "stupid

[PATCH 04/15] Btrfs: add ref_count and free function for btrfs_bio

2015-01-13 Thread Zhaolei
From: Zhao Lei 1: ref_count is simple than current RBIO_HOLD_BBIO_MAP_BIT flag to keep btrfs_bio's memory in raid56 recovery implement. 2: free function for bbio will make code clean and flexible, plus forced data type checking in compile. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie

[PATCH 03/15] Btrfs: Make raid_map array be inlined in btrfs_bio structure

2015-01-13 Thread Zhaolei
From: Zhao Lei It can make code more simple and clear, we need not care about free bbio and raid_map together. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/raid56.c | 77 ++--- fs/btrfs/raid56.h | 11 +++--- fs/btrfs/scrub.c | 31

[PATCH 09/15] Btrfs: Break loop when reach BTRFS_MAX_MIRRORS in scrub_setup_recheck_block()

2015-01-13 Thread Zhaolei
From: Zhao Lei Use break instead of useless loop should be more suitable in this case. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index 8ea5ce7..22779dd 100644 ---

[PATCH 11/15] Btrfs: Separate finding-right-mirror and writing-to-target's process in scrub_handle_errored_block()

2015-01-13 Thread Zhaolei
From: Zhao Lei In corrent code, code of finding-right-mirror and writing-to-target are mixed in logic, if we find a right mirror but failed in writing to target, it will treat as "hadn't found right block", and fill the target with sblock_bad. Actually, "failed in writing to target" does not mea

[PATCH 08/15] Btrfs: btrfs_rm_dev_replace_blocked(): Use wait_event()

2015-01-13 Thread Zhaolei
From: Zhao Lei Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 13 ++--- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 92109b7..5ec03d9 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs

[PATCH 07/15] Btrfs: Cleanup btrfs_bio_counter_inc_blocked()

2015-01-13 Thread Zhaolei
From: Zhao Lei 1: Remove no-need DEFINE_WAIT(wait) 2: Add likely() for BTRFS_FS_STATE_DEV_REPLACING condition 3: Use while loop instead of goto Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff -

[PATCH 06/15] Btrfs: Remove noneed force_write in scrub_write_block_to_dev_replace

2015-01-13 Thread Zhaolei
From: Zhao Lei It is always 1 in this place, because !1 case was already jumped out in previous code. Signed-off-by: Zhao Lei Signed-off-by: Miao Xie --- fs/btrfs/scrub.c | 19 +++ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub

  1   2   >