[PATCH] fstests: Fix generic/102 fail for btrfs

From: Zhao Lei generic/102 sometimes fails in newest btrfs toolchain, because it use non-mixed mode in default, which request more space for metadata, and no space for data writing. This patch force mixed mode for btrfs in generic/102. Signed-off-by: Zhao Lei

[PATCH] fstests: speedup generic/027 for new version of btrfs

From: Zhao Lei New version of btrfs create non-mixed blockgroups in all case. For generic/027, the filesystem in test is convert from mixed-blockgroup to non-mixed blockgroup. And test time is changed from 400s -> 2700s in my node. To test btrfs with all mountoptions,

[PATCH] btrfs-progs: Increase running state's priority in stat output

From: Zhao Lei zhao...@cn.fujitsu.com Anthony Plack anth...@plack.net reported a output bug in maillist: title: btrfs-progs SCRUB reporting aborted but still running - minor btrfs scrub status report it was aborted but still runs to completion. # btrfs scrub status /mnt/data scrub status

[PATCH v3 1/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com btrfs-convert sometimes show 'Assertion failed' in converting a nearly blank file system, as: create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating btrfs metadata.

[PATCH v3 0/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com It is v3 patchset of: Fix wrong address accessing by subthread in btrfs-convert. Changelog v2-v3: 1: Add $TOP path prefix to btrfs-convert command in: [PATCH v2 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert To

[PATCH v3 4/7] btrfs-progs: resst info-periodic.timer_fd's value after free

From: Zhao Lei zhao...@cn.fujitsu.com task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info-id for safe. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- task-utils.c

[PATCH] btrfs-progs-tests: Add -o loop to fsck-tests/012-leaf-corruption for loop device

From: Zhao Lei zhao...@cn.fujitsu.com To avoid following mount error in test: mount: /root/btrfs/progs/tests/fsck-tests/012-leaf-corruption/test.img is not a block device (maybe try `-o loop'?) Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- tests/fsck-tests/012-leaf-corruption/test.sh |

[PATCH v3 2/7] btrfs-progs: Move close timer handle code to line after sub process exit

From: Zhao Lei zhao...@cn.fujitsu.com The timer handle have possibility in using by sub thread, better to close it after sub process exit. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- task-utils.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/task-utils.c

[PATCH v3 6/7] btrfs-progs: Move code to create loop device to common

From: Zhao Lei zhao...@cn.fujitsu.com This code block is common used, move it to common function will make code clean. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- tests/common | 16 tests/fsck-tests/013-extent-tree-rebuild/test.sh | 11

[PATCH v3 5/7] btrfs-progs: Set info-periodic.timer_fd to 0 in init-fail

From: Zhao Lei zhao...@cn.fujitsu.com In current code, (info-periodic.timer_fd == 0) means it is not valid, as: if (info-periodic.timer_fd) { close(info-periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by:

[PATCH v3 3/7] btrfs-progs: Remove cleanup-timer code for btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- btrfs-convert.c | 4 +--- 1 file

[PATCH v3 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com Current code of btrfs-convert have a bug of thread conflict, which caused invalid memory accessing between threads, and make program panic. This patch add a test item for above bug, as: # ./misc-tests.sh [TEST] 001-btrfstune-features [TEST]

[PATCH] btrfs-progs: Show detail error message when write sb failed in write_dev_supers()

From: Zhao Lei zhao...@cn.fujitsu.com fsck-tests.sh failed and show following message in my node: # ./fsck-tests.sh [TEST] 001-bad-file-extent-bytenr disk-io.c:1444: write_dev_supers: Assertion `ret != BTRFS_SUPER_INFO_SIZE` failed.

[PATCH 1/2] btrfs-progs-tests: Introduce init_env() to initialize common env variant

From: Zhao Lei zhao...@cn.fujitsu.com For example, $TEST_DIR is common used in severial tests, and have duplicated code for initialize. These duplicated code not only benifits harddisk vendor, but have inconsistent details, as: convert-tests.sh: lack of mkdir

[PATCH 2/2] btrfs-progs-tests: Fix mount fail of 013-extent-tree-rebuild

From: Zhao Lei zhao...@cn.fujitsu.com When using loop device for test, fsck-tests/013-extent-tree-rebuild failed with following error message: # ./fsck-tests.sh ... [TEST] 013-extent-tree-rebuild failed: mount /data/btrfsprogs/tests/test.img /data/btrfsprogs/tests/mnt test failed

[PATCH] btrfs-progs-tests: Avoid outputting useless warning in ./fsck-tests.sh

From: Zhao Lei zhao...@cn.fujitsu.com 002-bad-transid outout 'transid verify failed' message in screen which is just a warning in btrfs-image in normal condition of this test. This patch move above warning into $RESULTS, to: 1: Avoid trouble screen output 2: Let user known detail if other error

[PATCH v2 6/7] btrfs-progs: Move code to create loop device to common

From: Zhao Lei zhao...@cn.fujitsu.com This code block is common used, move it to common function will make code clean. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- tests/common | 16 tests/fsck-tests/013-extent-tree-rebuild/test.sh | 11

[PATCH v2 1/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com btrfs-convert sometimes show 'Assertion failed' in converting a nearly blank file system, as: create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating btrfs metadata.

[PATCH v2 7/7] btrfs-progs: Introduce a misc test for thread conflict in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com Current code of btrfs-convert have a bug of thread conflict, which caused invalid memory accessing between threads, and make program panic. This patch add a test item for above bug, as: # ./misc-tests.sh [TEST] 001-btrfstune-features [TEST]

[PATCH v2 5/7] btrfs-progs: Set info-periodic.timer_fd to 0 in init-fail

From: Zhao Lei zhao...@cn.fujitsu.com In current code, (info-periodic.timer_fd == 0) means it is not valid, as: if (info-periodic.timer_fd) { close(info-periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by:

[PATCH] btrfs-progs-tests: Add '-o loop' to mount command line in convert-tests.sh

From: Zhao Lei zhao...@cn.fujitsu.com To fix following bug: # ./convert-tests.sh [TEST] ext2 4k nodesize, btrfs defaults failed: mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests/mnt # tail convert-tests-results.txt ... ### mount /root/btrfsprogs/tests/test.img

[PATCH v2 3/7] btrfs-progs: Remove cleanup-timer code for btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- btrfs-convert.c | 4 +--- 1 file

[PATCH v2 4/7] btrfs-progs: resst info-periodic.timer_fd's value after free

From: Zhao Lei zhao...@cn.fujitsu.com task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info-id for safe. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- task-utils.c

[PATCH v2 0/7] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com It is v2 patchset of: Fix wrong address accessing by subthread in btrfs-convert. Changelog: 1: Move bug test script from patch description into a single script in test/misc-tests, suggested-by: David Sterba dste...@suse.com 2: Move code for creating

[PATCH 5/5] btrfs-progs: Set info-periodic.timer_fd to 0 in init-fail

From: Zhao Lei zhao...@cn.fujitsu.com In current code, (info-periodic.timer_fd == 0) means it is not valid, as: if (info-periodic.timer_fd) { close(info-periodic.timer_fd); ... We need set its value from -1 to 0 in create-fail case, to make code like above works. Signed-off-by:

[PATCH 1/5] btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com We can reproduce this bug by a simple script: DEV=/dev/vdh for ((i = 0; i 100; i++)); do echo loop $i mkfs.ext4 $DEV /dev/null || { echo mkfs fail; break; } btrfs-convert $DEV || break done Result is like: loop 0 ... loop 1

[PATCH 2/5] btrfs-progs: Move close timer handle code to line after sub process exit

From: Zhao Lei zhao...@cn.fujitsu.com The timer handle have possibility in using by sub thread, better to close it after sub process exit. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- task-utils.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/task-utils.c

[PATCH 4/5] btrfs-progs: resst info-periodic.timer_fd's value after free

From: Zhao Lei zhao...@cn.fujitsu.com task_period_stop() is used to close a timer explicitly, to avoid the timer handle closed again by task_stop(), we should reset its value after close. Also add value-reset for info-id for safe. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- task-utils.c

[PATCH 3/5] btrfs-progs: Remove cleanup-timer code for btrfs-convert

From: Zhao Lei zhao...@cn.fujitsu.com No need to close timer handle afain in subthread-close-callback, it is closed by task_stop() automatically. This patch also add a fflush() to make log output on time. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- btrfs-convert.c | 4 +--- 1 file

[PATCH] btrfs-progs: Accurate errormsg for resize operation on no-enouth-free-space case

From: Zhao Lei zhao...@cn.fujitsu.com btrfs progs output following error message when doing resize on no-enouth-free-space case: # btrfs filesystem resize +10g /mnt/btrfs_5gb Resize '/mnt/btrfs_5gb' of '+10g' ERROR: unable to resize '/mnt/btrfs_5gb' - File too large # It is not a good

[PATCH] btrfs: Fix scrub panic when leaf accross stripes

From: Zhao Lei zhao...@cn.fujitsu.com Scrub panic in following operation: mkfs.ext4 /dev/vdh btrfs-convert /dev/vdh mount /dev/vdh /mnt/tmp1 btrfs scrub start -B /dev/vdh (panic) Reason: 1: In some case, leaf created by btrfs-convert was splited into 2 strips. 2: Scrub

[PATCH 2/2] btrfs: Bypass unrelated items before accessing its contents in scrub

From: Zhao Lei zhao...@cn.fujitsu.com When we access extent_root in scrub_stripe() and scrub_raid56_parity(), we need bypass unrelated tree item firstly before using its contents to do other condition. It is not a bug fix, only making code sequence in logic. Signed-off-by: Zhao Lei

[PATCH 1/2] btrfs: Load only necessary csums into list in scrub

From: Zhao Lei zhao...@cn.fujitsu.com We need not load csum of whole strip in scrub because strip is trimed before use, it is to say, what we really need to calculate csum is data between [extent_logical, extent_len). This patch changed to use above segment for btrfs_lookup_csums_range() in

[PATCH] btrfs: Fix calculate typo caused by ambiguous meaning of logic_end

From: Zhao Lei zhao...@cn.fujitsu.com For example, in scrub_raid56_parity(), following lines are used to judge is all data processed: place1: if (key.objectid logic_end) ... place2: if (logic_start = logic_end) ... ... (place2 is typo, is should be , it is copied from other place, where

[PATCH] btrfs: Show detail information when mount failed on missing devices

From: Zhao Lei zhao...@cn.fujitsu.com When mount failed because missing device, we can see following dmesg: [ 1060.267743] BTRFS: too many missing devices, writeable mount is not allowed [ 1060.273158] BTRFS: open_ctree failed This patch add missing_device_number and

[PATCH v2] btrfs: Add raid56 support for updating num_tolerated_disk_barrier_failures in btrfs_balance()

From: Zhao Lei zhao...@cn.fujitsu.com Code for updating fs_info-num_tolerated_disk_barrier_failures in btrfs_balance() lacks raid56 support. Reason: Above code was wroten in 2012-08-01, together with btrfs_calc_num_tolerated_disk_barrier_failures()'s first version. Then,

[PATCH 2/2] btrfs: Free checksum list on scrub_extent() fail

From: Zhao Lei zhao...@cn.fujitsu.com When scrub_extent() failed, we need to free previois created checksum list. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c

[PATCH 1/2] btrfs: Check cancel and pause in interval of scrub operation

From: Zhao Lei zhao...@cn.fujitsu.com Old code checking cancel and pause request inside scrub stripe operation, like: loop() { if (parity) { scrub_parity_stripe(); continue; } check_cancel_and_pause() scrub_normal_stripe(); } Reason is when introduce raid56

[PATCH] [RFC] btrfs: Avoid using single-type chunk on degree mode

From: Zhao Lei zhao...@cn.fujitsu.com We can get mount-fail in following operation: # mkfs a raid1 filesystem mkfs.btrfs -f -d raid1 -m raid1 /dev/vdd /dev/vde # destroy a disk dd if=/dev/zero of=/dev/vde bs=1M count=1 # do some fs operation on degraded mode mount -o degraded

[PATCH] btrfs: Show detail information when mount failed on missing devices

From: Zhao Lei zhao...@cn.fujitsu.com When mount failed because missing device, we can see following dmesg: [ 1060.267743] BTRFS: too many missing devices, writeable mount is not allowed [ 1060.273158] BTRFS: open_ctree failed This patch add missing_device_number and

[PATCH] btrfs: Cleanup for btrfs_calc_num_tolerated_disk_barrier_failures()

From: Zhao Lei zhao...@cn.fujitsu.com 1: Use ARRAY_SIZE(types) to replace a static-value variant: int num_types = 4; 2: Use 'continue' on condition to reduce one level tab if (!XXX) { code; ... } - if (XXX) continue; code; ... 3: Put setting

[PATCH] btrfs: Add raid56 support for updating num_tolerated_disk_barrier_failures in btrfs_balance()

From: Zhao Lei zhao...@cn.fujitsu.com Code for updating fs_info-num_tolerated_disk_barrier_failures in btrfs_balance() lacks raid56 support. Reason: Above code was wroten in 2012-08-01, together with btrfs_calc_num_tolerated_disk_barrier_failures()'s first version. Then,

[PATCH] btrfs: Avoid NULL pointer dereference of free_extent_buffer when read_tree_block() fail

From: Zhao Lei zhao...@cn.fujitsu.com When read_tree_block() failed, we can see following dmesg: [ 134.371389] BUG: unable to handle kernel NULL pointer dereference at 0063 [ 134.372236] IP: [813a4a51] free_extent_buffer+0x21/0x90 [ 134.372236] PGD 0 [ 134.372236]

[PATCH] btrfs: Fix lockdep warning of btrfs_run_delayed_iputs()

From: Zhao Lei zhao...@cn.fujitsu.com Liu Bo bo.li@oracle.com reported a lockdep warning of delayed_iput_sem in xfstests generic/241: [ 2061.345955] = [ 2061.346027] [ INFO: possible recursive locking detected ] [ 2061.346027] 4.1.0+ #268

[PATCH] btrfs: Remove noused chunk_tree and chunk_objectid from scrub_enumerate_chunks() and scrub_chunk()

From: Zhao Lei zhao...@cn.fujitsu.com These variables are not used from introduced version , remove them. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 9 ++--- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index

[PATCH v3 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

From: Zhao Lei zhao...@cn.fujitsu.com It can reduce current duplicated code which is similar to scrub_blocked_if_needed() but can not call it because little different. It also used by my next patch which is in same case. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 11

[PATCH v3 1/4] btrfs: Use ref_cnt for set_block_group_ro()

From: Zhao Lei zhao...@cn.fujitsu.com More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename

[PATCH v3 4/4] btrfs: Fix data checksum error cause by replace with io-load.

From: Zhao Lei zhao...@cn.fujitsu.com xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found

[PATCH v3 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

From: Zhao Lei zhao...@cn.fujitsu.com Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c

[PATCH v3 0/4] btrfs: Fix data checksum error cause by replace with io-load

From: Zhao Lei zhao...@cn.fujitsu.com This patchset is used to fix data checksum error cause by replace with io-load. It cause xfstests btrfs/070 and btrfs/071 sometimes failed. See description in [PATCH 4/4] for detail. Changelog v2-v3: 1: Fix a typo(caused in rebase) which make xfstests

[PATCH] btrfs: [RFC] Don't use workqueue for raid56 parity scrub

From: Zhao Lei zhao...@cn.fujitsu.com The code in workqueue only do fast initialization and submit a bio in end, plus, it will not called in interrupe context, no need to queue a work for this type of work. Call it directly will make code simple, easy to debug and reduce potential problem.

[PATCH] btrfs: add error handling for scrub_workers_get()

From: Zhao Lei zhao...@cn.fujitsu.com Although it is a rare case, we'd better free previous allocated memory on error. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 31 --- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git

[PATCH] btrfs: Update out-of-date skip parity stripe comment

From: Zhao Lei zhao...@cn.fujitsu.com Because btrfs support scrub raid56 parity stripe now. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c index a13f91a..5ee5630 100644

[PATCH] btrfs: cleanup noused initialization of dev in btrfs_end_bio()

From: Zhao Lei zhao...@cn.fujitsu.com It is introduced by: c404e0dc2c843b154f9a36c3aec10d0a715d88eb Btrfs: fix use-after-free in the finishing procedure of the device replace But seems no relationship with that bug, this patch revirt these code block for cleanup. Signed-off-by: Zhao Lei

[PATCH v2] btrfs: Fix lockdep warning of wr_ctx-wr_lock in scrub_free_wr_ctx()

From: Zhao Lei zhao...@cn.fujitsu.com lockdep report following warning in test: [25176.843958] = [25176.844519] [ INFO: inconsistent lock state ] [25176.845047] 4.1.0-rc3 #22 Tainted: GW [25176.845591] - [25176.846153]

[PATCH] btrfs: Fix lockdep warning of wr_ctx-wr_lock in scrub_free_wr_ctx()

From: Zhao Lei zhao...@cn.fujitsu.com lockdep report following warning in test: [25176.843958] = [25176.844519] [ INFO: inconsistent lock state ] [25176.845047] 4.1.0-rc3 #22 Tainted: GW [25176.845591] - [25176.846153]

[PATCH 4/4] btrfs: Fix data checksum error cause by replace with io-load.

From: Zhao Lei zhao...@cn.fujitsu.com xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found

[PATCH 1/4] btrfs: Use ref_cnt for set_block_group_ro()

From: Zhao Lei zhao...@cn.fujitsu.com More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename

[PATCH 3/4] btrfs: use scrub_pause_on/off() to reduce code in scrub_enumerate_chunks()

From: Zhao Lei zhao...@cn.fujitsu.com Use new intruduced scrub_pause_on/off() can make this code block clean and more readable. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/scrub.c

[PATCH 2/4] btrfs: Separate scrub_blocked_if_needed() to scrub_pause_on/off()

From: Zhao Lei zhao...@cn.fujitsu.com It can reduce current duplicated code which is similar to scrub_blocked_if_needed() but can not call it because little different. It also used by my next patch which is in same case. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/scrub.c | 11

[PATCH 1/2] btrfs: Use ref_cnt for set_block_group_ro()

From: Zhao Lei zhao...@cn.fujitsu.com More than one code call set_block_group_ro() and restore rw in fail. Old code use bool bit to save blockgroup's ro state, it can not support parallel case(it is confirmd exist in my debug log). This patch use ref count to store ro state, and rename

[PATCH 2/2] btrfs: Fix xfstests btrfs/070

From: Zhao Lei zhao...@cn.fujitsu.com xfstests btrfs/070 sometimes failed. In my test machine, its fail rate is about 30%. In another vm(vmware), its fail rate is about 50%. Reason: btrfs/070 do replace and defrag with fsstress simultaneously, after above operation, checksum error is found by

[PATCH 8/9] btrfs: wait for delayed iputs on no space

From: Zhao Lei zhao...@cn.fujitsu.com btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f $DEV # mount -o nodatacow $DEV $MNT # for ((i = 0; i 100; i++)); do dd if=/dev/zero

[PATCH 3/9] btrfs: Adjust commit-transaction condition to avoid NO_SPACE more

From: Zhao Lei zhao...@cn.fujitsu.com If we have any chance to make a successful write, we should not give up. This patch adjust commit-transaction condition from: pinned = wanted to left + pinned = wanted Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 3 ++- 1

[PATCH 7/9] btrfs: Support busy loop of write and delete

From: Zhao Lei zhao...@cn.fujitsu.com Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We

[PATCH 4/9] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

From: Zhao Lei zhao...@cn.fujitsu.com Bug1: space_info-bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache-pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info-bytes_readonly.

[PATCH v2 0/9] btrfs: Fix no_space on dd and rm loop

script in 2000 times. I'll add xfstests for this case later. This is available at fix_no_space branch on my tree: git://github.com/zhaoleidd/btrfs.git It is also included in integration-for-chris branch in above tree. Thanks Zhaolei Zhao Lei (9): btrfs: fix condition of commit transaction

[PATCH 9/9] btrfs: cleanup unused alloc_chunk varible

From: Zhao Lei zhao...@cn.fujitsu.com Remove int alloc_chunk in btrfs_check_data_free_space() for not necessary. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[PATCH 5/9] btrfs: add WARN_ON() to check is space_info op current

From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1-v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba dste...@suse.cz Signed-off-by: Zhao Lei

[PATCH 6/9] btrfs: Fix NO_SPACE bug caused by delayed-iput

From: Zhao Lei zhao...@cn.fujitsu.com Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call

[PATCH 2/9] btrfs: Fix tail space processing in find_free_dev_extent()

From: Zhao Lei zhao...@cn.fujitsu.com It is another reason for NO_SPACE case. When we found enough free space in loop and saved them to max_hole_start/size before, and tail space contains pending extent, origional innocent max_hole_start/size are reset in retry. As a result,

[PATCH 1/9] btrfs: fix condition of commit transaction

From: Zhao Lei zhao...@cn.fujitsu.com Old code bypass commit transaction when we don't have enough pinned space, but another case is there exist freed bgs in current transction, it have possibility to make alloc_chunk success. This patch modify the condition to: if (have_free_bg ||

[PATCH 3/9] btrfs: Adjust commit-transaction condition to avoid NO_SPACE more

From: Zhao Lei zhao...@cn.fujitsu.com If we have any chance to make a successful write, we should not give up. This patch adjust commit-transaction condition from: pinned = wanted to left + pinned = wanted Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 3 ++- 1

[PATCH 5/9] btrfs: add WARN_ON() to check is space_info op current

From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1-v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba dste...@suse.cz Signed-off-by: Zhao Lei

[PATCH 7/9] btrfs: Support busy loop of write and delete

From: Zhao Lei zhao...@cn.fujitsu.com Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We

[PATCH 4/9] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

From: Zhao Lei zhao...@cn.fujitsu.com Bug1: space_info-bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache-pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info-bytes_readonly.

[PATCH 1/9] btrfs: fix condition of commit transaction

From: Zhao Lei zhao...@cn.fujitsu.com Old code bypass commit transaction when we don't have enough pinned space, but another case is there exist freed bgs in current transction, it have possibility to make alloc_chunk success. This patch modify the condition to: if (have_free_bg ||

[PATCH 9/9] btrfs: cleanup unused alloc_chunk varible

From: Zhao Lei zhao...@cn.fujitsu.com Remove int alloc_chunk in btrfs_check_data_free_space() for not necessary. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[PATCH 8/9] btrfs: wait for delayed iputs on no space

From: Zhao Lei zhao...@cn.fujitsu.com btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f $DEV # mount -o nodatacow $DEV $MNT # for ((i = 0; i 100; i++)); do dd if=/dev/zero

[PATCH 2/9] btrfs: Fix tail space processing in find_free_dev_extent()

From: Zhao Lei zhao...@cn.fujitsu.com It is another reason for NO_SPACE case. When we found enough free space in loop and saved them to max_hole_start/size before, and tail space contains pending extent, origional innocent max_hole_start/size are reset in retry. As a result,

[PATCH 6/9] btrfs: Fix NO_SPACE bug caused by delayed-iput

From: Zhao Lei zhao...@cn.fujitsu.com Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call

[PATCH 0/9] btrfs: Fix no_space on dd and rm loop

times. Confirmed having-problem in v4.0-rc5 and no-problem on top of this patchset. I'll add xfstests for this case later. This is available at fix_no_space branch on my tree: git://github.com/zhaoleidd/btrfs.git It is also included in integration-for-chris branch in above tree. Thanks Zhaolei

[PATCH] btrfs: wait for delayed iputs on no space

From: Zhao Lei zhao...@cn.fujitsu.com btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f $DEV # mount -o nodatacow $DEV $MNT # for ((i = 0; i 100; i++)); do dd if=/dev/zero

[PATCH] btrfs: wait for delayed iputs on no space

From: Zhao Lei zhao...@cn.fujitsu.com This is another fix of no_space case. All patchs for fix no_space bug are available at fix_no_space branch on: git://github.com/zhaoleidd/btrfs Any suggestions are welcome. Zhao Lei (1): btrfs: wait for delayed iputs on no space

[PATCH 6/6] btrfs: Remove unused err = 0 line for raid_rmw_end_io()

From: Zhao Lei zhao...@cn.fujitsu.com Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/raid56.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 2285e78..c087870 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1464,7 +1464,6 @@

[PATCH 1/6] btrfs: Fix calculation of rbio-dbitmap's size calculation

From: Zhao Lei zhao...@cn.fujitsu.com Current code is trying to calculate rbio-dbitmap's size to make it align to sizeof(long), but implement haven't achived this object, it is align to sizeof(char) instead. This patch fixed above calculation, and use sizeof(long) instead of fixed 8 to increate

[PATCH 4/6] btrfs: Clear PageUptodate bit in alloc_rbio_parity_pages()

From: Zhao Lei zhao...@cn.fujitsu.com Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/raid56.c | 1 + 1 file changed, 1 insertion(+) diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 0d902ac..0a40d07 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1049,6 +1049,7 @@

[PATCH 3/6] btrfs: use rbio-nr_pages to reduce calculation

From: Zhao Lei zhao...@cn.fujitsu.com We can use rbio-stripe_npages to reduce unnecessary calculation in many code place. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/raid56.c | 19 +++ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git

[PATCH 2/6] btrfs: Use unified stripe_page's index calculation

From: Zhao Lei zhao...@cn.fujitsu.com We are using different index calculation method for stripe_page in current code: 1: (rbio-stripe_len / PAGE_CACHE_SIZE) * stripe_index + page_index 2: DIV_ROUND_UP(rbio-stripe_len, PAGE_CACHE_SIZE) * stripe_index + page_index 3: DIV_ROUND_UP(rbio-stripe_len *

[PATCH 5/6] btrfs: Use raid_write_end_io for scrub

From: Zhao Lei zhao...@cn.fujitsu.com No need to create additional end_io function for scrub, it can use existing raid_write_end_io() instead. This patch also fixed some wrong comments. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/raid56.c | 36

[PATCH] btrfs: Support busy loop of write and delete

From: Zhao Lei zhao...@cn.fujitsu.com Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We

[PATCH 0/1] btrfs: Fix NO_SPACE bug caused by delayed-iput

From: Zhao Lei zhao...@cn.fujitsu.com It is the last patch to fix following write fail case: while true; do write a file to 75% fs size delete above file sync or sleep done Above issue is caused by several reason, and fixed in following patch respectively: Btrfs: fix

[PATCH 1/1] btrfs: Fix NO_SPACE bug caused by delayed-iput

From: Zhao Lei zhao...@cn.fujitsu.com Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call

[PATCH v3 0/2] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

From: Zhao Lei zhao...@cn.fujitsu.com Changelog v1-v2: [PATCH 2/2] btrfs: add WARN_ON() to check is space_info op current Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba dste...@suse.cz Changelog v1-v2: drop patch of: Remove BUG_ON() when failed searching

[PATCH v3 1/2] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

From: Zhao Lei zhao...@cn.fujitsu.com Bug1: space_info-bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache-pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info-bytes_readonly.

[PATCH v3 2/2] btrfs: add WARN_ON() to check is space_info op current

From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1-v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba dste...@suse.cz Signed-off-by: Zhao Lei

[PATCH 3/3] btrfs: Remove BUG_ON() when failed searching block_group_cache in unpin_extent_range()

From: Zhao Lei zhao...@cn.fujitsu.com Above BUG_ON() was triggered only one time in my test, but hadn't happened again in same env. The reason maybe: A block group which include pinned space was removed before unpin_extent_range(), and no other block_group_cache after start pos, so the code

[PATCH 1/3] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

From: Zhao Lei zhao...@cn.fujitsu.com Bug1: space_info-bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache-pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info-bytes_readonly.

[PATCH 2/3] btrfs: add WARN_ON() to check is space_info op current

From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 8 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/extent-tree.c

[PATCH 2/2] btrfs: add WARN_ON() to check is space_info op current

From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 8 1 file changed, 8 insertions(+) diff --git a/fs/btrfs/extent-tree.c

  1   2   >