Re: general thoughts and questions + general and RAID5/6 stability?

2014-09-25 Thread Stefan G. Weichinger
Am 23.09.2014 um 15:38 schrieb Austin S Hemmelgarn: What features for example? Well, running 'mkfs.btrfs -O list-all' with 3.16 btrfs-progs gives the following list of features: mixed-bg- mixed data and metadata block groups extref- increased hard-link limit per file to

[PATCH 2/2 v8] Btrfs: be aware of btree inode write errors to avoid fs corruption

2014-09-25 Thread Filipe Manana
While we have a transaction ongoing, the VM might decide at any time to call btree_inode-i_mapping-a_ops-writepages(), which will start writeback of dirty pages belonging to btree nodes/leafs. This call might return an error or the writeback might finish with an error before we attempt to commit

[PATCH] Btrfs: return failure if btrfs_dev_replace_finishing() failed

2014-09-25 Thread Eryu Guan
device replace could fail due to another running scrub process, but this failure doesn't get returned to userspace. The following steps could reproduce this issue mkfs -t btrfs -f /dev/sdb1 /dev/sdb2 mount /dev/sdb1 /mnt/btrfs while true; do btrfs scrub

[PATCH v3] xfstests: generic: add dir fsync test, motivated by a btrfs bug

2014-09-25 Thread Filipe Manana
This test is motivated by a bug found in btrfs when replaying a directory from the fsync log. The issue was that if a directory entry is both found in the persisted metadata and in the fsync log, at log replay time the directory got set with a wrong i_size. This had the consequence of not being

Re: [PATCH v2] btrfs-progs: Add human readable incompat flags output for btrfs-show-super

2014-09-25 Thread David Sterba
On Wed, Sep 24, 2014 at 01:31:39PM +0800, Qu Wenruo wrote: Add human readable incompat flags output for btrfs-show-super, now no longer needs to calculate the hex flags by hand. Nice. Sample output: incompat_flags 0x69 ( MIXED_BACKREF |

Re: [PATCH] Btrfs: remove empty block groups automatically V3

2014-09-25 Thread David Sterba
On Thu, Sep 18, 2014 at 11:20:02AM -0400, Josef Bacik wrote: This patch has the side effect of no longer having a bunch of BUG_ON()'s in the chunk delete code, which will be helpful for both this and relocate. Thanks, Would be better to split the unrelated error handling changes to a separate

[PATCH 2/2 v9] Btrfs: be aware of btree inode write errors to avoid fs corruption

2014-09-25 Thread Filipe Manana
While we have a transaction ongoing, the VM might decide at any time to call btree_inode-i_mapping-a_ops-writepages(), which will start writeback of dirty pages belonging to btree nodes/leafs. This call might return an error or the writeback might finish with an error before we attempt to commit

XFS Tests

2014-09-25 Thread nick
Hello Fellow Developers, Today I ran the Xfs test suite on a test drive I have. Further moreI get a few failing tests and will post the results of my run ofxfstests with ./check below. If someone would like my local.conf for debugging purposes please let me known. Nick Ran: btrfs/001 btrfs/002

[PATCH 2/2 v10] Btrfs: be aware of btree inode write errors to avoid fs corruption

2014-09-25 Thread Filipe Manana
While we have a transaction ongoing, the VM might decide at any time to call btree_inode-i_mapping-a_ops-writepages(), which will start writeback of dirty pages belonging to btree nodes/leafs. This call might return an error or the writeback might finish with an error before we attempt to commit

Re: [PATCH 2/2 v10] Btrfs: be aware of btree inode write errors to avoid fs corruption

2014-09-25 Thread David Sterba
On Thu, Sep 25, 2014 at 05:51:45PM +0100, Filipe Manana wrote: + switch (eb-log_index) { + case -1: + set_bit(BTRFS_INODE_BTREE_ERR, btree_ino-runtime_flags); + break; + case 0: + set_bit(BTRFS_INODE_BTREE_LOG1_ERR, btree_ino-runtime_flags); +

[PATCH 2/2 v11] Btrfs: be aware of btree inode write errors to avoid fs corruption

2014-09-25 Thread Filipe Manana
While we have a transaction ongoing, the VM might decide at any time to call btree_inode-i_mapping-a_ops-writepages(), which will start writeback of dirty pages belonging to btree nodes/leafs. This call might return an error or the writeback might finish with an error before we attempt to commit

[PATCH 1/1 linux-next] btrfs: fix shadow warning on cmp

2014-09-25 Thread Fabian Frederick
cmp was declared twice in btrfs_compare_trees resulting in a shadow warning. This patch renames second internal variable. Signed-off-by: Fabian Frederick f...@skynet.be --- Compiled but untested. fs/btrfs/ctree.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git

Re: btrfs send does not work from readonly device

2014-09-25 Thread G EO
The snapshots underneath are set read only anyway in my case, since the they landed there by btrfs receive or I am thinking wrong here? Duncan, do you remember the bug number? Thanks! -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

[PATCH] Btrfs-progs: make BUG*() be more verbose

2014-09-25 Thread Josef Bacik
Currently these macros just tie to assert(), which gives us line number and such but no backtrace so no actual context. This patch adds support for spitting out a backtrace so we can see how we got to the given assert. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- Makefile | 2 +-

[PATCH] Btrfs-progs: don't assert when we have an invalid mapping

2014-09-25 Thread Josef Bacik
Asserting is no fun, we may be able to recover from this error in certain cases (like btrfs-image and btrfsck). Just do what the kernel does and spit out an error and return that there is only 1 copy. Thanks, Signed-off-by: Josef Bacik jba...@fb.com --- volumes.c | 16 ++-- 1 file

[PATCH] Btrfs-progs: let btrfs-image actually work on a balanced fs

2014-09-25 Thread Josef Bacik
We use the read extent buffer infrastructure to read the super block when we are creating a btrfs-image. This works out fine most of the time except when the fs has been balanced, then it fails to map the super block. So we could fix btrfs-image to read in the super in a special way, but thats

Re: 3.16 Managed to ENOSPC with 80% used

2014-09-25 Thread Dan Merillat
On Wed, Sep 24, 2014 at 6:23 PM, Holger Hoffstätte holger.hoffstae...@googlemail.com wrote: Basically it's been data allocation happy, since I haven't deleted 53GB at any point. Unfortunately, none of the chunks are at 0% usage so a balance -dusage=0 finds nothing to drop. Also try

Re: 3.16 Managed to ENOSPC with 80% used

2014-09-25 Thread Holger Hoffstätte
On Thu, 25 Sep 2014 17:05:11 -0400, Dan Merillat wrote: Deleting files or dropping snapshots is difficult when it's wedged as well, a lot of disk activity (journal thrash?) and no persistent progress - a reboot brigs the deleted files back. I eventually managed to empty a single data chunk

[PATCH 1/1 linux-next] Btrfs: remove redundant btrfs_verify_qgroup_counts declaration.

2014-09-25 Thread Fabian Frederick
Do like disk-io function declared under CONFIG_BTRFS_FS_RUN_SANITY_TESTS and keep prototype in qgroup.h only Signed-off-by: Fabian Frederick f...@skynet.be --- fs/btrfs/ctree.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 089f6da..67ef9d1 100644

Re: [PATCH] Btrfs-progs: let btrfs-image actually work on a balanced fs

2014-09-25 Thread Zach Brown
--- a/btrfs-image.c +++ b/btrfs-image.c @@ -1020,6 +1020,9 @@ static int copy_tree_blocks(struct btrfs_root *root, struct extent_buffer *eb, int i = 0; int ret; + if (btrfs_header_bytenr(eb) == 65536) + printf(We have bytenr 65536, belongs to %llu, level

Re: btrfs send does not work from readonly device

2014-09-25 Thread Duncan
On Thu, 25 Sep 2014 21:39:41 +0200 G EO 1g2e...@gmail.com wrote: The snapshots underneath are set read only anyway in my case, since the they landed there by btrfs receive or I am thinking wrong here? Duncan, do you remember the bug number? I don't. I remember seeing the discussion on the

Re: [PATCH v3 00/15] xfstests: new btrfs stress test cases

2014-09-25 Thread Eryu Guan
On Thu, Sep 18, 2014 at 01:49:24PM +0800, Eryu Guan wrote: This patchset add new stress test cases for btrfs by running two different btrfs operations simultaneously under fsstress to ensure btrfs doesn't hang or oops in such situations. btrfs scrub and btrfs check will be run after each test.

[PATCH v4 01/15] btrfs: new test to run btrfs balance and subvolume test simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs balance and subvolume create/mount/umount/delete simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 110 +++-- tests/btrfs/059 | 115

[PATCH v4 05/15] btrfs: new case to run btrfs balance and device replace operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs balance and replace operations simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 73 +++ tests/btrfs/063 | 122

[PATCH v4 06/15] btrfs: new case to run btrfs subvolume create/delete operations and device replace simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs subvolume create/mount/umount/delete and device replace operation simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/064 | 123 tests/btrfs/064.out | 2 +

[PATCH v4 00/15] xfstests: new btrfs stress test cases

2014-09-25 Thread Eryu Guan
This patchset add new stress test cases for btrfs by running two different btrfs operations simultaneously under fsstress to ensure btrfs doesn't hang or oops in such situations. btrfs scrub and btrfs check will be run after each test. The test matrix is the combination of 6 btrfs operations:

[PATCH v4 04/15] btrfs: new case to run btrfs balance and remount with different compress algorithms

2014-09-25 Thread Eryu Guan
Run btrfs balance and remount with different compress algorithms simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 14 +++ tests/btrfs/062 | 114

[PATCH v4 02/15] btrfs: new test to run btrfs balance and scrub simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs balance and scrub operations simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 9 + tests/btrfs/060 | 114 tests/btrfs/060.out | 2 + tests/btrfs/group

[PATCH v4 10/15] btrfs: new case to run device replace and scrub operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs replace operations and scrub simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/068 | 123 tests/btrfs/068.out | 2 + tests/btrfs/group | 1 + 3 files changed, 126

[PATCH v4 07/15] btrfs: new case to run btrfs subvolume create/delete operations and scrub simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs subvolume create/mount/umount/delete and btrfs scrub operation simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/065 | 115 tests/btrfs/065.out | 2 +

[PATCH v4 03/15] btrfs: new test to run btrfs balance and defrag operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs balance and defrag operations simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 20 + tests/btrfs/061 | 116 tests/btrfs/061.out | 2 +

[PATCH v4 09/15] btrfs: new case to run subvolume create/delete and remount with defferent compress algorithms

2014-09-25 Thread Eryu Guan
Run btrfs subvolume create/mount/umount/delete and remount with different compress algorithms simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/067 | 116 tests/btrfs/067.out |

[PATCH v4 08/15] btrfs: new case to run btrfs subvolume create/delete and defrag operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs subvolume create/mount/umount/delete and btrfs defrag operations simultaneously, with fsstress running in backgound. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/066 | 117 tests/btrfs/066.out | 2 +

[PATCH v4 11/15] btrfs: new case to run device replace and defrag operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs replace operations and defrag simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/069 | 125 tests/btrfs/069.out | 2 + tests/btrfs/group | 1 + 3 files changed, 128

[PATCH v4 14/15] btrfs: new case to run btrfs scrub and remount with different compress algorithms simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs scrub and remount with different compress algorithms simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/072 | 114 tests/btrfs/072.out | 2 + tests/btrfs/group | 1

[PATCH v4 12/15] btrfs: new case to run device replace and remount with different compress algorithms simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs replace operations and remount with different compress algorithms simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/070 | 123 tests/btrfs/070.out | 2 +

[PATCH v4 15/15] btrfs: new case to run defrag and remount with different compress algorithms simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs defrag operations and remount with different compress algorithms simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/073 | 116 tests/btrfs/073.out | 2 +

[PATCH v4 13/15] btrfs: new case to run btrfs scrub and defrag operations simultaneously

2014-09-25 Thread Eryu Guan
Run btrfs scrub and defrag operations simultaneously with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- tests/btrfs/071 | 116 tests/btrfs/071.out | 2 + tests/btrfs/group | 1 + 3 files changed, 119