[PATCH v2 4/6] btrfs-progs: mkfs: Error out gracefully for --rootdir

2017-10-18 Thread Qu Wenruo
--rootdir option will start a transaction to fill the fs, however if something goes wrong, from ENOSPC to lack of permission, we won't commit transaction and cause BUG_ON trigger by uncommitted transaction: -- extent buffer leak: start 29392896 len 16384 extent_io.c:579: free_extent_buffer: BU

[PATCH v2 3/6] btrfs-progs: mkfs: Fix overwritten return value for mkfs

2017-10-18 Thread Qu Wenruo
For mkfs failure, especially --rootdir errors like EPERM/ENOSPC, the out branch will overwrite return value, causing wrong status code. Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- mkfs/main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mkfs/main.c b/mk

[PATCH v2 2/6] btrfs-progs: mkfs: Avoid positive return value from cleanup_temp_chunks

2017-10-18 Thread Qu Wenruo
Since we're calling btrfs_search_slot() the return value can be positive. However we just pass that return value out, causing undefined return value. This can cause mkfs return 1, which indicates something wrong. Fix it. Signed-off-by: Qu Wenruo --- mkfs/main.c | 3 +++ 1 file changed, 3 inser

[PATCH v2 6/6] btrfs-progs: mkfs: Move source dir size calculation to its own files

2017-10-18 Thread Qu Wenruo
Also rename the function from size_sourcedir() to mkfs_size_dir(). Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- mkfs/main.c| 66 ++ mkfs/rootdir.c | 63 +++ mkfs/rootdir.

[PATCH v2 5/6] btrfs-progs: mkfs: Move image creation of rootdir to its own files

2017-10-18 Thread Qu Wenruo
In fact, --rootdir option is getting more and more independent from normal mkfs code. So move image creation function, make_image() and its related code to mkfs/rootdir.[ch], and rename the function to btrfs_mkfs_fill_dir(). Signed-off-by: Qu Wenruo Reviewed-by: Nikolay Borisov --- Makefile

[PATCH v2 1/6] btrfs-progs: Avoid BUG_ON for chunk allocation when ENOSPC happens

2017-10-18 Thread Qu Wenruo
When passing directory larger than block device using --rootdir parameter, we get the following backtrace: -- extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28 ./mkfs.btrfs(+0x1a05d)[0x557939e6b05d] ./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8] ./mkfs.btrfs

[PATCH v2 0/6] Rootdir refactor and small bug fixes

2017-10-18 Thread Qu Wenruo
Sorry for the v2 patchset, just added a new 3-line patch. But since it can screw up bisect, I re-send the whole patchset, to make the new patch just before mkfs return value fix, so bisect will work as it used to do. First 4 patches are small bug fixes which can be applied even we don't touch the

Re: [PATCH 01/21] Btrfs: rework outstanding_extents

2017-10-18 Thread Edmund Nadolski
just a few quick things for the changelog: On 09/29/2017 01:43 PM, Josef Bacik wrote: > Right now we do a lot of weird hoops around outstanding_extents in order > to keep the extent count consistent. This is because we logically > transfer the outstanding_extent count from the initial reservation

[PATCH v2] btrfs: Fix bug for misused dev_t when lookup in dev state hash table.

2017-10-18 Thread Gu Jinxiang
From: Gu JinXiang Fix bug of commit 74d46992e0d9 ("block: replace bi_bdev with a gendisk pointer and partitions index"). In this modify, use bio_dev(bio) to find dev state in function __btrfsic_submit_bio. But when dev_state added to hashtable, it is using dev_t of block_device. bio_dev(bio) re

Re: SOLVED - 32-bit kernel 4.13 bug - Mount failing - unable to find logical

2017-10-18 Thread Cameron Kelley
On 10-17-2017 10:10 PM, Roman Mamedov wrote: On Wed, 18 Oct 2017 09:24:01 +0800 Qu Wenruo wrote: On 2017-10-18 04:43, Cameron Kelley wrote: Hey btrfs gurus, I have a 4 disk btrfs filesystem that has suddenly stopped mounting after a recent reboot. The data is in an odd configuration due

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Austin S. Hemmelgarn
On 2017-10-18 07:59, Adam Borowski wrote: On Wed, Oct 18, 2017 at 07:30:55AM -0400, Austin S. Hemmelgarn wrote: On 2017-10-17 16:21, Adam Borowski wrote: It's a single-device filesystem, thus disconnects are obviously fatal. But, they never caused even a single bit of damage (as scrub goes), t

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Austin S. Hemmelgarn
On 2017-10-18 09:53, Peter Grandi wrote: I forget sometimes that people insist on storing large volumes of data on unreliable storage... Here obviously "unreliable" is used on the sense of storage that can work incorrectly, not in the sense of storage that can fail. Um, in what world is a devic

Unmountable fs. No root for superblock generation

2017-10-18 Thread Larkin Lowrey
I am unable to mount one my my filesystems.  The superblock thinks the latest generation is 2220927 but I can't seem to find a root with that number. I can find 2220926 and 2220928 but not 2220927. Is there anything that I can do to recover this FS? # btrfs check /dev/Cached/Backups checksum v

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Peter Grandi
> [ ... ] After all, btrfs would just have to discard one copy > of each chunk. [ ... ] One more thing that is not clear to me > is the replication profile of a volume. I see that balance can > convert chunks between profiles, for example from single to > raid1, but I don't see how the default pro

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Peter Grandi
>> I forget sometimes that people insist on storing large >> volumes of data on unreliable storage... Here obviously "unreliable" is used on the sense of storage that can work incorrectly, not in the sense of storage that can fail. > In my opinion the unreliability of the storage is the exact > r

Re: [PATCH] btrfs-progs: test: add new cli-test for subvol get/set-default

2017-10-18 Thread David Sterba
On Wed, Oct 18, 2017 at 11:00:43AM +0900, Misono, Tomohiro wrote: > Add new test to check functionality of subvol get/set-default. > > Signed-off-by: Tomohiro Misono Thanks, applied with the following diff to fix style and failures when the test is not run as root initially: - no command shortc

Re: [PATCH] Btrfs: ref-verify: Fix NULL vs IS_ERR() check in walk_down_tree()

2017-10-18 Thread David Sterba
On Wed, Oct 18, 2017 at 10:36:35AM +0300, Dan Carpenter wrote: > read_tree_block() returns error pointers, and never NULL and so I have > updated the error handling. > > Fixes: 74739121b4c7 ("Btrfs: add a extent ref verify tool") > Signed-off-by: Dan Carpenter Thanks, I've folded the fix into th

Re: [PATCH 0/5] Rootdir refactor and small bug fixes

2017-10-18 Thread Nikolay Borisov
On 18.10.2017 11:00, Qu Wenruo wrote: > First 3 patches are small bug fixes which can be applied even we don't > touch the functionality of --rootdir. > > The last two patches will refactor --rootdir related functions (mainly > size_sourcedir and make_image) to mkfs/rootdir.[ch]. > And rename th

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Adam Borowski
On Wed, Oct 18, 2017 at 07:30:55AM -0400, Austin S. Hemmelgarn wrote: > On 2017-10-17 16:21, Adam Borowski wrote: > > > > It's a single-device filesystem, thus disconnects are obviously fatal. > > > > But, > > > > they never caused even a single bit of damage (as scrub goes), thus > > > > provin

Re: Best strategie to remove devices from pool

2017-10-18 Thread Austin S. Hemmelgarn
On 2017-10-17 13:58, Cloud Admin wrote: Hi, I want to remove two devices from a BTRFS RAID 1 pool. It should be enough free space to do it, but what is the best strategie. Remove both device in one call 'btrfs dev rem /dev/sda1 /dev/sdb1' (for example) or should it be better in two separate calls

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-18 Thread Austin S. Hemmelgarn
On 2017-10-17 16:21, Adam Borowski wrote: On Tue, Oct 17, 2017 at 03:19:09PM -0400, Austin S. Hemmelgarn wrote: On 2017-10-17 13:06, Adam Borowski wrote: The thing is, reliability guarantees required vary WILDLY depending on your particular use cases. On one hand, there's "even an one-minute d

[PATCH 0/5] Rootdir refactor and small bug fixes

2017-10-18 Thread Qu Wenruo
First 3 patches are small bug fixes which can be applied even we don't touch the functionality of --rootdir. The last two patches will refactor --rootdir related functions (mainly size_sourcedir and make_image) to mkfs/rootdir.[ch]. And rename them to btrfs_mkfs_size_dir() and btrfs_mkfs_fill_dir(

[PATCH 1/5] btrfs-progs: Avoid BUG_ON for chunk allocation when ENOSPC happens

2017-10-18 Thread Qu Wenruo
When passing directory larger than block device using --rootdir parameter, we get the following backtrace: -- extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28 ./mkfs.btrfs(+0x1a05d)[0x557939e6b05d] ./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8] ./mkfs.btrfs

[PATCH 3/5] btrfs-progs: mkfs: Error out gracefully for --rootdir

2017-10-18 Thread Qu Wenruo
--rootdir option will start a transaction to fill the fs, however if something goes wrong, from ENOSPC to lack of permission, we won't commit transaction and cause BUG_ON trigger by uncommitted transaction: -- extent buffer leak: start 29392896 len 16384 extent_io.c:579: free_extent_buffer: BU

[PATCH 2/5] btrfs-progs: mkfs: Fix overwritten return value for mkfs

2017-10-18 Thread Qu Wenruo
For mkfs failure, especially --rootdir errors like EPERM/ENOSPC, the out branch will overwrite return value, causing wrong status code. Signed-off-by: Qu Wenruo --- mkfs/main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mkfs/main.c b/mkfs/main.c index 1b4cabc1ef90..

[PATCH 4/5] btrfs-progs: mkfs: Move image creation of rootdir to its own files

2017-10-18 Thread Qu Wenruo
In fact, --rootdir option is getting more and more independent from normal mkfs code. So move image creation function, make_image() and its related code to mkfs/rootdir.[ch], and rename the function to btrfs_mkfs_fill_dir(). Signed-off-by: Qu Wenruo --- Makefile | 4 +- mkfs/main.c|

[PATCH 5/5] btrfs-progs: mkfs: Move source dir size calculation to its own files

2017-10-18 Thread Qu Wenruo
Also rename the function from size_sourcedir() to mkfs_size_dir(). Signed-off-by: Qu Wenruo --- mkfs/main.c| 66 ++ mkfs/rootdir.c | 63 +++ mkfs/rootdir.h | 2 ++ 3 files changed, 67

[PATCH] Btrfs: ref-verify: Fix NULL vs IS_ERR() check in walk_down_tree()

2017-10-18 Thread Dan Carpenter
read_tree_block() returns error pointers, and never NULL and so I have updated the error handling. Fixes: 74739121b4c7 ("Btrfs: add a extent ref verify tool") Signed-off-by: Dan Carpenter diff --git a/fs/btrfs/ref-verify.c b/fs/btrfs/ref-verify.c index f65d78cf3c7e..34878699d363 100644 --- a/fs/