RE: [PATCH 0/4] btrfs-progs repair support for unaligned/mismatched device sizes

2017-10-16 Thread Ursul Hempel Egal
Hi, I have encountered the issue where a btrfs filesystem would not mount due to mismatch of super_total_bytes with fs_devices total_rw_bytes I was able to fix it with the new parameter --fix-dev-size Below I have listed the steps I took. Reference https://www.spinics.net/lists/linux-btrfs/msg

Re: [PATCH v3 2/2] btrfs-progs: doc: add description of missing and example of device remove

2017-10-16 Thread Duncan
Misono, Tomohiro posted on Tue, 17 Oct 2017 10:45:58 +0900 as excerpted: > On 2017/10/16 18:22, Nikolay Borisov wrote: >> >> >> On 16.10.2017 07:56, Misono, Tomohiro wrote: >>> >>> 1. Add explanation of 'missing' for 'device remove'. This is only >>> written in wikipage currently. >>> +++ b/Doc

kernel BUG at fs/btrfs/ctree.c:3182

2017-10-16 Thread Matt McKinnon
Hi All, Been having issues on one machine and I was wondering if I could get some help tracking the issue down. # uname -a Linux riperton 4.13.5-custom #1 SMP Sat Oct 7 18:28:16 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux # btrfs --version btrfs-progs v4.13.3 # btrfs fi show Label: none uuid:

Unmountable fs - missing generation?

2017-10-16 Thread Larkin Lowrey
I am unable to mount one my my filesystems.  The superblock thinks the latest generation is 2220927 but I can't seem to find a root with that number. I can find 2220926 and 2220928 but not 2220927. Is there anything that I can do to recover this FS? # btrfs check /dev/Cached/Backups checksum v

Re: [PATCH v3 2/2] btrfs-progs: doc: add description of missing and example of device remove

2017-10-16 Thread Misono, Tomohiro
On 2017/10/16 18:22, Nikolay Borisov wrote: > > > On 16.10.2017 07:56, Misono, Tomohiro wrote: >> This patch updates help/document of "btrfs device remove" in two points: >> >> 1. Add explanation of 'missing' for 'device remove'. This is only >> written in wikipage currently. >> (https://btrfs.wi

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-16 Thread Adam Borowski
On Mon, Oct 16, 2017 at 01:27:40PM -0400, Austin S. Hemmelgarn wrote: > On 2017-10-16 12:57, Zoltan wrote: > > On Mon, Oct 16, 2017 at 1:53 PM, Austin S. Hemmelgarn wrote: > In an ideal situation, scrubbing should not be an 'only if needed' thing, > even for a regular array that isn't dealing with

Re: [PATCH] btrfs: remove BUG_ON in btrfs_rm_dev_replace_free_srcdev()

2017-10-16 Thread Anand Jain
On 10/16/2017 10:45 PM, David Sterba wrote: On Thu, Oct 12, 2017 at 05:00:41PM +0800, Anand Jain wrote: That was only an extra check to tackle few bugs around this area, now its save to remove it. Signed-off-by: Anand Jain --- fs/btrfs/volumes.c | 8 ++-- 1 file changed, 2 insertions(

[PATCH v2] btrfs: remove BUG_ON in btrfs_rm_dev_replace_free_srcdev()

2017-10-16 Thread Anand Jain
Use ASSERT instead of BUG_ON. Signed-off-by: Anand Jain --- V2: Accepts David suggestion to keep the check as ASSERT. Adds more comments to explain the context. fs/btrfs/volumes.c | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btr

Re: [PATCH] btrfs-progs: qgroup: show subvol path when qgroup show

2017-10-16 Thread David Sterba
On Tue, Oct 10, 2017 at 04:37:09PM +0800, Lu Fengqi wrote: > Show the absolute subvol path for the associated level-0 qgroup. > > Signed-off-by: Lu Fengqi > --- > Documentation/btrfs-qgroup.asciidoc | 2 + > cmds-qgroup.c | 7 +- > qgroup.c|

Re: [PATCH v2] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
On Sat, Oct 14, 2017 at 01:33:17PM +0800, Eryu Guan wrote: > On Fri, Oct 13, 2017 at 01:40:05PM -0600, Liu Bo wrote: > > Currently running 'btrfs device delete' can end up with losing data > > raid profile (if any), this test is to reproduce the problem. > > > > The fix is > > "Btrfs: avoid l

[PATCH v2 RESEND] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
Currently running 'btrfs device delete' can end up with losing data raid profile (if any), this test is to reproduce the problem. The fix is "Btrfs: avoid losing data raid profile when deleting a device" Signed-off-by: Liu Bo --- v2: - Add _require_btrfs_dev_del_by_devid and limit the test

Re: [PATCH] Fstest: btrfs/151: test if device delete ends up with losing raid profile

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 09:57:11AM +0300, Nikolay Borisov wrote: > > > On 13.10.2017 21:08, Liu Bo wrote: > > On Thu, Oct 12, 2017 at 03:06:57PM +0800, Eryu Guan wrote: > >> On Mon, Oct 09, 2017 at 11:39:21AM -0600, Liu Bo wrote: > >>> Currently running 'btrfs device delete' can end up with losin

Re: [PATCH] Btrfs: add write_flags for compression bio

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 03:33:10PM +0200, David Sterba wrote: > On Fri, Oct 13, 2017 at 03:10:35PM -0600, Liu Bo wrote: > > Compression code path has only flaged bios with REQ_OP_WRITE no matter > > where the bios come from. This breaks the rule that sync writes and > > writeback writes need to be

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-16 Thread Austin S. Hemmelgarn
On 2017-10-16 12:57, Zoltan wrote: Hi, On Mon, Oct 16, 2017 at 1:53 PM, Austin S. Hemmelgarn wrote: you will need to scrub regularly to avoid data corruption Is there any indication that a scrub is needed? Before actually doing a scrub, is btrfs already aware that one of the devices did not

Re: [PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

2017-10-16 Thread Liu Bo
On Mon, Oct 16, 2017 at 12:22:44PM +0800, Anand Jain wrote: > > > On 10/14/2017 04:51 AM, Liu Bo wrote: > >On Wed, Oct 11, 2017 at 10:38:50AM +0300, Nikolay Borisov wrote: > >> > >> > >>On 10.10.2017 20:53, Liu Bo wrote: > >>>We've avoided data losing raid profile when doing balance, but it > >>>

Btrfs progs release 4.13.3

2017-10-16 Thread David Sterba
Hi, btrfs-progs version 4.13.3 have been released. This is a bugfix release, namely for the subvolume listing and deletion commands (again), and 'check --force'. Both were not caught by faulty tests, now fixed. Changes: * check: fix --force, wrong check for a mounted block device * build: fi

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-16 Thread Zoltan
Hi, On Mon, Oct 16, 2017 at 1:53 PM, Austin S. Hemmelgarn wrote: > you will need to scrub regularly to avoid data corruption Is there any indication that a scrub is needed? Before actually doing a scrub, is btrfs already aware that one of the devices did not receive all data due to being unavail

Re: Why isnt NOCOW attributes propogated on snapshot transfers?

2017-10-16 Thread Graham Cobb
On 16/10/17 14:28, David Sterba wrote: > On Sun, Oct 15, 2017 at 04:19:23AM +0300, Cerem Cem ASLAN wrote: >> `btrfs send | btrfs receive` removes NOCOW attributes. Is it a bug or >> a feature? If it's a feature, how can we keep these attributes if we >> need to? > > This is a known defficiency of

Re: [PATCH v3 0/4] btrfs: cleanup mount path

2017-10-16 Thread David Sterba
On Mon, Sep 25, 2017 at 04:26:30PM +0900, Misono, Tomohiro wrote: > Summary: > Cleanup mount path by avoiding calling btrfs_mount() twice. > No functional change. See below for longer explanation. > > Changelog: > v3: > Reorganized patches again into four and added comments to the source. > Ea

Re: [PATCH 09/21] btrfs: move extent_op cleanup to a helper

2017-10-16 Thread David Sterba
On Mon, Oct 16, 2017 at 05:05:19PM +0300, Nikolay Borisov wrote: > > > On 29.09.2017 22:43, Josef Bacik wrote: > > Move the extent_op cleanup for an empty head ref to a helper function to > > help simplify __btrfs_run_delayed_refs. > > > > Signed-off-by: Josef Bacik > > --- > > fs/btrfs/extent

Re: [PATCH 10/21] btrfs: breakout empty head cleanup to a helper

2017-10-16 Thread David Sterba
On Mon, Oct 16, 2017 at 05:07:21PM +0300, Nikolay Borisov wrote: > > > On 29.09.2017 22:43, Josef Bacik wrote: > > Move this code out to a helper function to further simplivy > > __btrfs_run_delayed_refs. > > > > Signed-off-by: Josef Bacik > > --- > > fs/btrfs/extent-tree.c | 80 > > +

Re: [PATCH v2] btrfs: Replace opencoded sizes with their symbolic constants

2017-10-16 Thread David Sterba
On Mon, Oct 16, 2017 at 04:48:40PM +0300, Nikolay Borisov wrote: > Currently btrfs' code uses a mix of opencoded sizes and defines from sizes.h. > Let's unifiy the code base to always use the symbolic constants. No functional > changes > > Signed-off-by: Nikolay Borisov Reviewed-by: David Sterba

Re: [PATCH] btrfs: remove BUG_ON in btrfs_rm_dev_replace_free_srcdev()

2017-10-16 Thread David Sterba
On Thu, Oct 12, 2017 at 05:00:41PM +0800, Anand Jain wrote: > That was only an extra check to tackle few bugs around this > area, now its save to remove it. > > Signed-off-by: Anand Jain > --- > fs/btrfs/volumes.c | 8 ++-- > 1 file changed, 2 insertions(+), 6 deletions(-) > > diff --git a/

Re: [PATCH 13/21] btrfs: remove delayed_ref_node from ref_head

2017-10-16 Thread Nikolay Borisov
On 29.09.2017 22:43, Josef Bacik wrote: > This is just excessive information in the ref_head, and makes the code > complicated. It is a relic from when we had the heads and the refs in > the same tree, which is no longer the case. With this removal I've > cleaned up a bunch of the cruft around

Re: [PATCH v3] btrfs: Use bd_dev to generate index when dev_state_hashtable add items.

2017-10-16 Thread David Sterba
On Wed, Oct 11, 2017 at 12:22:35PM +0300, Nikolay Borisov wrote: Patch added to queue, thanks. > On a slightly different note the check-integrity.c file could really > make use of the hashtable.h API but that's a bit more work. In any case > I think what could be done to reduce the likelihood of

Re: [PATCH v2] btrfs: fix false EIO for missing device

2017-10-16 Thread David Sterba
On Sat, Oct 14, 2017 at 08:34:02AM +0800, Anand Jain wrote: > When one of the device is missing, bbio_error() takes care > of setting the error status. And if its only IO that is > pending in that stripe, it fails to check the status of the > other IO at %bbio_error before setting the error %bi_sta

Re: [PATCH] btrfs: use need_full_stripe() in __btrfs_map_block()

2017-10-16 Thread Nikolay Borisov
On 16.10.2017 17:17, David Sterba wrote: > You confused me first by the bitwise operators here, while the patch is > actually for the logical ones (and correct). actualy I used ~ as the "not" and ^ as "and" rather than XOR. Not A AND Not B == NOT(A or B) that's what I wanted to express ... -- T

Re: [PATCH] btrfs: use need_full_stripe() in __btrfs_map_block()

2017-10-16 Thread David Sterba
On Thu, Oct 12, 2017 at 12:03:31PM +0300, Nikolay Borisov wrote: > > > On 12.10.2017 11:43, Anand Jain wrote: > > A cleanup patch, use need_full_stripe() to replace the open code. > > The code is equivalent by merit of De Morgan's law (I had to actually go > look up the equivalence classes befor

Re: [PATCH v2 3/3] btrfs-progs: device: add remove missing-all

2017-10-16 Thread Duncan
Misono, Tomohiro posted on Mon, 16 Oct 2017 13:35:08 +0900 as excerpted: > On 2017/10/16 12:30, Anand Jain wrote: >> >> >> On 10/13/2017 01:27 PM, Duncan wrote: >>> Misono, Tomohiro posted on Wed, 11 Oct 2017 11:18:50 +0900 as >>> excerpted: >>> Add 'btrfs remove missing-all' to remove all

Re: [PATCH] btrfs: fix call to btrfs_end_transaction without a transaction handler

2017-10-16 Thread David Sterba
On Fri, Oct 13, 2017 at 10:16:54AM +0800, Anand Jain wrote: > In btrfs_init_new_device() when btrfs_attach_transaction() fails we > shouldn't call btrfs_end_transaction(). Fix it. > > Signed-off-by: Anand Jain > Fixes: > btrfs: error out if btrfs_attach_transaction() fails > --- > Hi David, >

Re: [PATCH 10/21] btrfs: breakout empty head cleanup to a helper

2017-10-16 Thread Nikolay Borisov
On 29.09.2017 22:43, Josef Bacik wrote: > Move this code out to a helper function to further simplivy > __btrfs_run_delayed_refs. > > Signed-off-by: Josef Bacik > --- > fs/btrfs/extent-tree.c | 80 > -- > 1 file changed, 45 insertions(+), 35 del

Re: [PATCH 09/21] btrfs: move extent_op cleanup to a helper

2017-10-16 Thread Nikolay Borisov
On 29.09.2017 22:43, Josef Bacik wrote: > Move the extent_op cleanup for an empty head ref to a helper function to > help simplify __btrfs_run_delayed_refs. > > Signed-off-by: Josef Bacik > --- > fs/btrfs/extent-tree.c | 77 > ++ > 1 file change

Re: [PATCH v2] btrfs: cleanup extent locking sequence

2017-10-16 Thread David Sterba
On Mon, Oct 16, 2017 at 05:43:21AM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Code cleanup for better understanding: > Variable needs_unlock to be called extent_locked to show state as opposed to > action. Changed the type to int, to reduce code in the critical > path. > > Sig

[PATCH v2] btrfs: Replace opencoded sizes with their symbolic constants

2017-10-16 Thread Nikolay Borisov
Currently btrfs' code uses a mix of opencoded sizes and defines from sizes.h. Let's unifiy the code base to always use the symbolic constants. No functional changes Signed-off-by: Nikolay Borisov --- Changes since v1: * Don't modify values which represent counts rather than bytes fs/btrfs/ctr

Re: [PATCH] btrfs: use BLK_STS defines where needed

2017-10-16 Thread David Sterba
On Sat, Oct 14, 2017 at 08:35:56AM +0800, Anand Jain wrote: > At few places we could use BLK_STS_OK and BLK_STS_NOSUPP. > > Signed-off-by: Anand Jain > --- > fs/btrfs/compression.c | 3 ++- > fs/btrfs/inode.c | 4 ++-- > fs/btrfs/volumes.c | 2 +- > 3 files changed, 5 insertions(+), 4

Re: [PATCH] btrfs: Replace opencoded sizes with their symbolic constants

2017-10-16 Thread David Sterba
On Mon, Oct 16, 2017 at 04:29:56PM +0300, Nikolay Borisov wrote: > Currently btrfs' code uses a mix of opencoded sizes and defines from sizes.h. > Let's unifiy the code base to always use the symbolic constants. No functional > changes > > -#define BTRFS_DEFRAG_BATCH 1024 > +#define BTRFS_DEFRAG

Re: [PATCH] Btrfs: add write_flags for compression bio

2017-10-16 Thread David Sterba
On Fri, Oct 13, 2017 at 03:10:35PM -0600, Liu Bo wrote: > Compression code path has only flaged bios with REQ_OP_WRITE no matter > where the bios come from. This breaks the rule that sync writes and > writeback writes need to be differentiated from each other. What are the implications of the cha

Re: Why isnt NOCOW attributes propogated on snapshot transfers?

2017-10-16 Thread David Sterba
On Sun, Oct 15, 2017 at 04:19:23AM +0300, Cerem Cem ASLAN wrote: > `btrfs send | btrfs receive` removes NOCOW attributes. Is it a bug or > a feature? If it's a feature, how can we keep these attributes if we > need to? This is a known defficiency of send protocol v1. And there are more, listed on

[PATCH] btrfs: Replace opencoded sizes with their symbolic constants

2017-10-16 Thread Nikolay Borisov
Currently btrfs' code uses a mix of opencoded sizes and defines from sizes.h. Let's unifiy the code base to always use the symbolic constants. No functional changes Signed-off-by: Nikolay Borisov --- fs/btrfs/ctree.h | 2 +- fs/btrfs/disk-io.c | 2 +- fs/btrfs/file.c

[PATCH] btrfs: remove stillborn code

2017-10-16 Thread Nikolay Borisov
This code was first introduced in 31db9f7c23fb ("Btrfs: introduce BTRFS_IOC_SEND for btrfs send/receive") and it was not functional, then it got slightly refactored in e938c8ad543c ("Btrfs: code cleanups for send/receive"), alas it was still dead. So let's remove it for good! Signed-off-by: Nikola

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-16 Thread Austin S. Hemmelgarn
On 2017-10-15 04:30, Zoltán Ivánfi wrote: Hi, Thanks for the replies. As you both pointed out, I shouldn't have described the issue having to do with hotplugging. I got confused by this use-case being somewhat emphasized in the description of the bug I linked to. As for the question of why I th

Re: btrfs errors over NFS

2017-10-16 Thread Nikolay Borisov
On 16.10.2017 08:00, Steve Leung wrote: > On 10/13/2017 03:14 AM, Nikolay Borisov wrote: >> >> >> On 12.10.2017 20:44, Steve Leung wrote: >>> I've run into a btrfs error trying to do a -j8 build of android on a >>> btrfs >>> filesystem exported over NFSv3.  That in itself might be unwise, but >>>

[PATCH v2] btrfs: cleanup extent locking sequence

2017-10-16 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Code cleanup for better understanding: Variable needs_unlock to be called extent_locked to show state as opposed to action. Changed the type to int, to reduce code in the critical path. Signed-off-by: Goldwyn Rodrigues --- Changes since v1: fixed using extents_l

Assalamu`Alaikum.

2017-10-16 Thread Mohammad Ouattara
Greetings from Dr. mohammad ouattara. Assalamu`Alaikum. My Name is Dr. mohammad ouattara, I am a banker by profession. I'm from Ouagadougou, Burkina Faso, West Africa. My reason for contacting you is to transfer an abandoned $14.6M to your account. The owner of this fund died since 2004 with his

Re: [PATCH v3 2/2] btrfs-progs: doc: add description of missing and example of device remove

2017-10-16 Thread Nikolay Borisov
On 16.10.2017 07:56, Misono, Tomohiro wrote: > This patch updates help/document of "btrfs device remove" in two points: > > 1. Add explanation of 'missing' for 'device remove'. This is only > written in wikipage currently. > (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devi

Re: [PATCH v2 2/3] btrfs-progs: mkfs: Enhance minimal device size calculation to fix mkfs failure on small file

2017-10-16 Thread Nikolay Borisov
On 16.10.2017 11:22, Qu Wenruo wrote: > Since commit c11e36a29e84 ("Btrfs-progs: Do not force mixed block group > creation unless '-M' option is specified"), mkfs no longer use mixed > block group unless specified manually. > > This breaks the minimal device size calculation, which only consider

Re: [PATCH v2] Btrfs: avoid losing data raid profile when deleting a device

2017-10-16 Thread Nikolay Borisov
On 13.10.2017 23:51, Liu Bo wrote: > On Wed, Oct 11, 2017 at 10:38:50AM +0300, Nikolay Borisov wrote: >> >> >> On 10.10.2017 20:53, Liu Bo wrote: >>> We've avoided data losing raid profile when doing balance, but it >>> turns out that deleting a device could also result in the same >>> problem >>

Re: [PATCH v2 1/3] btrfs-progs: Refactor test_minimum_size to use calculated minimal device size

2017-10-16 Thread Lu Fengqi
On Mon, Oct 16, 2017 at 04:22:56PM +0800, Qu Wenruo wrote: >test_minimum_size() function is only called to check if provided device >is large enough. > >However the minimal device size only needs to be calculated once, and >can be reused everywhere. > >Refactor that function to make later modificat

[PATCH v2 2/3] btrfs-progs: mkfs: Enhance minimal device size calculation to fix mkfs failure on small file

2017-10-16 Thread Qu Wenruo
Since commit c11e36a29e84 ("Btrfs-progs: Do not force mixed block group creation unless '-M' option is specified"), mkfs no longer use mixed block group unless specified manually. This breaks the minimal device size calculation, which only considered mixed block group use case. This patch enhance

[PATCH v2 3/3] btrfs-progs: test/mkfs: Test if the minimal device size is valid

2017-10-16 Thread Qu Wenruo
New test case to test if the minimal device size given by "mkfs.btrfs" failure case is valid. Signed-off-by: Qu Wenruo --- v2: Nothing --- tests/common | 57 +++- tests/mkfs-tests/010-small-image/test.sh | 51

[PATCH v2 1/3] btrfs-progs: Refactor test_minimum_size to use calculated minimal device size

2017-10-16 Thread Qu Wenruo
test_minimum_size() function is only called to check if provided device is large enough. However the minimal device size only needs to be calculated once, and can be reused everywhere. Refactor that function to make later modification easier. Signed-off-by: Qu Wenruo --- v2: New patch, cleanu