Re: Status of RAID5/6

2018-03-30 Thread Goffredo Baroncelli
On 03/31/2018 07:03 AM, Zygo Blaxell wrote: >>> btrfs has no optimization like mdadm write-intent bitmaps; recovery >>> is always a full-device operation. In theory btrfs could track >>> modifications at the chunk level but this isn't even specified in the >>> on-disk format, much less implemented

Re: Status of RAID5/6

2018-03-30 Thread Zygo Blaxell
On Fri, Mar 30, 2018 at 06:14:52PM +0200, Goffredo Baroncelli wrote: > On 03/29/2018 11:50 PM, Zygo Blaxell wrote: > > On Wed, Mar 21, 2018 at 09:02:36PM +0100, Christoph Anton Mitterer wrote: > >> Hey. > >> > >> Some things would IMO be nice to get done/clarified (i.e. documented in > >> the Wiki

Re: Status of RAID5/6

2018-03-30 Thread Zygo Blaxell
On Fri, Mar 30, 2018 at 09:21:00AM +0200, Menion wrote: > Thanks for the detailed explanation. I think that a summary of this > should go in the btrfs raid56 wiki status page, because now it is > completely inconsistent and if a user comes there, ihe may get the > impression that the raid56 is jus

Re: [PATCH] Btrfs: print error messages when failing to read trees

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 06:11:45AM +0800, Liu Bo wrote: > When mount fails to read trees like fs tree, checksum tree, extent > tree, etc, there is not enough information about where went wrong. > > With this, messages like > > "BTRFS warning (device sdf): failed to read root (objectid=7): -5" >

Re: [PATCH] btrfs: delayed-inode: Don't double reserve qgroup metadata for btrfs_delayed_item_reserve_metadata()

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 08:49:22AM +0800, Qu Wenruo wrote: > [BUG] > With latest qgroup metadata reservation patch applied, it's possible to > hit BUG_ON() when running btrfs/042: > -- > run fstests btrfs/042 at 2018-03-28 12:14:26 > BTRFS: device fsid cc876c27-bf31-44d6-bd6a-2c19b8c8e1b8 dev

Re: [PATCH v2.1 2/2] btrfs: Validate child tree block's level and first key

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 09:08:11AM +0800, Qu Wenruo wrote: > We have several reports about node pointer points to incorrect child > tree blocks, which could have even wrong owner and level but still with > valid generation and checksum. > > Although btrfs check could handle it and print error mess

[PATCH] Btrfs: bail out on error during replay_dir_deletes

2018-03-30 Thread Liu Bo
If errors were returned by btrfs_next_leaf(), replay_dir_deletes needs to bail out, otherwise @ret would be forced to be 0 after 'break;' and the caller won't be aware of it. Signed-off-by: Liu Bo --- fs/btrfs/tree-log.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/b

[PATCH] Btrfs: fix NULL pointer dereference in log_dir_items

2018-03-30 Thread Liu Bo
0, 1 and <0 can be returned by btrfs_next_leaf(), and when <0 is returned, path->nodes[0] could be NULL, log_dir_items lacks such a check for <0 and we may run into a null pointer dereference panic. Signed-off-by: Liu Bo --- fs/btrfs/tree-log.c | 7 +-- 1 file changed, 5 insertions(+), 2 del

[PATCH] Btrfs: do not abort transaction when failing to insert hole extent

2018-03-30 Thread Liu Bo
This is running in a typical write path, not inside a critical path where we have to abort the running transaction, so it's OK to return errors to callers and eventually to userspace. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git

[PATCH] Btrfs: clean up resources during umount after trans is aborted

2018-03-30 Thread Liu Bo
Currently if some fatal errors occur, like all IO get -EIO, resources would be cleaned up when a) transaction is being committed or b) BTRFS_FS_STATE_ERROR is set However, in some rare cases, resources may be left alone after transaction gets aborted and umount may run into some ASSERT(), e.g. ASS

Re: [PATCH 0/5] libbtrfsutil: misc fixes

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 12:53:52AM -0700, Omar Sandoval wrote: > From: Omar Sandoval > This is a handful of minor fixes for libbtrfsutil that would be good to > get in before the release. Patch 1 fixes an issue reported by Tomohiro a > few weeks back. Patch 2 fixes a memory leak I noticed while wr

Re: [PATCH 0/3] btrfs-progs: Enhance btrfs-image to handle missing device

2018-03-30 Thread David Sterba
On Fri, Mar 30, 2018 at 03:35:25PM +0800, Qu Wenruo wrote: > Can be fetched from github: > https://github.com/adam900710/btrfs-progs/tree/btrfs_image_fix > > Bug report: > https://github.com/kdave/btrfs-progs/issues/118 > > In short, the problem is caused by some old code (read_extent_data() > fr

Re: [PATCH 3/3] btrfs-progs: tests/misc: Test if btrfs-image can handle RAID1 missing device

2018-03-30 Thread David Sterba
On Fri, Mar 30, 2018 at 03:35:28PM +0800, Qu Wenruo wrote: > Signed-off-by: Qu Wenruo Applied with the fixes below, thanks. > --- > tests/misc-tests/030-missing-device-image/test.sh | 57 > +++ > 1 file changed, 57 insertions(+) > create mode 100755 tests/misc-tests/030-mi

Re: [PATCH] btrfs-progs: tests/mkfs: Test if mkfs.btrfs --rootdir can handle broken soft link well

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 09:26:44AM +0800, Qu Wenruo wrote: > Signed-off-by: Qu Wenruo Test fixed and applied, thanks. > --- > .../016-rootdir-bad-symbolic-link/test.sh | 26 > ++ > 1 file changed, 26 insertions(+) > create mode 100755 tests/mkfs-tests/016-rootdir-

Re: [PATCH v2 3/3] btrfs-progs: fi usage: cleanup unnecessary permission error check

2018-03-30 Thread David Sterba
On Thu, Mar 29, 2018 at 05:23:39PM +0900, Misono Tomohiro wrote: > Since BTRFS_IOC_FS_INFO does not require root privilege, there is no > need to check EPERM error. Well, this has been changed in 3.16, but there's still 3.2 longterm kernel. It does not hurt to have the check and verbose message so

Re: [PATCH] btrfs-progs: Remove unused parameter

2018-03-30 Thread David Sterba
On Fri, Mar 30, 2018 at 08:49:55AM +0300, Nikolay Borisov wrote: > > > On 30.03.2018 05:56, Gu Jinxiang wrote: > > Parameter usagestr is not used, remove it. > > > > Signed-off-by: Gu Jinxiang > > Reviewed-by: Nikolay Borisov Applied, thanks. -- To unsubscribe from this list: send the line "

Re: Question, will ls -l eventually be able to show subvolumes?

2018-03-30 Thread Adam Borowski
On Fri, Mar 30, 2018 at 10:42:10AM +0100, Pete wrote: > I've just notice work going on to make rmdir be able to delete > subvolumes. Is there an intent to allow ls -l to display directories as > subvolumes? That's entirely up to coreutils guys. Meow! -- ᛊᚨᚾᛁᛏᚣ᛫ᛁᛊ᛫ᚠᛟᚱ᛫ᚦᛖ᛫ᚹᛖᚨᚲ -- To unsubscribe

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > On 2018/03/30 2:35, Goffredo Baroncelli wrote: >> Hi Misono, > > Hello, [...] >> My conclusion, is that your work is not consistent with the current >> implementation; so I am suggesting to create a new subcommand ("btrfs sub >> tree" ?) where you

Re: [RFC PATCH v3 0/7] btrfs-progs: Allow normal user to call "subvolume list/show"

2018-03-30 Thread Goffredo Baroncelli
On 03/30/2018 06:46 AM, Misono Tomohiro wrote: > [Un]fortunately, the stock "btrfs sub list" has the capability to show all > the subvolumes, even the ones not mounted, so this can be entirely replaced > by your API. s/can be entirely/can't be entirely/ -- gpg @keyserver.linux.it: Goffredo Bar

Re: Status of RAID5/6

2018-03-30 Thread Goffredo Baroncelli
On 03/29/2018 11:50 PM, Zygo Blaxell wrote: > On Wed, Mar 21, 2018 at 09:02:36PM +0100, Christoph Anton Mitterer wrote: >> Hey. >> >> Some things would IMO be nice to get done/clarified (i.e. documented in >> the Wiki and manpages) from users'/admin's POV: [...] > >> - changing raid lvls? > >

[PATCH] btrfs: Replace owner argument in add_pinned_bytes with a boolean

2018-03-30 Thread Nikolay Borisov
add_pinned_bytes reallyc ares whether the bytes being pinned are either data or metadata. To that effect is checks whether the 'owner' argument is less than BTRFS_FIRST_FREE_OBJECTID (256). This works because owner can really have 2 types of values: a) For metadata extents it holds the level at wh

Question, will ls -l eventually be able to show subvolumes?

2018-03-30 Thread Pete
I've just notice work going on to make rmdir be able to delete subvolumes. Is there an intent to allow ls -l to display directories as subvolumes? Pete -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo in

Re: [PATCH 5/5] libbtrfsutil: fix test assumptions about top-level subvolume

2018-03-30 Thread Misono Tomohiro
On 2018/03/29 16:53, Omar Sandoval wrote: > From: Omar Sandoval > > Since "btrfs-progs: mkfs: add uuid and otime to ROOT_ITEM of, FS_TREE", > the top-level subvolume has a non-zero UUID, ctime, and otime. Fix the > subvolume_info() test to not check for zero. Sorry, I didn't notice this. I chec

[PATCH 0/3] btrfs-progs: Enhance btrfs-image to handle missing device

2018-03-30 Thread Qu Wenruo
Can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/btrfs_image_fix Bug report: https://github.com/kdave/btrfs-progs/issues/118 In short, the problem is caused by some old code (read_extent_data() from ancient btrfs check code) and offset-by-one from btrfs-image. Which make

[PATCH 1/3] btrfs-progs: disk-io: Fix read_extent_data() error handler for missing device

2018-03-30 Thread Qu Wenruo
When device is missing, read_extent_data() (function exported from old btrfs check code) has the following problems: 1) Modify @len parameter if device is missing If device returned in @multi is missing, @len can be larger than @max_len (originl length). This could confusing caller and u

[PATCH 2/3] btrfs-progs: convert: Fix offset-by-one error in read_data_extent()

2018-03-30 Thread Qu Wenruo
For read_data_extent() in convert/main.c it's using mirror number in a incorrect way, which will not get correct copy for RAID1: -- for (cur_mirror = 0; cur_mirror < num_copies; cur_mirror++) { -- In such case, for RAID1 @cur_mirror will only be 0 and 1. However for 0 and 1 case,

[PATCH 3/3] btrfs-progs: tests/misc: Test if btrfs-image can handle RAID1 missing device

2018-03-30 Thread Qu Wenruo
Signed-off-by: Qu Wenruo --- tests/misc-tests/030-missing-device-image/test.sh | 57 +++ 1 file changed, 57 insertions(+) create mode 100755 tests/misc-tests/030-missing-device-image/test.sh diff --git a/tests/misc-tests/030-missing-device-image/test.sh b/tests/misc-tests/0

Re: Status of RAID5/6

2018-03-30 Thread Menion
Thanks for the detailed explanation. I think that a summary of this should go in the btrfs raid56 wiki status page, because now it is completely inconsistent and if a user comes there, ihe may get the impression that the raid56 is just broken Still I have the 1 bilion dollar question: from your wo

Re: [PATCH v2 0/5] btrfs-progs: extent buffer related refactor and cleanup

2018-03-30 Thread Lu Fengqi
On Fri, Mar 30, 2018 at 01:48:52PM +0800, Qu Wenruo wrote: >The patchset can be fetched from github: >https://github.com/adam900710/btrfs-progs/tree/eb_cleanup > >Just like kernel cleanup and refactors, this patchset will embed >btrfs_fs_info structure into extent_buffer. > >And fixes several possi