Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Andrei Borzenkov
18.09.2018 22:11, Austin S. Hemmelgarn пишет: > On 2018-09-18 14:38, Andrei Borzenkov wrote: >> 18.09.2018 21:25, Austin S. Hemmelgarn пишет: >>> On 2018-09-18 14:16, Andrei Borzenkov wrote: 18.09.2018 08:37, Chris Murphy пишет: > On Mon, Sep 17, 2018 at 11:24 PM, Andrei Borzenkov >

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Duncan
Chris Murphy posted on Tue, 18 Sep 2018 13:34:14 -0600 as excerpted: > I've run into some issue where grub2-mkconfig and grubby, can change the > grub.cfg, and then do a really fast reboot without cleanly unmounting > the volume - and what happens? Can't boot. The bootloader can't do log > replay

Re: [patch] file dedupe (and maybe clone) data corruption (was Re: [PATCH] generic: test for deduplication between different files)

2018-09-18 Thread Zygo Blaxell
On Mon, Sep 10, 2018 at 07:06:46PM +1000, Dave Chinner wrote: > On Thu, Sep 06, 2018 at 11:53:06PM -0400, Zygo Blaxell wrote: > > On Thu, Sep 06, 2018 at 06:38:09PM +1000, Dave Chinner wrote: > > > On Fri, Aug 31, 2018 at 01:10:45AM -0400, Zygo Blaxell wrote: > > > > On Thu, Aug 30, 2018 at

Re: btrfs panic problem

2018-09-18 Thread Qu Wenruo
On 2018/9/19 上午8:35, sunny.s.zhang wrote: > > 在 2018年09月19日 08:05, Qu Wenruo 写道: >> >> On 2018/9/18 上午8:28, sunny.s.zhang wrote: >>> Hi All, >>> >>> My OS(4.1.12) panic in kmem_cache_alloc, which is called by >>> btrfs_get_or_create_delayed_node. >> Any reproducer? >> >> Anyway we need a

Re: btrfs panic problem

2018-09-18 Thread sunny.s.zhang
在 2018年09月19日 08:05, Qu Wenruo 写道: On 2018/9/18 上午8:28, sunny.s.zhang wrote: Hi All, My OS(4.1.12) panic in kmem_cache_alloc, which is called by btrfs_get_or_create_delayed_node. Any reproducer? Anyway we need a reproducer as a testcase. I have had a try, but could not  reproduce yet.

Re: [PATCH 32/36] btrfs: clear delayed_refs_rsv for dirty bg cleanup

2018-09-18 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 01:58:03PM -0400, Josef Bacik wrote: > We keep track of dirty bg's as a reservation in the delayed_refs_rsv, so > when we abort and we cleanup those dirty bgs we need to drop their > reservation so we don't have accounting issues and lots of scary > messages on umount.

Re: [PATCH 33/36] btrfs: only free reserved extent if we didn't insert it

2018-09-18 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 01:58:04PM -0400, Josef Bacik wrote: > When we insert the file extent once the ordered extent completes we free > the reserved extent reservation as it'll have been migrated to the > bytes_used counter. However if we error out after this step we'll still > clear the

Re: [PATCH 17/36] btrfs: loop in inode_rsv_refill

2018-09-18 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 01:57:48PM -0400, Josef Bacik wrote: > With severe fragmentation we can end up with our inode rsv size being > huge during writeout, which would cause us to need to make very large > metadata reservations. However we may not actually need that much once > writeout is

Re: btrfs panic problem

2018-09-18 Thread Qu Wenruo
On 2018/9/18 上午8:28, sunny.s.zhang wrote: > Hi All, > > My OS(4.1.12) panic in kmem_cache_alloc, which is called by > btrfs_get_or_create_delayed_node. Any reproducer? Anyway we need a reproducer as a testcase. The code looks > > I found that the freelist of the slub is wrong. > > crash>

Re: [PATCH 16/36] btrfs: run delayed iputs before committing

2018-09-18 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 01:57:47PM -0400, Josef Bacik wrote: > Delayed iputs means we can have final iputs of deleted inodes in the > queue, which could potentially generate a lot of pinned space that could > be free'd. So before we decide to commit the transaction for ENOPSC > reasons, run the

Re: [PATCH 14/36] btrfs: reset max_extent_size properly

2018-09-18 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 01:57:45PM -0400, Josef Bacik wrote: > If we use up our block group before allocating a new one we'll easily > get a max_extent_size that's set really really low, which will result in > a lot of fragmentation. We need to make sure we're resetting the > max_extent_size when

Re: btrfs panic problem

2018-09-18 Thread sunny.s.zhang
Hi Duncan, Thank you for your advice. I understand what you mean.  But i have reviewed the latest btrfs code, and i think the issue is exist still. At 71 line, if the function of btrfs_get_delayed_node run over this line, then switch to other process, which run over the 1282 and release the

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Mon, Sep 17, 2018 at 9:44 PM, Chris Murphy wrote: > https://btrfs.wiki.kernel.org/index.php/FAQ#Does_grub_support_btrfs.3F > > Does anyone know if this is still a problem on Btrfs if grubenv has > xattr +C set? In which case it should be possible to overwrite and > there's no csums that are

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Tue, Sep 18, 2018 at 1:11 PM, Goffredo Baroncelli wrote: >> I think it's a problem, and near as I can tell it'll be a problem for >> all kinds of complex storage. I don't see how the bootloader itself >> can do an overwrite onto raid5 or raid6. > > >> That's certainly supported by GRUB for

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Tue, Sep 18, 2018 at 1:01 PM, Andrei Borzenkov wrote: > 18.09.2018 21:57, Chris Murphy пишет: >> On Tue, Sep 18, 2018 at 12:16 PM, Andrei Borzenkov >> wrote: >>> 18.09.2018 08:37, Chris Murphy пишет: >> The patches aren't upstream yet? Will they be? >>> >>> I do not know.

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
On 18/09/2018 20.52, Chris Murphy wrote: > On Tue, Sep 18, 2018 at 11:15 AM, Goffredo Baroncelli > wrote: >> On 18/09/2018 06.21, Chris Murphy wrote: >>> b. The bootloader code, would have to have sophisticated enough Btrfs >>> knowledge to know if the grubenv has been reflinked or snapshot, >>>

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Austin S. Hemmelgarn
On 2018-09-18 15:00, Chris Murphy wrote: On Tue, Sep 18, 2018 at 12:25 PM, Austin S. Hemmelgarn wrote: It actually is independent of /boot already. I've got it running just fine on my laptop off of the EFI system partition (which is independent of my /boot partition), and thus have no issues

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Austin S. Hemmelgarn
On 2018-09-18 14:57, Chris Murphy wrote: On Tue, Sep 18, 2018 at 12:16 PM, Andrei Borzenkov wrote: 18.09.2018 08:37, Chris Murphy пишет: The patches aren't upstream yet? Will they be? I do not know. Personally I think much easier is to make grub location independent of /boot, allowing

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Austin S. Hemmelgarn
On 2018-09-18 14:38, Andrei Borzenkov wrote: 18.09.2018 21:25, Austin S. Hemmelgarn пишет: On 2018-09-18 14:16, Andrei Borzenkov wrote: 18.09.2018 08:37, Chris Murphy пишет: On Mon, Sep 17, 2018 at 11:24 PM, Andrei Borzenkov wrote: 18.09.2018 07:21, Chris Murphy пишет: On Mon, Sep 17, 2018

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Tue, Sep 18, 2018 at 12:25 PM, Austin S. Hemmelgarn wrote: > It actually is independent of /boot already. I've got it running just fine > on my laptop off of the EFI system partition (which is independent of my > /boot partition), and thus have no issues with handling of the grubenv file. >

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Andrei Borzenkov
18.09.2018 21:57, Chris Murphy пишет: > On Tue, Sep 18, 2018 at 12:16 PM, Andrei Borzenkov > wrote: >> 18.09.2018 08:37, Chris Murphy пишет: > >>> The patches aren't upstream yet? Will they be? >>> >> >> I do not know. Personally I think much easier is to make grub location >> independent of

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Tue, Sep 18, 2018 at 12:16 PM, Andrei Borzenkov wrote: > 18.09.2018 08:37, Chris Murphy пишет: >> The patches aren't upstream yet? Will they be? >> > > I do not know. Personally I think much easier is to make grub location > independent of /boot, allowing grub be installed in separate

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Chris Murphy
On Tue, Sep 18, 2018 at 11:15 AM, Goffredo Baroncelli wrote: > On 18/09/2018 06.21, Chris Murphy wrote: >> b. The bootloader code, would have to have sophisticated enough Btrfs >> knowledge to know if the grubenv has been reflinked or snapshot, >> because even if +C, it may not be valid to

Re: btrfs receive incremental stream on another uuid

2018-09-18 Thread Hugo Mills
On Tue, Sep 18, 2018 at 06:28:37PM +, Gervais, Francois wrote: > > No. It is already possible (by setting received UUID); it should not be > made too open to easy abuse. > > > Do you mean edit the UUID in the byte stream before btrfs receive? No, there's an ioctl to change the received

Re: btrfs receive incremental stream on another uuid

2018-09-18 Thread Andrei Borzenkov
18.09.2018 21:28, Gervais, Francois пишет: >> No. It is already possible (by setting received UUID); it should not be > made too open to easy abuse. > > > Do you mean edit the UUID in the byte stream before btrfs receive? > No, I mean setting received UUID on subvolume. Unfortunately, it is

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Andrei Borzenkov
18.09.2018 21:25, Austin S. Hemmelgarn пишет: > On 2018-09-18 14:16, Andrei Borzenkov wrote: >> 18.09.2018 08:37, Chris Murphy пишет: >>> On Mon, Sep 17, 2018 at 11:24 PM, Andrei Borzenkov >>> wrote: 18.09.2018 07:21, Chris Murphy пишет: > On Mon, Sep 17, 2018 at 9:44 PM, Chris Murphy

Re: Move data and mount point to subvolume

2018-09-18 Thread Hans van Kranenburg
On 09/18/2018 08:10 PM, Marc Joliet wrote: > Am Sonntag, 16. September 2018, 14:50:04 CEST schrieb Hans van Kranenburg: >> The last example, where you make a subvolume and move everything into >> it, will not do what you want. Since a subvolume is a separate new >> directoty/file hierarchy, mv

Re: btrfs receive incremental stream on another uuid

2018-09-18 Thread Gervais, Francois
> No. It is already possible (by setting received UUID); it should not be made too open to easy abuse. Do you mean edit the UUID in the byte stream before btrfs receive?

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Austin S. Hemmelgarn
On 2018-09-18 14:16, Andrei Borzenkov wrote: 18.09.2018 08:37, Chris Murphy пишет: On Mon, Sep 17, 2018 at 11:24 PM, Andrei Borzenkov wrote: 18.09.2018 07:21, Chris Murphy пишет: On Mon, Sep 17, 2018 at 9:44 PM, Chris Murphy wrote:

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Andrei Borzenkov
18.09.2018 08:37, Chris Murphy пишет: > On Mon, Sep 17, 2018 at 11:24 PM, Andrei Borzenkov > wrote: >> 18.09.2018 07:21, Chris Murphy пишет: >>> On Mon, Sep 17, 2018 at 9:44 PM, Chris Murphy >>> wrote: https://btrfs.wiki.kernel.org/index.php/FAQ#Does_grub_support_btrfs.3F Does

Re: btrfs receive incremental stream on another uuid

2018-09-18 Thread Andrei Borzenkov
18.09.2018 20:56, Gervais, Francois пишет: > > Hi, > > I'm trying to apply a btrfs send diff (done through -p) to another subvolume > with the same content as the proper parent but with a different uuid. > > I looked through btrfs receive and I get the feeling that this is not > possible

Re: Move data and mount point to subvolume

2018-09-18 Thread Marc Joliet
Am Sonntag, 16. September 2018, 14:50:04 CEST schrieb Hans van Kranenburg: > The last example, where you make a subvolume and move everything into > it, will not do what you want. Since a subvolume is a separate new > directoty/file hierarchy, mv will turn into a cp and rm operation > (without

btrfs receive incremental stream on another uuid

2018-09-18 Thread Gervais, Francois
Hi, I'm trying to apply a btrfs send diff (done through -p) to another subvolume with the same content as the proper parent but with a different uuid. I looked through btrfs receive and I get the feeling that this is not possible right now. I'm thinking of adding a -p option to btrfs

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-18 Thread Goffredo Baroncelli
On 18/09/2018 06.21, Chris Murphy wrote: > b. The bootloader code, would have to have sophisticated enough Btrfs > knowledge to know if the grubenv has been reflinked or snapshot, > because even if +C, it may not be valid to overwrite, and COW must > still happen, and there's no way the code in

[PATCH V5 RESEND] Btrfs: enchanse raid1/10 balance heuristic

2018-09-18 Thread Timofey Titovets
From: Timofey Titovets Currently btrfs raid1/10 balancer bаlance requests to mirrors, based on pid % num of mirrors. Make logic understood: - if one of underline devices are non rotational - Queue leght to underline devices By default try use pid % num_mirrors guessing, but: - If one of

[PATCH RESEND] Btrfs: make should_defrag_range() understood compressed extents

2018-09-18 Thread Timofey Titovets
From: Timofey Titovets Both, defrag ioctl and autodefrag - call btrfs_defrag_file() for file defragmentation. Kernel default target extent size - 256KiB. Btrfs progs default - 32MiB. Both bigger then maximum size of compressed extent - 128KiB. That lead to rewrite all compressed data on

Re: btrfs panic problem

2018-09-18 Thread sunny.s.zhang
Add Junxiao 在 2018年09月18日 13:05, Duncan 写道: sunny.s.zhang posted on Tue, 18 Sep 2018 08:28:14 +0800 as excerpted: My OS(4.1.12) panic in kmem_cache_alloc, which is called by btrfs_get_or_create_delayed_node. I found that the freelist of the slub is wrong. [Not a dev, just a btrfs list

[PATCH] btrfs-progs: tests: Add the testcase for subvolume name length limit test

2018-09-18 Thread Su Yanjun
Total of three conditions are tested. One for short name, one with name length 255, the last one with more than 255. This case should pass after commit 'btrfs-progs: change filename limit to 255 when creating subvolume'. Signed-off-by: Su Yanjun --- .../033-filename-length-limit/test.sh

Re: [PATCH v3 5/7] btrfs-progs: lowmem: do missing check of last item after check_inode_item()

2018-09-18 Thread Qu Wenruo
On 2018/9/18 下午4:01, Su Yue wrote: > > > On 9/18/18 1:32 PM, Qu Wenruo wrote: >> >> >> On 2018/9/17 下午9:24, Su Yue wrote: >>> >>> >>> On 2018/9/17 8:53 PM, Qu Wenruo wrote: On 2018/9/17 下午3:28, Su Yue wrote: > After call of check_inode_item(), path may point to the last

Re: [PATCH v3 5/7] btrfs-progs: lowmem: do missing check of last item after check_inode_item()

2018-09-18 Thread Su Yue
On 9/18/18 1:32 PM, Qu Wenruo wrote: On 2018/9/17 下午9:24, Su Yue wrote: On 2018/9/17 8:53 PM, Qu Wenruo wrote: On 2018/9/17 下午3:28, Su Yue wrote: After call of check_inode_item(), path may point to the last unchecked slot of the leaf. The outer walk_up_tree() always treats the