Punch hole on full fs

2018-09-19 Thread anand . jain
Test script [1] tries to punch hole on a full FS and it works fine as long as the hole size and the offset is aligned with the sectorsize and the extent, so that it could just drop the relevant extent to create the hole. The reason why this test fails for non aligned hole size and offset

[PATCH v8 5/6] Btrfs: rename get_chunk_map() and make it non-static

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval The Btrfs swap code is going to need it, so give it a btrfs_ prefix and make it non-static. Reviewed-by: Nikolay Borisov Signed-off-by: Omar Sandoval --- fs/btrfs/volumes.c | 29 ++--- fs/btrfs/volumes.h | 2 ++ 2 files changed, 20 insertions(+),

[PATCH v8 0/6] Btrfs: implement swap file support

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval Hi, This series implements swap file support for Btrfs. Changes from v7 [1]: - Expanded a few commit messages - Added Johannes' acked-by on patches 1 and 2 - Rebased on v4.19-rc4 No functional changes. Thanks! 1: https://www.spinics.net/lists/linux-btrfs/msg81933.html

[PATCH v8 4/6] Btrfs: prevent ioctls from interfering with a swap file

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval A later patch will implement swap file support for Btrfs, but before we do that, we need to make sure that the various Btrfs ioctls cannot change a swap file. When a swap file is active, we must make sure that the extents of the file are not moved and that they don't become

[PATCH v8 6/6] Btrfs: support swap files

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval Btrfs has not allowed swap files since commit 35054394c4b3 ("Btrfs: stop providing a bmap operation to avoid swapfile corruptions"). However, now that the proper restrictions are in place, Btrfs can support swap files through the swap file a_ops, similar to iomap in commit

[PATCH v8 1/6] mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval The SWP_FILE flag serves two purposes: to make swap_{read,write}page() go through the filesystem, and to make swapoff() call ->swap_deactivate(). For Btrfs, we want the latter but not the former, so split this flag into two. This makes us always call ->swap_deactivate() if

[PATCH v8 2/6] mm: export add_swap_extent()

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval Btrfs currently does not support swap files because swap's use of bmap does not work with copy-on-write and multiple devices. See commit 35054394c4b3 ("Btrfs: stop providing a bmap operation to avoid swapfile corruptions"). However, the swap code has a mechanism for the

[PATCH v8 3/6] vfs: update swap_{,de}activate documentation

2018-09-19 Thread Omar Sandoval
From: Omar Sandoval The documentation for these functions is wrong in several ways: - swap_activate() is called with the inode locked - swap_activate() takes a swap_info_struct * and a sector_t * - swap_activate() can also return a positive number of extents it added itself -

Re: inline extents

2018-09-19 Thread Chris Murphy
Adding fsdevel@, linux-ext4, and btrfs@ (which has a separate subject on this same issue) On Wed, Sep 19, 2018 at 7:45 PM, Dave Chinner wrote: >On Wed, Sep 19, 2018 at 10:23:38AM -0600, Chris Murphy wrote: >> Fedora 29 has a new feature to test if boot+startup fails, so the >> bootloader can

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Qu Wenruo
On 2018/9/20 上午4:11, Hans van Kranenburg wrote: > On 09/19/2018 10:04 PM, Martin Steigerwald wrote: >> Hans van Kranenburg - 19.09.18, 19:58: >>> However, as soon as we remount the filesystem with space_cache=v2 - >>> writes drop to just around 3-10 MB/s to each disk. If we remount to

Re: cannot mount btrfs as root

2018-09-19 Thread anand . jain
On 09/19/2018 09:19 PM, Zbigniew 'zibi' Jarosik wrote: Hi! I can't mount my RAID1 set as rootfs, but as normal mount to running system works normally. It was working normal, than some day it died. When booted to rootfs as root=UUID=98c94774-d93a-400e-a275-58cc3ac2a58a rootflags=subvol=@root

Re: state of btrfs snapshot limitations?

2018-09-19 Thread James A. Robinson
On Wed, Sep 19, 2018 at 4:04 PM Pete wrote: > snapshots. You need to delete it out of them as well which defeats the > idea of read only snapshots if you are using them. I wouldn't say it defeats the idea of read-only snapshots. If you want to be able to "go back in time" and see what changed,

Re: state of btrfs snapshot limitations?

2018-09-19 Thread Pete
On 09/19/2018 03:41 PM, Piotr Pawłow wrote: > Hello, >> If the limit is 100 or less I'd need use a more complicated >> rotation scheme. > > If you just want to thin them out over time without having selected "special" > monthly, yearly etc snapshots, then my favorite scheme is to just compare

Re: [RFC PATCH v2 0/4] btrfs-progs: build distinct binaries for specific btrfs subcommands

2018-09-19 Thread Axel Burri
In Reply to: On 30/08/2018 04.38, Misono Tomohiro wrote: > > Hello, > > Not directly related this series and just FYI, > I'm working to allow sub show/list to non-privileged user as long > as he can access to the subvolume: > https://www.spinics.net/lists/linux-btrfs/msg79285.html > > Hopefully

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Nikolay Borisov
On 19.09.2018 23:11, Hans van Kranenburg wrote: > On 09/19/2018 10:04 PM, Martin Steigerwald wrote: >> Hans van Kranenburg - 19.09.18, 19:58: >>> However, as soon as we remount the filesystem with space_cache=v2 - >>> writes drop to just around 3-10 MB/s to each disk. If we remount to

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Hans van Kranenburg
On 09/19/2018 10:04 PM, Martin Steigerwald wrote: > Hans van Kranenburg - 19.09.18, 19:58: >> However, as soon as we remount the filesystem with space_cache=v2 - >> >>> writes drop to just around 3-10 MB/s to each disk. If we remount to >>> space_cache - lots of writes, system unresponsive. Again

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Martin Steigerwald
Hans van Kranenburg - 19.09.18, 19:58: > However, as soon as we remount the filesystem with space_cache=v2 - > > > writes drop to just around 3-10 MB/s to each disk. If we remount to > > space_cache - lots of writes, system unresponsive. Again remount to > > space_cache=v2 - low writes, system

Re: btrfs send hangs after partial transfer and blocks all IO

2018-09-19 Thread Jürgen Herrmann
Am 13.9.2018 14:35, schrieb Nikolay Borisov: On 13.09.2018 15:30, Jürgen Herrmann wrote: OK, I will install kdump later and perform a dump after the hang. One more noob question beforehand: does this dump contain sensitive information, for example the luks encryption key for the disk etc? A

Re: btrfs send hangs after partial transfer and blocks all IO

2018-09-19 Thread Jürgen Herrmann
Am 13.9.2018 18:22, schrieb Chris Murphy: (resend to all) On Thu, Sep 13, 2018 at 9:44 AM, Nikolay Borisov wrote: On 13.09.2018 18:30, Chris Murphy wrote: This is the 2nd or 3rd thread containing hanging btrfs send, with kernel 4.18.x. The subject of one is "btrfs send hung in pipe_wait"

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-19 Thread Austin S. Hemmelgarn
On 2018-09-19 15:08, Goffredo Baroncelli wrote: On 18/09/2018 19.15, Goffredo Baroncelli wrote: b. The bootloader code, would have to have sophisticated enough Btrfs knowledge to know if the grubenv has been reflinked or snapshot, because even if +C, it may not be valid to overwrite, and COW

Re: GRUB writing to grubenv outside of kernel fs code

2018-09-19 Thread Goffredo Baroncelli
On 18/09/2018 19.15, Goffredo Baroncelli wrote: >> b. The bootloader code, would have to have sophisticated enough Btrfs >> knowledge to know if the grubenv has been reflinked or snapshot, >> because even if +C, it may not be valid to overwrite, and COW must >> still happen, and there's no way the

Re: [PATCH v7 2/6] mm: export add_swap_extent()

2018-09-19 Thread Johannes Weiner
On Wed, Sep 19, 2018 at 11:28:00AM -0700, Omar Sandoval wrote: > On Wed, Sep 19, 2018 at 02:09:09PM -0400, Johannes Weiner wrote: > > On Tue, Sep 11, 2018 at 03:34:45PM -0700, Omar Sandoval wrote: > > > From: Omar Sandoval > > > > > > Btrfs will need this for swap file support. > > > > > >

Re: [PATCH v7 1/6] mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS

2018-09-19 Thread Johannes Weiner
On Wed, Sep 19, 2018 at 11:12:02AM -0700, Omar Sandoval wrote: > On Wed, Sep 19, 2018 at 02:02:32PM -0400, Johannes Weiner wrote: > > On Tue, Sep 11, 2018 at 03:34:44PM -0700, Omar Sandoval wrote: > > > @@ -2411,8 +2412,10 @@ static int setup_swap_extents(struct > > > swap_info_struct *sis,

[PATCH 8/9] btrfs: Make more generic the code for RAID 6 rebuilding

2018-09-19 Thread Goffredo Baroncelli
From: Goffredo Baroncelli The original code which handles the recovery of a RAID 6 disks array assumes that all reads are multiple of 1 << GRUB_DISK_SECTOR_BITS and it assumes that all the I/O is done via the struct grub_diskfilter_segment. This is not true for the btrfs code. In order to reuse

Re: btrfs panic problem

2018-09-19 Thread Liu Bo
On Mon, Sep 17, 2018 at 5:28 PM, sunny.s.zhang wrote: > Hi All, > > My OS(4.1.12) panic in kmem_cache_alloc, which is called by > btrfs_get_or_create_delayed_node. > > I found that the freelist of the slub is wrong. > > crash> struct kmem_cache_cpu 887e7d7a24b0 > > struct kmem_cache_cpu { >

Re: [PATCH v7 2/6] mm: export add_swap_extent()

2018-09-19 Thread Omar Sandoval
On Wed, Sep 19, 2018 at 02:09:09PM -0400, Johannes Weiner wrote: > On Tue, Sep 11, 2018 at 03:34:45PM -0700, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Btrfs will need this for swap file support. > > > > Signed-off-by: Omar Sandoval > > That looks reasonable. After reading the last

Re: [PATCH v7 1/6] mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS

2018-09-19 Thread Omar Sandoval
On Wed, Sep 19, 2018 at 02:02:32PM -0400, Johannes Weiner wrote: > On Tue, Sep 11, 2018 at 03:34:44PM -0700, Omar Sandoval wrote: > > @@ -2411,8 +2412,10 @@ static int setup_swap_extents(struct > > swap_info_struct *sis, sector_t *span) > > > > if (mapping->a_ops->swap_activate) { > >

Re: [PATCH v7 2/6] mm: export add_swap_extent()

2018-09-19 Thread Johannes Weiner
On Tue, Sep 11, 2018 at 03:34:45PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Btrfs will need this for swap file support. > > Signed-off-by: Omar Sandoval That looks reasonable. After reading the last patch, it's somewhat understandable why you cannot simply implemnet ->bmap and

Re: [PATCH v7 1/6] mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS

2018-09-19 Thread Johannes Weiner
On Tue, Sep 11, 2018 at 03:34:44PM -0700, Omar Sandoval wrote: > @@ -2411,8 +2412,10 @@ static int setup_swap_extents(struct swap_info_struct > *sis, sector_t *span) > > if (mapping->a_ops->swap_activate) { > ret = mapping->a_ops->swap_activate(sis, swap_file, span); > +

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Hans van Kranenburg
Hi, On 09/19/2018 10:43 AM, Tomasz Chmielewski wrote: > I have a mysql slave which writes to a RAID-1 btrfs filesystem (with > 4.17.14 kernel) on 3 x ~1.9 TB SSD disks; filesystem is around 40% full. > > The slave receives around 0.5-1 MB/s of data from the master over the > network, which is

Re: [PATCH v7 0/6] Btrfs: implement swap file support

2018-09-19 Thread Omar Sandoval
On Tue, Sep 11, 2018 at 03:34:43PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Hi, > > This series implements swap file support for Btrfs. > > Changes from v6 [1]: > > - Moved btrfs_get_chunk_map() comment to function body > - Added more comments about pinned block group/device

Re: btrfs panic problem

2018-09-19 Thread Nikolay Borisov
On 19.09.2018 02:53, sunny.s.zhang wrote: > Hi Duncan, > > Thank you for your advice. I understand what you mean.  But i have > reviewed the latest btrfs code, and i think the issue is exist still. > > At 71 line, if the function of btrfs_get_delayed_node run over this > line, then switch to

state of btrfs snapshot limitations?

2018-09-19 Thread Piotr Pawłow
Hello, > If the limit is 100 or less I'd need use a more complicated > rotation scheme. If you just want to thin them out over time without having selected "special" monthly, yearly etc snapshots, then my favorite scheme is to just compare the age of a snapshot to the distance to its

Re: cannot mount btrfs as root

2018-09-19 Thread Qu Wenruo
On 2018/9/19 下午9:19, Zbigniew 'zibi' Jarosik wrote: > Hi! > > I can't mount my RAID1 set as rootfs, but as normal mount to running > system works normally. It was working normal, than some day it died. > > When booted to rootfs as > root=UUID=98c94774-d93a-400e-a275-58cc3ac2a58a

cannot mount btrfs as root

2018-09-19 Thread Zbigniew 'zibi' Jarosik
Hi! I can't mount my RAID1 set as rootfs, but as normal mount to running system works normally. It was working normal, than some day it died. When booted to rootfs as root=UUID=98c94774-d93a-400e-a275-58cc3ac2a58a rootflags=subvol=@root i got: BTRFS error (device bcache2): devid 5 uuid

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Remi Gauvin
On 2018-09-19 04:43 AM, Tomasz Chmielewski wrote: > I have a mysql slave which writes to a RAID-1 btrfs filesystem (with > 4.17.14 kernel) on 3 x ~1.9 TB SSD disks; filesystem is around 40% full. > > The slave receives around 0.5-1 MB/s of data from the master over the > network, which is then

Re: [PATCH RFC] btrfs: delayed-inode: Use spinlock to protect btrfs_inode::delayed_node

2018-09-19 Thread Nikolay Borisov
On 19.09.2018 09:59, Qu Wenruo wrote: > In the following case, we could trigger a use-after-free bug: > > CPU0| CPU1 > - > btrfs_remove_delayed_node|

Re: very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Qu Wenruo
On 2018/9/19 下午4:43, Tomasz Chmielewski wrote: > I have a mysql slave which writes to a RAID-1 btrfs filesystem (with > 4.17.14 kernel) on 3 x ~1.9 TB SSD disks; filesystem is around 40% full. This sounds a little concerning. Not about the the usage percentage itself. but the size and how many

very poor performance / a lot of writes to disk with space_cache (but not with space_cache=v2)

2018-09-19 Thread Tomasz Chmielewski
I have a mysql slave which writes to a RAID-1 btrfs filesystem (with 4.17.14 kernel) on 3 x ~1.9 TB SSD disks; filesystem is around 40% full. The slave receives around 0.5-1 MB/s of data from the master over the network, which is then saved to MySQL's relay log and executed. In ideal

Re: [PATCH RFC] btrfs: delayed-inode: Use spinlock to protect btrfs_inode::delayed_node

2018-09-19 Thread Qu Wenruo
On 2018/9/19 下午2:59, Qu Wenruo wrote: > In the following case, we could trigger a use-after-free bug: > > CPU0| CPU1 > - > btrfs_remove_delayed_node| btrfs_get_delayed_node

[PATCH RFC] btrfs: delayed-inode: Use spinlock to protect btrfs_inode::delayed_node

2018-09-19 Thread Qu Wenruo
In the following case, we could trigger a use-after-free bug: CPU0| CPU1 - btrfs_remove_delayed_node| btrfs_get_delayed_node |- delayed_node =| |- node =