Re: [Ocfs2-devel] [PATCH 2/7] vfs: flush and wait for io when setting the immutable flag via SETFLAGS

2019-06-25 Thread Jan Kara
On Mon 24-06-19 20:04:39, Darrick J. Wong wrote: > On Mon, Jun 24, 2019 at 02:58:17PM -0700, Darrick J. Wong wrote: > > On Mon, Jun 24, 2019 at 01:37:37PM +0200, Jan Kara wrote: > > > On Fri 21-06-19 16:57:07, Darrick J. Wong wrote: > > > > From: Darrick J. Wong > > > > > > > > When we're using F

Re: [PATCH 1/2] btrfs: inode: Don't compress if NODATASUM or NODATACOW set

2019-06-25 Thread Qu Wenruo
Ping? This patch should fix the problem of compressed extent even when nodatasum is set. It has been one year but we still didn't get a conclusion on where force_compress should behave. But at least to me, NODATASUM is a strong exclusion for compress, no matter whatever option we use, we should

Re: [PATCH v4 0/7] vfs: make immutable files actually immutable

2019-06-25 Thread Christoph Hellwig
On Fri, Jun 21, 2019 at 04:56:50PM -0700, Darrick J. Wong wrote: > Hi all, > > The chattr(1) manpage has this to say about the immutable bit that > system administrators can set on files: > > "A file with the 'i' attribute cannot be modified: it cannot be deleted > or renamed, no link can be crea

CoW overhead from old extents?

2019-06-25 Thread Roman Mamedov
Hello, I have a number of VM images in sparse NOCOW files, with: # du -B M -sc * ... 46030Mtotal and: # du -B M -sc --apparent-size * ... 96257Mtotal But despite there being nothing else on the filesystem and no snapshots, # df -B M . ... 1M-blocks Used Avai

Re: [PATCH 04/11] btrfs: move the space_info handling code to space-info.c

2019-06-25 Thread David Sterba
On Tue, Jun 18, 2019 at 04:09:19PM -0400, Josef Bacik wrote: > --- /dev/null > +++ b/fs/btrfs/space-info.c > @@ -0,0 +1,177 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +/* > + * Copyright (C) 2019 Facebook. All rights reserved. > + */ How does the copyright claim work here? You're just moving

Re: [PATCH 04/11] btrfs: move the space_info handling code to space-info.c

2019-06-25 Thread Josef Bacik
On Tue, Jun 25, 2019 at 01:58:43PM +0200, David Sterba wrote: > On Tue, Jun 18, 2019 at 04:09:19PM -0400, Josef Bacik wrote: > > --- /dev/null > > +++ b/fs/btrfs/space-info.c > > @@ -0,0 +1,177 @@ > > +/* SPDX-License-Identifier: GPL-2.0 */ > > +/* > > + * Copyright (C) 2019 Facebook. All rights r

Re: CoW overhead from old extents?

2019-06-25 Thread Qu Wenruo
On 2019/6/25 下午6:41, Roman Mamedov wrote: > Hello, > > I have a number of VM images in sparse NOCOW files, with: NODATACOW and no snapshot? Then unless some thing like balance or defrag, it should mostly behave much like regular fs. > > # du -B M -sc * > ... > 46030M total > > an

Re: 5.2rc5 corruption, many "is compressed but inode flag doesn't allow"

2019-06-25 Thread Chris Murphy
On Tue, Jun 25, 2019 at 12:58 AM Nikolay Borisov wrote: > > > > On 25.06.19 г. 5:49 ч., Chris Murphy wrote: > > False alarm, not a new issue at all! > > > > I have a different system on kernel 5.1.11 using Btrfs as root with > > persistent systemd-journald storage, and compress=zstd. And I never >

RFC: btrfs-progs json formatting

2019-06-25 Thread David Sterba
Hi, I'd like to get some feedback on the json output, the overall structure of the information and naming. Code: git://github.com/kdave/btrfs-progs.git preview-json The one example command that supports it is $ ./btrfs --format json subvolume show /mnt The test tests/cli-tests/011-* can demon

Re: 5.2rc5 corruption, many "is compressed but inode flag doesn't allow"

2019-06-25 Thread Chris Murphy
On Tue, Jun 25, 2019 at 12:58 AM Nikolay Borisov wrote: > > But why are your nocompress files being compressed? $ sudo strace -p 624 -k -t -o sysdjourndstrace.txt https://drive.google.com/open?id=1IspAjQ6b9dVizjqrX4E6ZErzUKH0CQBl Maybe there's a better way to see what's going on. I do see chattr

Re: [PATCH 04/11] btrfs: move the space_info handling code to space-info.c

2019-06-25 Thread David Sterba
On Tue, Jun 25, 2019 at 08:54:30AM -0400, Josef Bacik wrote: > On Tue, Jun 25, 2019 at 01:58:43PM +0200, David Sterba wrote: > > On Tue, Jun 18, 2019 at 04:09:19PM -0400, Josef Bacik wrote: > > > --- /dev/null > > > +++ b/fs/btrfs/space-info.c > > > @@ -0,0 +1,177 @@ > > > +/* SPDX-License-Identifi

Re: [PATCH 00/11] btrfs: move the space_info code out of extent-tree.c

2019-06-25 Thread David Sterba
On Tue, Jun 18, 2019 at 04:09:15PM -0400, Josef Bacik wrote: > This is the first pass at making extent-tree.c much smaller. I've > purposefully > done no other cleanups or changes. The places where I needed to modify > callers > were done in separate patches. The only time I moved and changed

Re: 5.2rc5 corruption, many "is compressed but inode flag doesn't allow"

2019-06-25 Thread Chris Murphy
The order I see is: rename() currentsyslog archivesyslog openat() currentsyslog fsync() currentsyslog fsync() dir of currentsyslog fallocate() currentsyslog repeats for the user log Where I get lost is at 10:04:18 ioctl(103, FS_IOC_SETFLAGS, 0x7ffc6429954c) = 0 > /usr/lib64/libc-2.29.9000.so(

Re: [PATCH v2 0/6] RAID1 with 3- and 4- copies

2019-06-25 Thread David Sterba
On Mon, Jun 10, 2019 at 02:29:40PM +0200, David Sterba wrote: > Hi, > > this patchset brings the RAID1 with 3 and 4 copies as a separate > feature as outlined in V1 > (https://lore.kernel.org/linux-btrfs/cover.1531503452.git.dste...@suse.com/). > > This should help a bit in the raid56 situation,

Re: CoW overhead from old extents?

2019-06-25 Thread Austin S. Hemmelgarn
On 2019-06-25 06:41, Roman Mamedov wrote: Hello, I have a number of VM images in sparse NOCOW files, with: # du -B M -sc * ... 46030M total and: # du -B M -sc --apparent-size * ... 96257M total But despite there being nothing else on the filesystem and no snapsh

Re: [PATCH v4 0/7] vfs: make immutable files actually immutable

2019-06-25 Thread Darrick J. Wong
On Tue, Jun 25, 2019 at 03:36:31AM -0700, Christoph Hellwig wrote: > On Fri, Jun 21, 2019 at 04:56:50PM -0700, Darrick J. Wong wrote: > > Hi all, > > > > The chattr(1) manpage has this to say about the immutable bit that > > system administrators can set on files: > > > > "A file with the 'i' att

Re: RFC: btrfs-progs json formatting

2019-06-25 Thread Hans van Kranenburg
Hi, On 6/25/19 6:09 PM, David Sterba wrote: > Hi, > > I'd like to get some feedback on the json output, the overall structure > of the information and naming. > > Code: git://github.com/kdave/btrfs-progs.git preview-json > > The one example command that supports it is > > $ ./btrfs --format j

Re: [PATCH] btrfs: Simplify update space_info in __reserve_metadata_bytes()

2019-06-25 Thread David Sterba
On Wed, Jun 19, 2019 at 09:12:49AM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > Simplification. > We don't need an if-else-if structure where we can use a > simple OR since both conditions are performing the > same action. The short-circuit for OR will ensure that if > the first

Re: [PATCH 4/8] btrfs: cleanup the target logic in __btrfs_block_rsv_release

2019-06-25 Thread David Sterba
On Thu, Jun 20, 2019 at 11:23:36AM +0300, Nikolay Borisov wrote: > > > On 19.06.19 г. 20:47 ч., Josef Bacik wrote: > > This works for all callers already, but if we wanted to use the helper > > for the global_block_rsv it would end up trying to refill itself. Fix > > the logic to be able to be u

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Goldwyn Rodrigues
On 9:05 24/06, Christoph Hellwig wrote: > On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > > From: Goldwyn Rodrigues > > > > btrfs uses page->private as well to store extent_buffer. Make > > the check stricter to make sure we are using page->private for iop by > > comparing i

Re: [PATCH 1/6] iomap: Use a IOMAP_COW/srcmap for a read-modify-write I/O

2019-06-25 Thread Goldwyn Rodrigues
On 9:07 24/06, Christoph Hellwig wrote: > xfs will need to be updated to fill in the additional iomap for the > COW case. Has this series been tested on xfs? > No, I have not tested this, or make xfs set IOMAP_COW. I will try to do it in the next iteration. > I can't say I'm a huge fan of this

Re: [PATCH 1/6] iomap: Use a IOMAP_COW/srcmap for a read-modify-write I/O

2019-06-25 Thread Goldwyn Rodrigues
On 17:46 21/06, Darrick J. Wong wrote: > On Fri, Jun 21, 2019 at 02:28:23PM -0500, Goldwyn Rodrigues wrote: > > From: Goldwyn Rodrigues > > > > Introduces a new type IOMAP_COW, which means the data at offset > > must be read from a srcmap and copied before performing the > > write on the offset.

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Goldwyn Rodrigues
On 17:21 21/06, Darrick J. Wong wrote: > On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > > From: Goldwyn Rodrigues > > > > btrfs uses page->private as well to store extent_buffer. Make > > the check stricter to make sure we are using page->private for iop by > > comparing ibl

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Filipe Manana
On Tue, Jun 25, 2019 at 8:58 PM Goldwyn Rodrigues wrote: > > On 9:05 24/06, Christoph Hellwig wrote: > > On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > > > From: Goldwyn Rodrigues > > > > > > btrfs uses page->private as well to store extent_buffer. Make > > > the check stri

Re: [PATCH] btrfs-progs: scrub: Fix scrub cancel/resume not to skip most of the disk

2019-06-25 Thread Graham Cobb
On 18/06/2019 09:08, Graham R. Cobb wrote: > When a scrub completes or is cancelled, statistics are updated for reporting > in a later btrfs scrub status command and for resuming the scrub. Most > statistics (such as bytes scrubbed) are additive so scrub adds the statistics > from the current run t

Re: [PATCH v4 0/7] vfs: make immutable files actually immutable

2019-06-25 Thread Andreas Dilger
On Jun 25, 2019, at 12:03 PM, Darrick J. Wong wrote: > > On Tue, Jun 25, 2019 at 03:36:31AM -0700, Christoph Hellwig wrote: >> On Fri, Jun 21, 2019 at 04:56:50PM -0700, Darrick J. Wong wrote: >>> Hi all, >>> >>> The chattr(1) manpage has this to say about the immutable bit that >>> system admini

Re: [PATCH 1/6] iomap: Use a IOMAP_COW/srcmap for a read-modify-write I/O

2019-06-25 Thread Shiyang Ruan
On 6/26/19 3:14 AM, Goldwyn Rodrigues wrote: On 9:07 24/06, Christoph Hellwig wrote: xfs will need to be updated to fill in the additional iomap for the COW case. Has this series been tested on xfs? No, I have not tested this, or make xfs set IOMAP_COW. I will try to do it in the next it

Re: btrfs vs write caching firmware bugs (was: Re: BTRFS recovery not possible)

2019-06-25 Thread Zygo Blaxell
On Mon, Jun 24, 2019 at 11:31:35AM -0600, Chris Murphy wrote: > On Sun, Jun 23, 2019 at 7:52 PM Qu Wenruo wrote: > > > > > > > > On 2019/6/24 上午4:45, Zygo Blaxell wrote: > > > I first observed these correlations back in 2016. We had a lot of WD > > > Green and Black drives in service at the time-

[PATCH 1/5] mm/fs: don't allow writes to immutable files

2019-06-25 Thread Darrick J. Wong
From: Darrick J. Wong The chattr manpage has this to say about immutable files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file can not be opened in write mode."

[PATCH v5 0/5] vfs: make immutable files actually immutable

2019-06-25 Thread Darrick J. Wong
Hi all, The chattr(1) manpage has this to say about the immutable bit that system administrators can set on files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file

[PATCH 4/5] vfs: don't allow most setxattr to immutable files

2019-06-25 Thread Darrick J. Wong
From: Darrick J. Wong The chattr manpage has this to say about immutable files: "A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata can not be modified, and the file can not be opened in write mode."

[PATCH 2/5] vfs: flush and wait for io when setting the immutable flag via SETFLAGS

2019-06-25 Thread Darrick J. Wong
From: Darrick J. Wong When we're using FS_IOC_SETFLAGS to set the immutable flag on a file, we need to ensure that userspace can't continue to write the file after the file becomes immutable. To make that happen, we have to flush all the dirty pagecache pages to disk to ensure that we can fail a

[PATCH 3/5] vfs: flush and wait for io when setting the immutable flag via FSSETXATTR

2019-06-25 Thread Darrick J. Wong
From: Darrick J. Wong When we're using FS_IOC_FSSETXATTR to set the immutable flag on a file, we need to ensure that userspace can't continue to write the file after the file becomes immutable. To make that happen, we have to flush all the dirty pagecache pages to disk to ensure that we can fail

[PATCH 5/5] vfs: don't allow writes to swap files

2019-06-25 Thread Darrick J. Wong
From: Darrick J. Wong Don't let userspace write to an active swap file because the kernel effectively has a long term lease on the storage and things could get seriously corrupted if we let this happen. Signed-off-by: Darrick J. Wong --- fs/attr.c |3 +++ mm/filemap.c |3 +++ mm/m

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Goldwyn Rodrigues
On 21:04 25/06, Filipe Manana wrote: > On Tue, Jun 25, 2019 at 8:58 PM Goldwyn Rodrigues wrote: > > > > On 9:05 24/06, Christoph Hellwig wrote: > > > On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > > > > From: Goldwyn Rodrigues > > > > > > > > btrfs uses page->private as wel

Re: [PATCH 5/5] vfs: don't allow writes to swap files

2019-06-25 Thread Al Viro
On Tue, Jun 25, 2019 at 07:33:31PM -0700, Darrick J. Wong wrote: > --- a/fs/attr.c > +++ b/fs/attr.c > @@ -236,6 +236,9 @@ int notify_change(struct dentry * dentry, struct iattr * > attr, struct inode **de > if (IS_IMMUTABLE(inode)) > return -EPERM; > > + if (IS_SWAPFILE(

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Christoph Hellwig
On Tue, Jun 25, 2019 at 01:56:59PM -0500, Goldwyn Rodrigues wrote: > Btrfs uses page->private to identify which extent_buffer it belongs to. > So, if you read, it fills the page->private. Then you try to write to > it, iomap will assume it to be iomap_page pointer. Yes, and that is going to run in

Re: [PATCH 1/6] iomap: Use a IOMAP_COW/srcmap for a read-modify-write I/O

2019-06-25 Thread Christoph Hellwig
On Fri, Jun 21, 2019 at 05:46:24PM -0700, Darrick J. Wong wrote: > On Fri, Jun 21, 2019 at 02:28:23PM -0500, Goldwyn Rodrigues wrote: > > From: Goldwyn Rodrigues > > > > Introduces a new type IOMAP_COW, which means the data at offset > > must be read from a srcmap and copied before performing the

Re: [PATCH 1/6] iomap: Use a IOMAP_COW/srcmap for a read-modify-write I/O

2019-06-25 Thread Christoph Hellwig
On Tue, Jun 25, 2019 at 02:14:42PM -0500, Goldwyn Rodrigues wrote: > > I can't say I'm a huge fan of this two iomaps in one method call > > approach. I always though two separate iomap iterations would be nicer, > > but compared to that even the older hack with just the additional > > src_addr see

Re: [PATCH 3/6] iomap: Check iblocksize before transforming page->private

2019-06-25 Thread Nikolay Borisov
On 26.06.19 г. 6:03 ч., Goldwyn Rodrigues wrote: > On 21:04 25/06, Filipe Manana wrote: >> On Tue, Jun 25, 2019 at 8:58 PM Goldwyn Rodrigues wrote: >>> >>> On 9:05 24/06, Christoph Hellwig wrote: On Fri, Jun 21, 2019 at 02:28:25PM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrig