[PATCH v2] btrfs: transaction: Commit transaction more frequently for BPF

2019-08-15 Thread Qu Wenruo
Btrfs has btrfs_end_transaction_throttle() which could try to commit transaction when needed. However under most cases btrfs_end_transaction_throttle() won't really commit transaction, due to the hard timing requirement. Now introduce a new error injection point, btrfs_need_trans_pressure(), to a

btrfs corruption: invalid mode: has 00 expect valid S_IF* bit(s)

2019-08-15 Thread Jim Geo
Hello, my /home folder is on a single device btrfs partition. When I ls a directory, I get these messages on ls: ls: cannot access 'file1' : Input/output error ls: cannot access 'file2' : Input/output error ls: cannot access 'file3' : Input/output error and dmesg says: [31209.938486] BTRFS criti

Re: btrfs corruption: invalid mode: has 00 expect valid S_IF* bit(s)

2019-08-15 Thread Qu Wenruo
On 2019/8/15 下午4:39, Jim Geo wrote: > Hello, > > my /home folder is on a single device btrfs partition. > When I ls a directory, I get these messages on ls: > ls: cannot access 'file1' : Input/output error > ls: cannot access 'file2' : Input/output error > ls: cannot access 'file3' : Input/outp

Re: btrfs corruption: invalid mode: has 00 expect valid S_IF* bit(s)

2019-08-15 Thread Qu Wenruo
On 2019/8/15 下午5:41, Jim Geo wrote: > Στις Πέμ, 15 Αυγ 2019 στις 12:12 μ.μ., ο/η Qu Wenruo > έγραψε: >> >> >> >> On 2019/8/15 下午4:39, Jim Geo wrote: >>> Hello, >>> >>> my /home folder is on a single device btrfs partition. >>> When I ls a directory, I get these messages on ls: >>> ls: cannot ac

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Tim Walberg
Had to wait for 'btrfs recover' to finish before I proceed farther. Kernel is 4.19.45, tools are 4.19.1 File system is a 3-disk RAID10 with WD3003FZEX (WD Black 3TB) Output from attempting to mount: # mount -o ro,usebackuproot /dev/sdc1 /mnt mount: wrong fs type, bad option, bad superblock on /

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Tim Walberg
Also - here's 'btrfs inspect-internal dump-super /dev/sdc1': superblock: bytenr=65536, device=/dev/sdc1 - csum_type 0 (crc32c) csum_size 4 csum 0x4331039b [match] bytenr 65536 flags 0x1 ( WRITTEN ) magic _BHRfS_M [

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Qu Wenruo
On 2019/8/15 下午9:52, Tim Walberg wrote: > Had to wait for 'btrfs recover' to finish before I proceed farther. > > Kernel is 4.19.45, tools are 4.19.1 > > File system is a 3-disk RAID10 with WD3003FZEX (WD Black 3TB) > > Output from attempting to mount: > > # mount -o ro,usebackuproot /dev/sdc

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Tim Walberg
'dump-super -Ffa' from all three devices attached. 'btrfs restore' did appear to recover most of the main data, minus snapshots, which would have greatly increased the required time and capacity, since I was recovering to XFS. 'btrfs rescue chunk-recover' ran, but failed to fix anything. 'btrfs r

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Qu Wenruo
On 2019/8/15 下午10:21, Tim Walberg wrote: > 'dump-super -Ffa' from all three devices attached. > > 'btrfs restore' did appear to recover most of the main data, minus > snapshots, which would have greatly increased the required time and > capacity, since I was recovering to XFS. That's why I reco

Re: [PATCH 1/1] btrfs: Simplify parsing max_inline mount option

2019-08-15 Thread Vladimir Panteleev
On Thu, 15 Aug 2019 at 04:54, Anand Jain wrote: > This causes regression, max_inline = 0 is a valid parameter. Thank you for catching that. Apologies for making such a rudimentary mistake. I will apply more diligence and resubmit.

[PATCH 1/2] fstests: make generic/500 xfs+ext4 only

2019-08-15 Thread Josef Bacik
I recently fixed some bugs in btrfs's enospc handling that made it start failing generic/500. The point of this test is to make the thin provisioned device run out of space, which results in an EIO being seen on a device from the file systems perspective. This is fine for xfs and ext4 who's metad

[PATCH 2/2] fstests: move generic/500 -> shared/001

2019-08-15 Thread Josef Bacik
Now that generic/500 is not for all file systems, move it to shared. Signed-off-by: Josef Bacik --- tests/generic/group | 1 - tests/{generic/500 => shared/001} | 0 tests/{generic/500.out => shared/001.out} | 0 tests/shared/group| 1 + 4 fi

Re: recovering from "parent transid verify failed"

2019-08-15 Thread Tim Walberg
Thanks for all the help! If I get a chance later today, I may try the patch set, but in the interest of getting things back online quicker, I may just have to recreate and restore the recovered data. The snapshots are no great loss - they're just one level of daily backups.

Re: [PATCH 1/2] fstests: make generic/500 xfs+ext4 only

2019-08-15 Thread Darrick J. Wong
On Thu, Aug 15, 2019 at 11:00:31AM -0400, Josef Bacik wrote: > I recently fixed some bugs in btrfs's enospc handling that made it start > failing generic/500. > > The point of this test is to make the thin provisioned device run out of > space, which results in an EIO being seen on a device from t

Re: [PATCH 1/2] fstests: make generic/500 xfs+ext4 only

2019-08-15 Thread Josef Bacik
On Thu, Aug 15, 2019 at 08:24:25AM -0700, Darrick J. Wong wrote: > On Thu, Aug 15, 2019 at 11:00:31AM -0400, Josef Bacik wrote: > > I recently fixed some bugs in btrfs's enospc handling that made it start > > failing generic/500. > > > > The point of this test is to make the thin provisioned devic

[PATCH] fstests: generic/500 doesn't work for btrfs

2019-08-15 Thread Josef Bacik
Btrfs does COW, so when we unlink the file we need to update metadata and write it to a new location, which we can't do because the thinp is full. This results in an EIO during a metadata write, which makes us flip read only, thus making it impossible to fstrim the fs. Just make it so we skip thi

[PATCH 3/5] Btrfs: stop clearing EXTENT_DIRTY in inode I/O tree

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval Since commit fee187d9d9dd ("Btrfs: do not set EXTENT_DIRTY along with EXTENT_DELALLOC"), we never set EXTENT_DIRTY in inode->io_tree, so we can simplify and stop trying to clear it. Signed-off-by: Omar Sandoval --- fs/btrfs/extent_io.c | 6 ++ fs/btrfs/file.c

[RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval Hello, This series adds a way to write compressed data directly to Btrfs. The intended use case is making send/receive on compressed file systems more efficient; however, the interface is general enough that it could be used in other scenarios. Patch 5 is the main change; see

[RFC PATCH 5/5] Btrfs: add ioctl for directly writing compressed data

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval This adds an API for writing compressed data directly to the filesystem. The use case that I have in mind is send/receive: currently, when sending data from one compressed filesystem to another, the sending side decompresses the data and the receiving side recompresses it befo

[RFC PATCH 4/5] fs: export rw_verify_area()

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval I'm adding a Btrfs ioctl to write compressed data, and rather than duplicating the checks in rw_verify_area(), let's just export it. Signed-off-by: Omar Sandoval --- fs/internal.h | 5 - fs/read_write.c| 1 + include/linux/fs.h | 1 + 3 files changed, 2 inserti

[PATCH 1/5] Btrfs: use correct count in btrfs_file_write_iter()

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval generic_write_checks() may modify iov_iter_count(), so we must get the count after the call, not before. Using the wrong one has a couple of consequences: 1. We check a longer range in check_can_nocow() for nowait than we're actually writing. 2. We create extra hole extent

[PATCH 2/5] Btrfs: treat RWF_{,D}SYNC writes as sync for CRCs

2019-08-15 Thread Omar Sandoval
From: Omar Sandoval In btrfs_file_write_iter(), we treat a write as synchrononous if the file is marked as synchronous. However, with pwritev2(), a write with RWF_SYNC or RWF_DSYNC is also synchronous even if the file isn't by default. Make sure we bump the sync_writers counter in that case, too,

Re: [RFC PATCH 0/5] Btrfs: add interface for writing compressed extent directly

2019-08-15 Thread Omar Sandoval
On Thu, Aug 15, 2019 at 02:04:01PM -0700, Omar Sandoval wrote: > From: Omar Sandoval > > Hello, > > This series adds a way to write compressed data directly to Btrfs. The > intended use case is making send/receive on compressed file systems more > efficient; however, the interface is general eno

Re: [PATCH] fstests: generic/500 doesn't work for btrfs

2019-08-15 Thread Darrick J. Wong
On Thu, Aug 15, 2019 at 02:26:59PM -0400, Josef Bacik wrote: > Btrfs does COW, so when we unlink the file we need to update metadata > and write it to a new location, which we can't do because the thinp is > full. This results in an EIO during a metadata write, which makes us > flip read only, thu

Re: [PATCH v3 02/27] btrfs: Get zone information of zoned block devices

2019-08-15 Thread Anand Jain
On 8/8/19 5:30 PM, Naohiro Aota wrote: If a zoned block device is found, get its zone information (number of zones and zone size) using the new helper function btrfs_get_dev_zonetypes(). To avoid costly run-time zone report commands to test the device zones type during block allocation, attach t

Re: [PATCH v3 01/27] btrfs: introduce HMZONED feature flag

2019-08-15 Thread Anand Jain
On 8/8/19 5:30 PM, Naohiro Aota wrote: This patch introduces the HMZONED incompat flag. The flag indicates that the volume management will satisfy the constraints imposed by host-managed zoned block devices. Signed-off-by: Damien Le Moal Signed-off-by: Naohiro Aota Reviewed-by: Anand Jain

Re: [PATCH v3 03/27] btrfs: Check and enable HMZONED mode

2019-08-15 Thread Anand Jain
On 8/8/19 5:30 PM, Naohiro Aota wrote: HMZONED mode cannot be used together with the RAID5/6 profile for now. Introduce the function btrfs_check_hmzoned_mode() to check this. This function will also check if HMZONED flag is enabled on the file system and if the file system consists of zoned devic