Re: [PATCH 7/8] nowait aio: xfs

2017-04-03 Thread Christoph Hellwig
> + if (unaligned_io) { > + /* If we are going to wait for other DIO to finish, bail */ > + if ((iocb->ki_flags & IOCB_NOWAIT) && > + atomic_read(&inode->i_dio_count)) > + return -EAGAIN; > inode_dio_wait(inode); This c

Re: [PATCH 5/8] nowait aio: return on congested block device

2017-04-03 Thread Christoph Hellwig
Please make this a REQ_* flag so that it can be passed in the bio, the request and as an argument to the get_request functions instead of testing for a bio. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo

Re: [PATCH 1/8] nowait aio: Introduce IOCB_RW_FLAG_NOWAIT

2017-04-03 Thread Christoph Hellwig
On Mon, Apr 03, 2017 at 01:53:00PM -0500, Goldwyn Rodrigues wrote: > From: Goldwyn Rodrigues > > This flag informs kernel to bail out if an AIO request will block > for reasons such as file allocations, or a writeback triggered, > or would block while allocating requests while performing > direct

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Robert Krig
On 03.04.2017 16:25, Robert Krig wrote: > > I'm gonna run a extensive memory check once I get home, since you > mentioned corrupt memory might be an issue here. > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org >

Re: python-btrfs v6: python 3 only + what now?

2017-04-03 Thread Hans van Kranenburg
On 04/02/2017 01:51 AM, Hans van Kranenburg wrote: > [bla bla bla] > > == Debian packages! == > > And... the debian packages are currently in the NEW queue of Debian! > https://ftp-master.debian.org/new/python-btrfs_6-2.html > Big thanks to kilobyte for acting as mentor for my uploads! This m

Re: [PATCH v2] btrfs: drop the nossd flag when remounting with -o ssd

2017-04-03 Thread Hans van Kranenburg
On 04/03/2017 02:24 PM, David Sterba wrote: > On Fri, Mar 31, 2017 at 10:24:57PM +0200, Hans van Kranenburg wrote: Adding the 'nossd_spread' would be good to have, even if it might be just a marginal usecase. >> >> Please no, don't make it more complex if not needed. > > The only use is

[PATCH] Btrfs: cleanup submit_one_bio

2017-04-03 Thread Liu Bo
@bio_offset is passed into submit_bio_hook and is used at btrfs_wq_submit_bio(), but only dio code makes use of @bio_offset, so remove other dead code. Cc: David Sterba Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 7 +-- 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/fs/bt

[PATCH] Btrfs: update scrub_parity to use u64 stripe_len

2017-04-03 Thread Liu Bo
Commit 3d8da6781760 ("Btrfs: fix divide error upon chunk's stripe_len") changed stripe_len in struct map_lookup to u64, but didn't update stripe_len in struct scrub_parity. This updates the type and switches to div64_u64_rem to match u64 divisor. Cc: David Sterba Signed-off-by: Liu Bo --- fs/b

[PATCH] Btrfs: switch to div64_u64 if with a u64 divisor

2017-04-03 Thread Liu Bo
This is fixing code pieces where we use div_u64 when passing a u64 divisor. Cc: David Sterba Signed-off-by: Liu Bo --- fs/btrfs/dev-replace.c | 2 +- fs/btrfs/scrub.c | 4 ++-- fs/btrfs/volumes.c | 8 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/dev

[PATCH 1/2 v2] btrfs-progs: fix btrfs send & receive with -e flag

2017-04-03 Thread Christian Brauner
The old check here tried to ensure that empty streams are not considered valid. The old check however, will always fail when only one run through the while(1) loop is needed and honor_end_cmd is set. So this: btrfs send /some/subvol | btrfs receive -e /some/ will consistently fail because -e caus

[PATCH 0/2 v2] btrfs-progs: fix btrfs send & receive with -e flag

2017-04-03 Thread Christian Brauner
Hi guys, This is the second version of the patch. It contains no functional changes but merely adds a second small patch to adapt the test to use -e with btrfs receive in order to terminate on an end marker in the stream. Thanks to David for pointing this out. Here's the description of the patch a

[PATCH 2/2 v2] tests: use receive -e to terminate on end marker

2017-04-03 Thread Christian Brauner
Signed-off-by: Christian Brauner --- tests/misc-tests/018-recv-end-of-stream/test.sh | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/misc-tests/018-recv-end-of-stream/test.sh b/tests/misc-tests/018-recv-end-of-stream/test.sh index d39683e9..90655929 100755

[PATCH 2/8] nowait aio: Return if cannot get hold of i_rwsem

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues A failure to lock i_rwsem would mean there is I/O being performed by another thread. So, let's bail. Reviewed-by: Christoph Hellwig --- mm/filemap.c | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index 1694623..e08f

[PATCH 1/8] nowait aio: Introduce IOCB_RW_FLAG_NOWAIT

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues This flag informs kernel to bail out if an AIO request will block for reasons such as file allocations, or a writeback triggered, or would block while allocating requests while performing direct I/O. Unfortunately, aio_flags is not checked for validity, which would break

[PATCH 8/8] nowait aio: btrfs

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail + i_rwsem is not lockable + NODATACOW or PREALLOC is not set + Cannot nocow at the desired location + Writing beyond end of file which is not allocated --- fs/btrfs/file.c | 25 - fs/btrfs/inod

[PATCH 4/8] nowait-aio: Introduce IOMAP_NOWAIT

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues IOCB_NOWAIT translates to IOMAP_NOWAIT for iomaps. This is used by XFS in the XFS patch. --- fs/iomap.c| 2 ++ include/linux/iomap.h | 1 + 2 files changed, 3 insertions(+) diff --git a/fs/iomap.c b/fs/iomap.c index 141c3cd..d1c8175 100644 --- a/fs/iomap.c ++

[PATCH 5/8] nowait aio: return on congested block device

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues A new flag BIO_NOWAIT is introduced to identify bio's orignating from iocb with IOCB_NOWAIT. This flag indicates to return immediately if a request cannot be made instead of retrying. To facilitate this, QUEUE_FLAG_NOWAIT is set to devices which support this. While curren

[PATCH 7/8] nowait aio: xfs

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues If IOCB_NOWAIT is set, bail if the i_rwsem is not lockable immediately. IF IOMAP_NOWAIT is set, return EAGAIN in xfs_file_iomap_begin if it needs allocation either due to file extension, writing to a hole, or COW or waiting for other DIOs to finish. --- fs/xfs/xfs_file.c

[PATCH 6/8] nowait aio: ext4

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Return EAGAIN if any of the following checks fail for direct I/O: + i_rwsem is lockable + Writing beyond end of file (will trigger allocation) + Blocks are not allocated at the write location --- fs/ext4/file.c | 48 +++- fs

[PATCH 3/8] nowait aio: return if direct write will trigger writeback

2017-04-03 Thread Goldwyn Rodrigues
From: Goldwyn Rodrigues Find out if the write will trigger a wait due to writeback. If yes, return -EAGAIN. This introduces a new function filemap_range_has_page() which returns true if the file's mapping has a page within the range mentioned. Return -EINVAL for buffered AIO: there are multiple

[PATCH 0/8 v4] No wait AIO

2017-04-03 Thread Goldwyn Rodrigues
Formerly known as non-blocking AIO. This series adds nonblocking feature to asynchronous I/O writes. io_submit() can be delayed because of a number of reason: - Block allocation for files - Data writebacks for direct I/O - Sleeping because of waiting to acquire i_rwsem - Congested block device

Re: BTRFS corruption after hardware issues

2017-04-03 Thread Brian B
On 04/03/2017 11:26 AM, Brian B wrote: > I had a stick of memory go bad in my server, and after removing it my > BTRFS filesystem seems to be damaged. > > If I mount the volume rw and try to use the volume, most actions will > hang, and I get btrfs-transactio related errors in the logs. > > 'btrfsc

Re: [PATCH] btrfs-progs: fix btrfs send & receive with -e flag

2017-04-03 Thread David Sterba
On Mon, Apr 03, 2017 at 06:07:48PM +0200, Christian Brauner wrote: > Hi guys, > > Friendly ping. Just checking in on this patch since I haven't heard back so > far > and this is a blocker in some scenarios where we're using btrfs. the test misc/018-recv-end-of-stream fails after your patch, so t

Re: [PATCH] fstests: generic: Check if cycle mount and sleep can affect fiemap result

2017-04-03 Thread Darrick J. Wong
On Mon, Apr 03, 2017 at 03:09:23PM +0800, Qu Wenruo wrote: > As long as we don't modify the on-disk data, fiemap result should always > be constant. > > Operation like cycle mount and sleep should not affect fiemap result. > While unfortunately, btrfs doesn't follow that behavior. > > Btrfs fiema

Re: [PATCH] btrfs-progs: fix btrfs send & receive with -e flag

2017-04-03 Thread Christian Brauner
Hi guys, Friendly ping. Just checking in on this patch since I haven't heard back so far and this is a blocker in some scenarios where we're using btrfs. Thanks! Christian On Fri, Mar 24, 2017 at 04:00:57PM +0100, Christian Brauner wrote: > The old check here tried to ensure that empty streams a

BTRFS corruption after hardware issues

2017-04-03 Thread Brian B
I had a stick of memory go bad in my server, and after removing it my BTRFS filesystem seems to be damaged. If I mount the volume rw and try to use the volume, most actions will hang, and I get btrfs-transactio related errors in the logs. 'btrfsck --repair' seems to sit in a loop, printing the sa

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Hans van Kranenburg
On 04/03/2017 04:20 PM, Robert Krig wrote: > > > On 03.04.2017 16:08, Hans van Kranenburg wrote: >> On 04/03/2017 12:11 PM, Robert Krig wrote: >> The corruption is at item 157. Can you attach all of the output, or >> pastebin it? >> > > I've attached the entire log of btrfs-debug-tree. This was

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Robert Krig
On 03.04.2017 16:20, Robert Krig wrote: > > On 03.04.2017 16:08, Hans van Kranenburg wrote: >> On 04/03/2017 12:11 PM, Robert Krig wrote: >> The corruption is at item 157. Can you attach all of the output, or >> pastebin it? >> > > I've attached the entire log of btrfs-debug-tree. This was genera

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Robert Krig
On 03.04.2017 16:08, Hans van Kranenburg wrote: > On 04/03/2017 12:11 PM, Robert Krig wrote: > The corruption is at item 157. Can you attach all of the output, or > pastebin it? > I've attached the entire log of btrfs-debug-tree. This was generated with btrfs-progs 4.7.3 If it makes a differen

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Hans van Kranenburg
On 04/03/2017 03:50 PM, Robert Krig wrote: > > > On 03.04.2017 12:11, Robert Krig wrote: >> Hi guys, I seem to have run into a spot of trouble with my btrfs partition. >> >> I've got 4 x 8TB in a RAID1 BTRFS configuration. >> >> I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Hans van Kranenburg
On 04/03/2017 12:11 PM, Robert Krig wrote: > Hi guys, I seem to have run into a spot of trouble with my btrfs partition. > > I've got 4 x 8TB in a RAID1 BTRFS configuration. > > I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs > progs version v4.7.3 > > Server has 8GB of Ram.

Re: [RFC PATCH v1 00/30] fs: inode->i_version rework and optimization

2017-04-03 Thread Jan Kara
On Sun 02-04-17 09:05:26, Dave Chinner wrote: > On Thu, Mar 30, 2017 at 12:12:31PM -0400, J. Bruce Fields wrote: > > On Thu, Mar 30, 2017 at 07:11:48AM -0400, Jeff Layton wrote: > > > On Thu, 2017-03-30 at 08:47 +0200, Jan Kara wrote: > > > > Because if above is acceptable we could make reported i_

Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Robert Krig
On 03.04.2017 12:11, Robert Krig wrote: > Hi guys, I seem to have run into a spot of trouble with my btrfs partition. > > I've got 4 x 8TB in a RAID1 BTRFS configuration. > > I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs > progs version v4.7.3 > > Server has 8GB of Ram. > >

Re: Is btrfs-convert able to deal with sparse files in a ext4 filesystem?

2017-04-03 Thread Austin S. Hemmelgarn
On 2017-04-01 05:48, Kai Herlemann wrote: Hi, I have on my ext4 filesystem some sparse files, mostly images from ext4 filesystems. Is btrfs-convert (4.9.1) able to deal with sparse files or can that cause any problems? I would tend to agree with some of the other people who have commented here,

Re: [PATCH v2] btrfs: drop the nossd flag when remounting with -o ssd

2017-04-03 Thread David Sterba
On Fri, Mar 31, 2017 at 10:08:50PM +0200, Adam Borowski wrote: > And when turning on nossd, drop ssd_spread. > > Reported-by: Hans van Kranenburg > Signed-off-by: Adam Borowski I've folded the two patches and queued for 4.11. Thanks. -- To unsubscribe from this list: send the line "unsubscribe

Re: [PATCH v2] btrfs: drop the nossd flag when remounting with -o ssd

2017-04-03 Thread David Sterba
On Fri, Mar 31, 2017 at 10:24:57PM +0200, Hans van Kranenburg wrote: > >> Adding the 'nossd_spread' would be good to have, even if it might be > >> just a marginal usecase. > > Please no, don't make it more complex if not needed. The only use is when ssd,ssd_spread are on and then I'd just want t

Re: mix ssd and hdd in single volume

2017-04-03 Thread Austin S. Hemmelgarn
On 2017-04-01 02:06, UGlee wrote: We are working on a small NAS server for home user. The product is equipped with a small fast SSD (around 60-120GB) and a large HDD (2T to 4T). We have two choices: 1. using bcache to accelerate io operation 2. combining SSD and HDD into a single btrfs volume.

Re: [PATCH] btrfs: delete unused member nobarriers

2017-04-03 Thread David Sterba
Please update the changelog to say why it's ok to remove it, eg. the commit that removed the last user. commit b25de9d6da49b1a8760a89672283128aa8c78345 Author: Christoph Hellwig Date: Fri Apr 24 21:41:01 2015 +0200 block: remove BIO_EOPNOTSUPP Otherwise the patch is ok. -- To unsubscribe

Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"

2017-04-03 Thread Robert Krig
Hi guys, I seem to have run into a spot of trouble with my btrfs partition. I've got 4 x 8TB in a RAID1 BTRFS configuration. I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs progs version v4.7.3 Server has 8GB of Ram. I was running duperemove using a hashfile, which seemed t

Re: mix ssd and hdd in single volume

2017-04-03 Thread Roman Mamedov
On Mon, 3 Apr 2017 11:30:44 +0300 Marat Khalili wrote: > You may want to look here: https://www.synology.com/en-global/dsm/Btrfs > . Somebody forgot to tell Synology, which already supports btrfs in all > hardware-capable devices. I think Rubicon has been crossed in > 'mass-market NAS[es]', fo

Re: Is btrfs-convert able to deal with sparse files in a ext4 filesystem?

2017-04-03 Thread Roman Mamedov
On Sun, 2 Apr 2017 09:30:46 +0300 Andrei Borzenkov wrote: > 02.04.2017 03:59, Duncan пишет: > > > > 4) In fact, since an in-place convert is almost certainly going to take > > more time than a blow-away and restore from backup, > > This caught my eyes. Why? In-place convert just needs to recre

Re: mix ssd and hdd in single volume

2017-04-03 Thread Marat Khalili
On 02/04/17 03:13, Duncan wrote: Meanwhile, since you appear to be designing a mass-market product, it's worth mentioning that btrfs is considered, certainly by its devs and users on this list, to be "still stabilizing, not fully stable and mature." [...] That doesn't sound like a particularly go

Re: Do different btrfs volumes compete for CPU?

2017-04-03 Thread Marat Khalili
On 01/04/17 13:17, Peter Grandi wrote: That "USB-connected is a rather bad idea. On the IRC channel #Btrfs whenever someone reports odd things happening I ask "is that USB?" and usually it is and then we say "good luck!" :-). You're right, but USB/eSATA arrays are dirt cheap in comparison with s

[PATCH] fstests: generic: Check if cycle mount and sleep can affect fiemap result

2017-04-03 Thread Qu Wenruo
As long as we don't modify the on-disk data, fiemap result should always be constant. Operation like cycle mount and sleep should not affect fiemap result. While unfortunately, btrfs doesn't follow that behavior. Btrfs fiemap sometimes return merged result, while after cycle mount, it returns spl