Re: RAID 6 corrupted

2017-05-18 Thread Adam Borowski
On Thu, May 18, 2017 at 06:08:56AM +, Duncan wrote: > Adam Borowski posted on Thu, 18 May 2017 05:29:57 +0200 as excerpted: > > On Thu, May 18, 2017 at 04:09:38AM +0200, Łukasz Wróblewski wrote: > >> I will try when stable 4.12 comes out. > > > > It won't come out for ~2.5 months. > > ?? 4.1

btrfs metadata reclaim behavior/performance characteristics

2017-05-18 Thread Nikolay Borisov
Hello list, As a result of the perf regression discovered in [0] I also took the time to investigate the behavior of the metadata reclaim algorithm and how it chagned with the introduction of the ticketed reservations. My tests were performed on the following commits: Pre-ticketed => c83f8effe

Re: [RFC PATCH 0/2] Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/16/2017 10:07 PM, Bart Van Assche wrote: On Tue, 2017-05-16 at 17:39 +0800, Anand Jain wrote: BTRFS wanted a block device flush function which does not wait for its completion, so that the flush for the next device can be called in the same thread. Here is a RFC patch to provide the fun

Re: [RFC PATCH 0/2] Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/18/2017 01:14 AM, David Sterba wrote: On Tue, May 16, 2017 at 02:07:23PM +, Bart Van Assche wrote: On Tue, 2017-05-16 at 17:39 +0800, Anand Jain wrote: BTRFS wanted a block device flush function which does not wait for its completion, so that the flush for the next device can be cal

Re: [PATCH 1/2] block: Introduce blkdev_issue_flush_no_wait()

2017-05-18 Thread Anand Jain
On 05/16/2017 07:56 PM, Christoph Hellwig wrote: On Tue, May 16, 2017 at 05:39:13PM +0800, Anand Jain wrote: blkdev_issue_flush() is a blocking function and returns only after the flush bio is completed, so a module handling more than one device can't issue flush for all the devices unless it

Re: [PATCH] btrfs: add mount umount logs

2017-05-18 Thread Anand Jain
On 05/16/2017 05:25 PM, Qu Wenruo wrote: Yes, this is indeed a nice feature. But what about only outputting this info when BTRFS_DEBUG is selected? I'm sure there will be some guys not liking such kernel message flooding everywhere when doing testing. Thanks. Will keep in mind to make that

[PATCH RFC] vfs: add mount umount logs

2017-05-18 Thread Anand Jain
By looking at the logs we should be able to know when was the FS mounted and unmounted and the options used, so to help forensic investigations. Signed-off-by: Anand Jain --- You may want to know that, during boot and shutdown this adds roughly 25 lines more logs depending on the config, and it l

Re: [PATCH] btrfs: add mount umount logs

2017-05-18 Thread Anand Jain
On 05/16/2017 07:23 PM, David Sterba wrote: On Tue, May 16, 2017 at 04:41:49PM +0800, Anand Jain wrote: By looking at the logs we should be able to know when the FS was mounted and unmounted and the options used, so to help forensic investigations. Could be a useful feature, but you're addin

Re: Can't remount a BTRFS partition read write after a drive failure

2017-05-18 Thread Sylvain Leroux
On 05/17/2017 09:19 AM, Ivan Sizov wrote: > > The drive is not reliable. And I noticed when there is an error and the > > USB device appears to be dead to the kernel, I am later unable to > > remount rw the drive. I can mount it read only though. > > This seems to be a systematic behavior. And it o

Re: [PATCH RFC] vfs: add mount umount logs

2017-05-18 Thread Carlos Maiolino
On Thu, May 18, 2017 at 06:08:04PM +0800, Anand Jain wrote: > By looking at the logs we should be able to know when was the FS > mounted and unmounted and the options used, so to help forensic > investigations. > > Signed-off-by: Anand Jain > --- > You may want to know that, during boot and shutd

[PATCH 14/15] blk-mq: switch ->queue_rq return value to blk_status_t

2017-05-18 Thread Christoph Hellwig
Use the same values for use for request completion errors as the return value from ->queue_rq. BLK_STS_RESOURCE is special cased to cause a requeue, and all the others are completed as-is. Signed-off-by: Christoph Hellwig --- block/blk-mq.c| 37 --

[PATCH 09/15] dm: don't return errnos from ->map

2017-05-18 Thread Christoph Hellwig
Instead use the special DM_MAPIO_KILL return value to return -EIO just like we do for the request based path. Note that dm-log-writes returned -ENOMEM in a few places, which now becomes -EIO instead. No consumer treats -ENOMEM special so this shouldn't be an issue (and it should use a mempool to

[PATCH 11/15] fs-writeback: move wbc_to_write_flags out of line

2017-05-18 Thread Christoph Hellwig
This way writeback.h doesn't need to pull in blk_types.h. Signed-off-by: Christoph Hellwig --- fs/fs-writeback.c | 13 + include/linux/writeback.h | 11 +-- 2 files changed, 14 insertions(+), 10 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 63

[PATCH 06/15] fs: simplify dio_bio_complete

2017-05-18 Thread Christoph Hellwig
Only read bio->bi_error once in the common path. Signed-off-by: Christoph Hellwig --- fs/direct-io.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/fs/direct-io.c b/fs/direct-io.c index 04247a6c3f73..bb711e4b86c2 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -477

[PATCH 12/15] block: merge blk_types.h into bio.h

2017-05-18 Thread Christoph Hellwig
We've cleaned up our headers sufficiently that we don't need this split anymore. Signed-off-by: Christoph Hellwig --- block/blk-wbt.c| 2 +- drivers/target/target_core_pscsi.c | 2 +- include/linux/bio.h| 307 +++- include/l

[PATCH 08/15] dm mpath: merge do_end_io_bio into multipath_end_io_bio

2017-05-18 Thread Christoph Hellwig
This simplifies the code and especially the error passing a bit and will help with the next patch. Signed-off-by: Christoph Hellwig --- drivers/md/dm-mpath.c | 42 -- 1 file changed, 16 insertions(+), 26 deletions(-) diff --git a/drivers/md/dm-mpath.c b/d

[PATCH 10/15] dm: change ->end_io calling convention

2017-05-18 Thread Christoph Hellwig
Turn the error paramter into a pointer so that target drivers can change the value, and make sure only DM_ENDIO_* values are returned from the methods. Signed-off-by: Christoph Hellwig --- drivers/md/dm-cache-target.c | 4 ++-- drivers/md/dm-flakey.c| 8 drivers/md/dm-log-wri

[PATCH 07/15] block_dev: propagate bio_iov_iter_get_pages error in __blkdev_direct_IO

2017-05-18 Thread Christoph Hellwig
Once we move the block layer to its own status code we'll still want to propagate the bio_iov_iter_get_pages, so restructure __blkdev_direct_IO to take ret into account when returning the errno. --- fs/block_dev.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/fs/block_

[PATCH 05/15] fs: remove the unused error argument to dio_end_io()

2017-05-18 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- fs/btrfs/inode.c | 6 +++--- fs/direct-io.c | 3 +-- include/linux/fs.h | 2 +- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 17cbe9306faf..758b2666885e 100644 --- a/fs/btrfs/inode.c +++ b/fs/btr

[PATCH 03/15] gfs2: remove the unused sd_log_error field

2017-05-18 Thread Christoph Hellwig
Signed-off-by: Christoph Hellwig --- fs/gfs2/incore.h | 1 - fs/gfs2/lops.c | 4 +--- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/fs/gfs2/incore.h b/fs/gfs2/incore.h index b7cf65d13561..aa3d44527fa2 100644 --- a/fs/gfs2/incore.h +++ b/fs/gfs2/incore.h @@ -815,7 +815,6 @@ struc

[PATCH 01/15] nvme-lightnvm: use blk_execute_rq in nvme_nvm_submit_user_cmd

2017-05-18 Thread Christoph Hellwig
Instead of reinventing it poorly. Signed-off-by: Christoph Hellwig Reviewed-by: Javier González --- drivers/nvme/host/lightnvm.c | 12 +--- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/drivers/nvme/host/lightnvm.c b/drivers/nvme/host/lightnvm.c index f5df78ed1e10..f3885

[PATCH 04/15] dm: fix REQ_RAHEAD handling

2017-05-18 Thread Christoph Hellwig
A few (but not all) dm targets use a special EWOULDBLOCK error code for failing REQ_RAHEAD requests that fail due to a lack of available resources. But no one else knows about this magic code, and lower level drivers also don't generate it when failing read-ahead requests for similar reasons. So r

dedicated error codes for the block layer

2017-05-18 Thread Christoph Hellwig
This series introduces a new blk_status_t error code type for the block layer so that we can have tigher control and explicit semantics for block layer errors. All but the last three patches are cleanups that lead to the new type. The series it mostly limited to the block layer and drivers, and t

[PATCH 02/15] scsi/osd: don't save block errors into req_results

2017-05-18 Thread Christoph Hellwig
We will only have sense data if the command exectured and got a SCSI result, so this is pointless. Signed-off-by: Christoph Hellwig --- drivers/scsi/osd/osd_initiator.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/osd/osd_initiator.c b/drivers/scsi/osd/osd_ini

[PATCH] Btrfs: work around maybe-uninitialized warning

2017-05-18 Thread Arnd Bergmann
A rewrite of btrfs_submit_direct_hook appears to have introduced a warning: fs/btrfs/inode.c: In function 'btrfs_submit_direct_hook': fs/btrfs/inode.c:8467:14: error: 'bio' may be used uninitialized in this function [-Werror=maybe-uninitialized] Where the 'bio' variable was previously initialize

Re: [RFC PATCH v3.2 5/6] btrfs: qgroup: Introduce extent changeset for qgroup reserve functions

2017-05-18 Thread David Sterba
On Thu, May 18, 2017 at 08:24:26AM +0800, Qu Wenruo wrote: > >> +static inline void extent_changeset_init(struct extent_changeset > >> *changeset) > >> +{ > >> + changeset->bytes_changed = 0; > >> + ulist_init(&changeset->range_changed); > >> +} > >> + > >> +static inline struct extent_changeset

confusing "no space left" -- how to troubleshoot and "be prepared"?

2017-05-18 Thread Yaroslav Halchenko
our python-based program crashed with File "/home/yoh/proj/datalad/datalad/venv-tests/local/lib/python2.7/site-packages/gitdb/stream.py", line 695, in write os.write(self._fd, data) OSError: [Errno 28] No space left on device but as far as I could see there still should be both data and m

Re: [fs-btrfs] question about apparent useless function call

2017-05-18 Thread David Sterba
On Wed, May 17, 2017 at 05:37:44PM -0500, Gustavo A. R. Silva wrote: > While looking into Coverity ID 1398149 I ran into the following piece > of code at fs/btrfs/zlib.c:295: > > 295done: > 296zlib_inflateEnd(&workspace->strm); > 297if (data_in) > 298kunmap(pages_

Fwd: confusing "no space left" -- how to troubleshoot and "be prepared"?

2017-05-18 Thread Peter Becker
2017-05-18 15:41 GMT+02:00 Yaroslav Halchenko : > > our python-based program crashed with > > File > "/home/yoh/proj/datalad/datalad/venv-tests/local/lib/python2.7/site-packages/gitdb/stream.py", > line 695, in write > os.write(self._fd, data) > OSError: [Errno 28] No space left on device >

Re: [PATCH 12/15] block: merge blk_types.h into bio.h

2017-05-18 Thread Bart Van Assche
On Thu, 2017-05-18 at 15:18 +0200, Christoph Hellwig wrote: > We've cleaned up our headers sufficiently that we don't need this split > anymore. Hello Christoph, Request-based drivers need the structure definitions from and the type definitions from but do not need the definition of struct bio.

Btrfs progs release 4.11

2017-05-18 Thread David Sterba
Hi, btrfs-progs version 4.11 have been released. It's a small release with handful of fixes and minor other updates. Changes: * receive: fix handling empty stream with -e (multi-stream) * send dump: fix printing long file names * stability fixes for: dump-super, print-tree, check * option

Re: btrfs metadata reclaim behavior/performance characteristics

2017-05-18 Thread Liu Bo
On Thu, May 18, 2017 at 11:40:05AM +0300, Nikolay Borisov wrote: > Hello list, > > As a result of the perf regression discovered in [0] I also took the time to > investigate the behavior of the metadata reclaim algorithm and how it chagned > with the introduction of the ticketed reservations. My

Re: dedicated error codes for the block layer

2017-05-18 Thread David Sterba
On Thu, May 18, 2017 at 03:17:57PM +0200, Christoph Hellwig wrote: > This series introduces a new blk_status_t error code type for the block > layer so that we can have tigher control and explicit semantics for > block layer errors. > > All but the last three patches are cleanups that lead to the

Re: [PATCH] btrfs: use correct types for page indices in btrfs_page_exists_in_range

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 04:00:35PM +0200, David Sterba wrote: > Variables start_idx and end_idx are supposed to hold a page index > derived from the file offsets. The int type is not the right one though, > offsets larger than 1 << 44 will get silently trimmed off the high bits. > (1 << 44 is 16TiB

Re: btrfs metadata reclaim behavior/performance characteristics

2017-05-18 Thread Nikolay Borisov
On 18.05.2017 17:45, Liu Bo wrote: > On Thu, May 18, 2017 at 11:40:05AM +0300, Nikolay Borisov wrote: > > Just some random thoughts here. > > Hmm, not sure if this matters, but fstests now doesn't set --mixed even if the > disk size is as small as 256mb. So are you testing a mixed btrfs or

Re: [PATCH RFC] vfs: add mount umount logs

2017-05-18 Thread Darrick J. Wong
On Thu, May 18, 2017 at 06:08:04PM +0800, Anand Jain wrote: > By looking at the logs we should be able to know when was the FS > mounted and unmounted and the options used, so to help forensic > investigations. > > Signed-off-by: Anand Jain > --- > You may want to know that, during boot and shutd

Re: btrfs metadata reclaim behavior/performance characteristics

2017-05-18 Thread Liu Bo
On Thu, May 18, 2017 at 06:41:08PM +0300, Nikolay Borisov wrote: > > > On 18.05.2017 17:45, Liu Bo wrote: > > On Thu, May 18, 2017 at 11:40:05AM +0300, Nikolay Borisov wrote: > > > > > > > Just some random thoughts here. > > > > Hmm, not sure if this matters, but fstests now doesn't set --mi

Re: [PATCH RFC] vfs: add mount umount logs

2017-05-18 Thread Anand Jain
On 05/18/2017 08:23 PM, Carlos Maiolino wrote: On Thu, May 18, 2017 at 06:08:04PM +0800, Anand Jain wrote: By looking at the logs we should be able to know when was the FS mounted and unmounted and the options used, so to help forensic investigations. Signed-off-by: Anand Jain --- You may wa

Re: [PATCH v2 1/2] btrfs: Separate space_info create/update

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 06:07:42PM +0300, Nikolay Borisov wrote: > Currently the struct space_info creation code is intermixed in the > udpate_space_info function. There are well-defined points at which the we > actually want to create brand-new space_info structs (e.g. during mount of > the filesy

Re: [PATCH v2 2/2] btrfs: Refactor update_space_info

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 06:07:43PM +0300, Nikolay Borisov wrote: > Following the factoring out of the creation code udpate_space_info can only > be called for already-existing space_info structs. Remove superfulous error > handling and use the return value to return a pointer to the found space_inf

Re: [PATCH 1/3] btrfs: fix race with relocation recovery and fs_root setup

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 11:38:34AM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > If we have to recover relocation during mount, we'll ultimately have to > evict the orphan inode. That goes through the reservation dance, where > priority_reclaim_metadata_space and flush_space expect fs_in

Re: [PATCH 2/3] btrfs: cleanup root usage by btrfs_get_alloc_profile

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 11:38:35AM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > There are two places where we don't already know what kind of alloc > profile we need before calling btrfs_get_alloc_profile, but we need > access to a root everywhere we call it. > > This patch adds helpers

Re: [PATCH 3/3] btrfs: remove root usage from can_overcommit

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 11:38:36AM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > can_overcommit using the root to determine the allocation profile > is the only use of a root in the call graph below reserve_metadata_bytes. > > It turns out that we only need to know whether the allocation

Re: [PATCH RFC] vfs: add mount umount logs

2017-05-18 Thread Anand Jain
On 05/19/2017 01:39 AM, Darrick J. Wong wrote: On Thu, May 18, 2017 at 06:08:04PM +0800, Anand Jain wrote: By looking at the logs we should be able to know when was the FS mounted and unmounted and the options used, so to help forensic investigations. Signed-off-by: Anand Jain --- You may wa

Re: [PATCH 1/2] btrfs-progs: Fix restoring image from multi devices fs into single device

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 02:41:21AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > We correctly build an image from a multiple devices filesystem but when > restoring the image into a single device we were missing updating the > number of devices in the superblock to the value 1 (we a

Re: [PATCH 2/2] btrfs-progs: test for restoring multiple devices fs into a single device

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 02:41:31AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that we are able to create an image from a multiple devices fs, that > we are able to restore that image into a single device and finally that we > are able to mount it. > > Signed-off-by: Filipe

Re: [PATCH] btrfs: fix memory leak in update_space_info failure path

2017-05-18 Thread Liu Bo
On Wed, May 17, 2017 at 09:49:37AM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > If we fail to add the space_info kobject, we'll leak the memory > for the percpu counter. > > Fixes: 6ab0a2029c (btrfs: publish allocation data in sysfs) > Cc: # v3.14+ > Signed-off-by: Jeff Mahoney Looks

Re: [PATCH 03/10] btrfs: scrub: simplify cleanup of wr_ctx in scrub_free_ctx

2017-05-18 Thread Liu Bo
On Tue, May 16, 2017 at 07:10:29PM +0200, David Sterba wrote: > We don't need to take the mutex and zero out wr_cur_bio, as this is > called after the scrub finished. > > Signed-off-by: David Sterba Looks good. Reviewed-by: Liu Bo Thanks, -liubo -- To unsubscribe from this list: send the lin

Re: [RFC PATCH v3.2 5/6] btrfs: qgroup: Introduce extent changeset for qgroup reserve functions

2017-05-18 Thread Qu Wenruo
At 05/18/2017 09:45 PM, David Sterba wrote: On Thu, May 18, 2017 at 08:24:26AM +0800, Qu Wenruo wrote: +static inline void extent_changeset_init(struct extent_changeset *changeset) +{ + changeset->bytes_changed = 0; + ulist_init(&changeset->range_changed); +} + +static inline struc

Re: [PATCH 06/10] btrfs: scrub: use bool for flush_all_writes

2017-05-18 Thread Liu Bo
On Tue, May 16, 2017 at 07:10:38PM +0200, David Sterba wrote: > flush_all_writes is an atomic but does not use the semantics at all, > it's just on/off indicator, we can use bool. > It might use atomic to avoid reordering, but I'm not sure if reordering could really happen here. Thanks, -liubo

Re: Btrfs progs release 4.11

2017-05-18 Thread Tsutomu Itoh
Hi David, Could you please apply the following patch? [PATCH] btrfs-progs: tests: remove variable quotation from convert-tests http://marc.info/?l=linux-btrfs&m=149490441323349&w=2 Thanks, Tsutomu On 2017/05/18 23:35, David Sterba wrote: > Hi, > > btrfs-progs version 4.11 have been released.

Re: [PATCH 07/10] btrfs: scrub: use fs_info::sectorsize and drop it from scrub context

2017-05-18 Thread Liu Bo
On Tue, May 16, 2017 at 07:10:41PM +0200, David Sterba wrote: > As we now have the node/block sizes in fs_info, we can use them and can > drop the local copies. > > Signed-off-by: David Sterba Reviewed-by: Liu Bo Thanks, -liubo -- To unsubscribe from this list: send the line "unsubscribe linu

Re: Can't remount a BTRFS partition read write after a drive failure

2017-05-18 Thread Duncan
Sylvain Leroux posted on Thu, 18 May 2017 09:22:55 +0200 as excerpted: > Thank you Chris, Ivan, for your answers. > Here we are in a very special use case. But I think we would see a > similar behavior if some drive case or cable was dying, the > administrator replaced it, but was unable to remou

Re: [PATCH v4 05/27] btrfs: btrfs_wait_tree_block_writeback can be void return

2017-05-18 Thread Liu Bo
On Tue, May 09, 2017 at 11:49:08AM -0400, Jeff Layton wrote: > Nothing checks its return value. Reviewed-by: Liu Bo -liubo > > Signed-off-by: Jeff Layton > --- > fs/btrfs/disk-io.c | 6 +++--- > fs/btrfs/disk-io.h | 2 +- > 2 files changed, 4 insertions(+), 4 deletions(-) > > diff --git a/fs

4.11.0: kernel BUG at fs/btrfs/ctree.h:1779!

2017-05-18 Thread Marc MERLIN
Looks like all the unhelpful BUG() aren't gone yet :-/ This one is really not helpful, I don't even know which one of my filesystems caused the crash :( Why is this not remounting the filesystem read only? Really, from a user and admin perspective, this is really not helpful. Could someone who k

Re: btrfs list corruption and soft lockups while testing writeback error handling

2017-05-18 Thread Liu Bo
On Fri, May 12, 2017 at 10:22:47AM -0400, Jeff Layton wrote: > On Fri, 2017-05-12 at 08:12 -0400, Jeff Layton wrote: > > On Thu, 2017-05-11 at 15:56 -0400, Chris Mason wrote: > > > On 05/11/2017 03:52 PM, Jeff Layton wrote: > > > > On Thu, 2017-05-11 at 07:13 -0400, Jeff Layton wrote: > > > > > I f

Re: [LKP] [btrfs] "fio: pid=2214, got signal=7" error showed in fio test for btrfs

2017-05-18 Thread Ye Xiaolong
On 03/29, Ye Xiaolong wrote: >Attach kmsg and reproduce script. > >Note: kernel cmdline contained "memmap=104G!4G memmap=104G!132G" Hi, Any feedback about this issue? Thanks, Xiaolong > >Thanks, >Xiaolong > >On 03/29, kernel test robot wrote: >>Hi, >> >>We detected below error messages in fio p