[Cluster-devel] [GFS2 PATCH] GFS2: Add to tail, not head, of transaction

2018-05-30 Thread Bob Peterson
Hi, Before this patch, frunction gfs2_trans_add_meta called list_add to add a buffer to a transaction, tr_buf. Later, in the before_commit functions, it traversed the list in sequential order, which meant that they were processed in a sub-optimal order. For example, blocks could go out in 54321

[Cluster-devel] [GFS2 PATCH] GFS2: rgrp free blocks used incorrectly

2018-05-30 Thread Bob Peterson
Hi, Before this patch, several functions in rgrp.c checked the value of rgd->rd_free_clone. That does not take into account blocks that were reserved by a multi-block reservation. This causes a problem when space gets tight in the file system. For example, when function gfs2_inplace_reserve

[Cluster-devel] [GFS2 PATCH] GFS2: Fix allocation error bug with recursive rgrp glocking

2018-05-30 Thread Bob Peterson
Hi, Before this patch function gfs2_write_begin, upon discovering an error, called gfs2_trim_blocks while the rgrp glock was still held. That's because gfs2_inplace_release is not called until later. This patch reorganizes the logic a bit so gfs2_inplace_release is called to release the lock

Re: [Cluster-devel] [PATCH v5 00/14] gfs2 iomap write support

2018-05-30 Thread Bob Peterson
Hi, - Original Message - > Here's an update of my gfs2 iomap write patch queue, with support for > buffered writes as well as direct I/O reads and writes through iomap. > This update fixes a few bugs in the gfs2 specific parts. The patches > are still meant for the upcoming merge window.

Re: [Cluster-devel] [RHEL6.10 PATCH] GFS2: allow gfs2_write_calc_reserv to use minimum values for rindex

2018-05-30 Thread Bob Peterson
Hi, Please ignore the last email. I sent it to the wrong list. This is a rhel6 patch, not meant for the development kernel tree. Bob Peterson - Original Message - > Hi, > > QE found problems with the previous patch for bug #1384184. > This replacement patch reverts the first patch and

[Cluster-devel] [RHEL6.10 PATCH] GFS2: allow gfs2_write_calc_reserv to use minimum values for rindex

2018-05-30 Thread Bob Peterson
Hi, QE found problems with the previous patch for bug #1384184. This replacement patch reverts the first patch and adds new code that passes the QE tests. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1384184 Upstream Status: N/A (not needed upstream) Build Info:

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Andreas Gruenbacher
On 30 May 2018 at 12:12, Steven Whitehouse wrote: > Hi, > > On 30/05/18 11:10, Christoph Hellwig wrote: >> >> On Wed, May 30, 2018 at 11:02:08AM +0100, Steven Whitehouse wrote: >>> >>> In that case, maybe it would be simpler to drop it for GFS2. Unless we >>> are getting a lot of benefit from

Re: [Cluster-devel] [PATCH] gfs2-utils: Fix bashism in test script

2018-05-30 Thread Andrew Price
On 30/05/18 08:09, Valentin Vidic wrote: Tests fail when /bin/sh is not /bin/bash due to missing let command: 21: Values of rg_data0, rg_data, rg_bitbytesFAILED (mkfs.at:146) /<>/tests/rgrifieldscheck.sh: 19: let: not found Applied - thanks! Andy --- tests/rgrifieldscheck.sh

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 11:10, Christoph Hellwig wrote: On Wed, May 30, 2018 at 11:02:08AM +0100, Steven Whitehouse wrote: In that case,  maybe it would be simpler to drop it for GFS2. Unless we are getting a lot of benefit from it, then we should probably just follow the generic pattern here.

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Christoph Hellwig
On Wed, May 30, 2018 at 11:02:08AM +0100, Steven Whitehouse wrote: > In that case,  maybe it would be simpler to drop it for GFS2. Unless we > are getting a lot of benefit from it, then we should probably just follow > the generic pattern here. Eventually we'll move everything to iomap, so >

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 10:59, Christoph Hellwig wrote: On Wed, May 30, 2018 at 10:30:32AM +0100, Steven Whitehouse wrote: I may have missed the context here, but I thought that the boundary was a generic thing meaning "there will have to be a metadata read before more blocks can be mapped" so I'm

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Christoph Hellwig
On Wed, May 30, 2018 at 10:30:32AM +0100, Steven Whitehouse wrote: > I may have missed the context here, but I thought that the boundary was a > generic thing meaning "there will have to be a metadata read before more > blocks can be mapped" so I'm not sure why that would now be GFS2 specific?

Re: [Cluster-devel] [PATCH v5 14/14] gfs2: Handle stuffed files in iomap_{begin, end}

2018-05-30 Thread Christoph Hellwig
> + if (gfs2_is_stuffed(ip)) { > + unstuff = pos + length > gfs2_max_stuffed_size(ip); > + > + if (!unstuff) { > + iomap->page = grab_cache_page_write_begin( > + inode->i_mapping, 0, flags); > +

[Cluster-devel] [PATCH v5 10/14] iomap: Complete partial direct I/O writes synchronously

2018-05-30 Thread Andreas Gruenbacher
According to xfstest generic/240, applications seem to expect direct I/O writes to either complete as a whole or to fail; short direct I/O writes are apparently not appreciated. This means that when only part of an asynchronous direct I/O write succeeds, we can either fail the entire write, or we

[Cluster-devel] [PATCH v5 05/14] gfs2: Iomap cleanups and improvements

2018-05-30 Thread Andreas Gruenbacher
Clean up gfs2_iomap_alloc and gfs2_iomap_get. Document how gfs2_iomap_alloc works: it now needs to be called separately after gfs2_iomap_get where necessary; this will be used later by iomap write. Move gfs2_iomap_ops into bmap.c. Introduce a new gfs2_iomap_get_alloc helper and use it in

[Cluster-devel] [PATCH v5 02/14] gfs2: hole_size improvement

2018-05-30 Thread Andreas Gruenbacher
Reimplement function hole_size based on a generic function for walking the metadata tree and rename hole_size to gfs2_hole_size. While previously, multiple invocations of hole_size were sometimes needed to walk across the entire hole, the new implementation always returns the entire hole at once

[Cluster-devel] [PATCH v5 12/14] gfs2: Remove gfs2_write_{begin, end}

2018-05-30 Thread Andreas Gruenbacher
Now that generic_file_write_iter is no longer used, there are no remaining users of these address space operations. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/aops.c | 210 - 1 file changed, 210 deletions(-) diff --git a/fs/gfs2/aops.c

[Cluster-devel] [PATCH v5 04/14] gfs2: Remove ordered write mode handling from gfs2_trans_add_data

2018-05-30 Thread Andreas Gruenbacher
In journaled data mode, we need to add each buffer head to the current transaction. In ordered write mode, we only need to add the inode to the ordered inode list. So far, both cases are handled in gfs2_trans_add_data. This makes the code look misleading and is inefficient for small block sizes

[Cluster-devel] [PATCH v5 08/14] gfs2: iomap buffered write support

2018-05-30 Thread Andreas Gruenbacher
With the traditional page-based writes, blocks are allocated separately for each page written to. With iomap writes, we can allocate a lot more blocks at once, with a fraction of the allocation overhead for each page. Split calculating the number of blocks that can be allocated at a given

[Cluster-devel] [PATCH v5 01/14] gfs2: Update find_metapath comment

2018-05-30 Thread Andreas Gruenbacher
Signed-off-by: Andreas Gruenbacher --- fs/gfs2/bmap.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c index 0590e93494f7..fcf2f7d166de 100644 --- a/fs/gfs2/bmap.c +++ b/fs/gfs2/bmap.c @@ -176,8 +176,8 @@ int gfs2_unstuff_dinode(struct

[Cluster-devel] [PATCH v5 09/14] gfs2: gfs2_extent_length cleanup

2018-05-30 Thread Andreas Gruenbacher
Now that gfs2_extent_length is no longer used for determining the size of a hole and always with an upper size limit, the function can be simplified. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/bmap.c | 24 1 file changed, 8 insertions(+), 16 deletions(-) diff --git

[Cluster-devel] [PATCH v5 13/14] iomap: inline data should be an iomap type, not a flag

2018-05-30 Thread Andreas Gruenbacher
From: Christoph Hellwig Inline data is fundamentally different from our normal mapped case in that it doesn't even have a block address. So instead of having a flag for it it should be an entirely separate iomap range type. [Minimally adjusted to fit into the gfs2 iomap-write patch queue.]

[Cluster-devel] [PATCH v5 07/14] iomap: Mark newly allocated buffer heads as new

2018-05-30 Thread Andreas Gruenbacher
In iomap_to_bh, not only mark buffer heads in IOMAP_UNWRITTEN maps as new, but also buffer heads in IOMAP_MAPPED maps with the IOMAP_F_NEW flag set. This will be used by filesystems like gfs2, which allocate blocks in iomap->begin. Minor corrections to the comment for IOMAP_UNWRITTEN maps.

[Cluster-devel] [PATCH v5 00/14] gfs2 iomap write support

2018-05-30 Thread Andreas Gruenbacher
Here's an update of my gfs2 iomap write patch queue, with support for buffered writes as well as direct I/O reads and writes through iomap. This update fixes a few bugs in the gfs2 specific parts. The patches are still meant for the upcoming merge window. The first five patches are minor gfs2

[Cluster-devel] [PATCH v5 11/14] gfs2: iomap direct I/O support

2018-05-30 Thread Andreas Gruenbacher
The page unmapping previously done in gfs2_direct_IO is now done generically in iomap_dio_rw. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/aops.c | 100 +-- fs/gfs2/bmap.c | 15 +- fs/gfs2/file.c | 138 + 3

[Cluster-devel] [PATCH v5 03/14] gfs2: gfs2_stuffed_write_end cleanup

2018-05-30 Thread Andreas Gruenbacher
First, change the sanity check in gfs2_stuffed_write_end to check for the actual write size instead of the requested write size. Second, use the existing teardown code in gfs2_write_end instead of duplicating it in gfs2_stuffed_write_end. Signed-off-by: Andreas Gruenbacher --- fs/gfs2/aops.c |

[Cluster-devel] [PATCH v5 14/14] gfs2: Handle stuffed files in iomap_{begin, end}

2018-05-30 Thread Andreas Gruenbacher
Proof of concept: move the gfs2 stuffed file handling from the iomap_write_{begin,end} handlers to the iomap_{begin,end} handlers. With this, the page written to is locked before faulting in the page to read from, so when both refer to the same page, we end up with a deadlock as demonstrated by

[Cluster-devel] [PATCH v5 06/14] iomap: Add write_{begin, end} iomap operations

2018-05-30 Thread Andreas Gruenbacher
Add write_begin and write_end operations to struct iomap_ops to provide a way of overriding the default behavior of iomap_write_begin and iomap_write_end. This is needed for implementing data journaling: in the data journaling case, pages are written into the journal before being written back to

Re: [Cluster-devel] [PATCH 11/34] iomap: move IOMAP_F_BOUNDARY to gfs2

2018-05-30 Thread Steven Whitehouse
Hi, On 30/05/18 06:50, Darrick J. Wong wrote: On Wed, May 23, 2018 at 04:43:34PM +0200, Christoph Hellwig wrote: Just define a range of fs specific flags and use that in gfs2 instead of exposing this internal flag flobally. Signed-off-by: Christoph Hellwig Looks ok to me, but better if the

[Cluster-devel] [PATCH] gfs2-utils: Fix bashism in test script

2018-05-30 Thread Valentin Vidic
Tests fail when /bin/sh is not /bin/bash due to missing let command: 21: Values of rg_data0, rg_data, rg_bitbytesFAILED (mkfs.at:146) /<>/tests/rgrifieldscheck.sh: 19: let: not found --- tests/rgrifieldscheck.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git