Re: linux-next: manual merge of the block tree with the fscrypt tree
Hi all,
On Thu, 9 Jul 2020 13:02:54 +1000 Stephen Rothwell
wrote:
>
> Today's linux-next merge of the block tree got a conflict in:
>
> fs/buffer.c
>
> between commit:
>
> 4f74d15fe408 ("ext4: add inline encryption support")
>
> from the fscrypt tree and commit:
>
> ed9b3196d2b2 ("fs: remove a weird comment in submit_bh_wbc")
>
> from the block tree.
>
>
> diff --cc fs/buffer.c
> index dc5e05b47646,2725ebbcfdc2..
> --- a/fs/buffer.c
> +++ b/fs/buffer.c
> @@@ -3039,14 -3040,7 +3039,10 @@@ static int submit_bh_wbc(int op, int op
> if (test_set_buffer_req(bh) && (op == REQ_OP_WRITE))
> clear_buffer_write_io_error(bh);
>
> - /*
> - * from here on down, it's all bio -- do the initial mapping,
> - * submit_bio -> generic_make_request may further map this bio around
> - */
> bio = bio_alloc(GFP_NOIO, 1);
> +
> +fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
> +
> bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
> bio_set_dev(bio, bh->b_bdev);
> bio->bi_write_hint = write_hint;
This is now also a conflict between the zonefs tree and the fscrypt
tree (since the zonefs tree has merged part of the block tree).
--
Cheers,
Stephen Rothwell
pgplabmJZksMB.pgp
Description: OpenPGP digital signature
linux-next: manual merge of the block tree with the fscrypt tree
Hi all,
Today's linux-next merge of the block tree got a conflict in:
fs/buffer.c
between commit:
4f74d15fe408 ("ext4: add inline encryption support")
from the fscrypt tree and commit:
ed9b3196d2b2 ("fs: remove a weird comment in submit_bh_wbc")
from the block tree.
I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging. You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.
--
Cheers,
Stephen Rothwell
diff --cc fs/buffer.c
index dc5e05b47646,2725ebbcfdc2..
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@@ -3039,14 -3040,7 +3039,10 @@@ static int submit_bh_wbc(int op, int op
if (test_set_buffer_req(bh) && (op == REQ_OP_WRITE))
clear_buffer_write_io_error(bh);
- /*
-* from here on down, it's all bio -- do the initial mapping,
-* submit_bio -> generic_make_request may further map this bio around
-*/
bio = bio_alloc(GFP_NOIO, 1);
+
+ fscrypt_set_bio_crypt_ctx_bh(bio, bh, GFP_NOIO);
+
bio->bi_iter.bi_sector = bh->b_blocknr * (bh->b_size >> 9);
bio_set_dev(bio, bh->b_bdev);
bio->bi_write_hint = write_hint;
pgpnSejFEB0sp.pgp
Description: OpenPGP digital signature
Re: linux-next: manual merge of the block tree with the fscrypt tree
On Wed, Jan 16, 2019 at 07:27:25AM -0700, Jens Axboe wrote:
> On 1/16/19 12:39 AM, Ming Lei wrote:
> > On Tue, Jan 15, 2019 at 08:17:36PM -0700, Jens Axboe wrote:
> >> On 1/15/19 8:13 PM, Ming Lei wrote:
> >>> On Tue, Jan 15, 2019 at 07:55:39PM -0700, Jens Axboe wrote:
> On 1/15/19 7:25 PM, Stephen Rothwell wrote:
> > Hi all,
> >
> > Today's linux-next merge of the block tree got a conflict in:
> >
> > fs/ext4/readpage.c
> >
> > between commit:
> >
> > acc9eb0a6073 ("ext4: add fs-verity read support")
> >
> > from the fscrypt tree and commit:
> >
> > eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate
> > over multi-page bvec")
> >
> > from the block tree.
> >
> > I fixed it up (see below - the former moved the code modified by the
> > latter) and can carry the fix as necessary. This is now fixed as far as
> > linux-next is concerned, but any non trivial conflicts should be
> > mentioned
> > to your upstream maintainer when your tree is submitted for merging.
> > You may also want to consider cooperating with the maintainer of the
> > conflicting tree to minimise any particularly complex conflicts.
>
> Ming, I'm pulling this, I thought we agreed none of these bullshit
> renames? The fact that a patch looks like this:
>
> - for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)
> \
> + for_each_segment(bv, (it)->bvecs, __cur_iter,
> __cur_iter) \
>
> is SUPER annoying and does NOTHING but to cause merge conflicts.
>
> Resend it without that.
> >>>
> >>> We need to differentiate 'segment' with 'bvec' in bvec helpers, which is
> >>> usually seldom used by drivers. For example, only two in-tree users(ceph,
> >>> iov_iter).
> >>> That is why I rename it, and seems Christoph prefers to do it too.
> >>
> >> If you want to do a rename, then we do it after. I don't want to deal with
> >> weeks and weeks of fallout from this. Write a rename script that we can
> >> then run at the end of the next merge window. You're going to be playing
> >> catch-up until that happens if we go the current route, and honestly
> >> I'm not at all interested in the fallout from that.
> >>
> >> I know exactly what will happen until 5.1-rc opens, and what my tree will
> >> look like from having to deal with this. And then I know exactly what Linus
> >> is going to say, and I can't even argue against it, since he'll be totally
> >> right.
> >>
> >> Hence it's not going to happen this way.
> >
> > I can remove the renaming in patch 'block: rename bvec helpers', but
> > change on bio_for_each_segment_all() is inevitable, and it is still an
> > API change, so merge conflict can't avoid too.
>
> That's not what I'm complaining about, API changes are inevitable for
> something like this. What I'm complaining about is the very example
> I posted above, and which has already caused issues. That's a frivolous
> name change. Don't do it.
Hi Jens,
I may avoid the rename for avoiding merge conflict.
However, given we have been back and forth for this naming thing several
times, I'd rather explain the story one more time:
In V13's patch4, we rename:
for_each_bvec
bvec_iter_bvec
bvec_iter_len
bvec_iter_page
bvec_iter_offset
into:
for_each_segment
segment_iter_bvec
segment_iter_len
segment_iter_page
segment_iter_offset
so that these helpers named with 'segment' only deal with single-page
bvec, or called segment. We re-introduce helpers named with 'bvec'
for multi-page bvec, as suggested by Christoph.
If the above renaming has to be avoided, we need one new name
for multi-page bvec helpers, previous I name it as 'mp_bvec', such as:
for_each_mp_bvec
mp_bvec_iter_bvec
mp_bvec_iter_len
mp_bvec_iter_page
mp_bvec_iter_offset
However, Christoph were not very happy with the above name.
Given you mentioned one more rename after the patches are merged should
be fine, I understand the new name of 'mp_bvec' should be fine for you
because it is just short-life.
Please confirm or provide another better name, so that we can move on.
Thanks,
Ming
Re: linux-next: manual merge of the block tree with the fscrypt tree
On 1/16/19 12:39 AM, Ming Lei wrote:
> On Tue, Jan 15, 2019 at 08:17:36PM -0700, Jens Axboe wrote:
>> On 1/15/19 8:13 PM, Ming Lei wrote:
>>> On Tue, Jan 15, 2019 at 07:55:39PM -0700, Jens Axboe wrote:
On 1/15/19 7:25 PM, Stephen Rothwell wrote:
> Hi all,
>
> Today's linux-next merge of the block tree got a conflict in:
>
> fs/ext4/readpage.c
>
> between commit:
>
> acc9eb0a6073 ("ext4: add fs-verity read support")
>
> from the fscrypt tree and commit:
>
> eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
> multi-page bvec")
>
> from the block tree.
>
> I fixed it up (see below - the former moved the code modified by the
> latter) and can carry the fix as necessary. This is now fixed as far as
> linux-next is concerned, but any non trivial conflicts should be mentioned
> to your upstream maintainer when your tree is submitted for merging.
> You may also want to consider cooperating with the maintainer of the
> conflicting tree to minimise any particularly complex conflicts.
Ming, I'm pulling this, I thought we agreed none of these bullshit
renames? The fact that a patch looks like this:
- for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)
\
+ for_each_segment(bv, (it)->bvecs, __cur_iter, __cur_iter)
\
is SUPER annoying and does NOTHING but to cause merge conflicts.
Resend it without that.
>>>
>>> We need to differentiate 'segment' with 'bvec' in bvec helpers, which is
>>> usually seldom used by drivers. For example, only two in-tree users(ceph,
>>> iov_iter).
>>> That is why I rename it, and seems Christoph prefers to do it too.
>>
>> If you want to do a rename, then we do it after. I don't want to deal with
>> weeks and weeks of fallout from this. Write a rename script that we can
>> then run at the end of the next merge window. You're going to be playing
>> catch-up until that happens if we go the current route, and honestly
>> I'm not at all interested in the fallout from that.
>>
>> I know exactly what will happen until 5.1-rc opens, and what my tree will
>> look like from having to deal with this. And then I know exactly what Linus
>> is going to say, and I can't even argue against it, since he'll be totally
>> right.
>>
>> Hence it's not going to happen this way.
>
> I can remove the renaming in patch 'block: rename bvec helpers', but
> change on bio_for_each_segment_all() is inevitable, and it is still an
> API change, so merge conflict can't avoid too.
That's not what I'm complaining about, API changes are inevitable for
something like this. What I'm complaining about is the very example
I posted above, and which has already caused issues. That's a frivolous
name change. Don't do it.
--
Jens Axboe
Re: linux-next: manual merge of the block tree with the fscrypt tree
On Tue, Jan 15, 2019 at 08:17:36PM -0700, Jens Axboe wrote:
> On 1/15/19 8:13 PM, Ming Lei wrote:
> > On Tue, Jan 15, 2019 at 07:55:39PM -0700, Jens Axboe wrote:
> >> On 1/15/19 7:25 PM, Stephen Rothwell wrote:
> >>> Hi all,
> >>>
> >>> Today's linux-next merge of the block tree got a conflict in:
> >>>
> >>> fs/ext4/readpage.c
> >>>
> >>> between commit:
> >>>
> >>> acc9eb0a6073 ("ext4: add fs-verity read support")
> >>>
> >>> from the fscrypt tree and commit:
> >>>
> >>> eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
> >>> multi-page bvec")
> >>>
> >>> from the block tree.
> >>>
> >>> I fixed it up (see below - the former moved the code modified by the
> >>> latter) and can carry the fix as necessary. This is now fixed as far as
> >>> linux-next is concerned, but any non trivial conflicts should be mentioned
> >>> to your upstream maintainer when your tree is submitted for merging.
> >>> You may also want to consider cooperating with the maintainer of the
> >>> conflicting tree to minimise any particularly complex conflicts.
> >>
> >> Ming, I'm pulling this, I thought we agreed none of these bullshit
> >> renames? The fact that a patch looks like this:
> >>
> >> - for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)
> >>\
> >> + for_each_segment(bv, (it)->bvecs, __cur_iter, __cur_iter)
> >>\
> >>
> >> is SUPER annoying and does NOTHING but to cause merge conflicts.
> >>
> >> Resend it without that.
> >
> > We need to differentiate 'segment' with 'bvec' in bvec helpers, which is
> > usually seldom used by drivers. For example, only two in-tree users(ceph,
> > iov_iter).
> > That is why I rename it, and seems Christoph prefers to do it too.
>
> If you want to do a rename, then we do it after. I don't want to deal with
> weeks and weeks of fallout from this. Write a rename script that we can
> then run at the end of the next merge window. You're going to be playing
> catch-up until that happens if we go the current route, and honestly
> I'm not at all interested in the fallout from that.
>
> I know exactly what will happen until 5.1-rc opens, and what my tree will
> look like from having to deal with this. And then I know exactly what Linus
> is going to say, and I can't even argue against it, since he'll be totally
> right.
>
> Hence it's not going to happen this way.
I can remove the renaming in patch 'block: rename bvec helpers', but
change on bio_for_each_segment_all() is inevitable, and it is still an
API change, so merge conflict can't avoid too.
Thanks,
Ming
Re: linux-next: manual merge of the block tree with the fscrypt tree
On 1/15/19 8:13 PM, Ming Lei wrote:
> On Tue, Jan 15, 2019 at 07:55:39PM -0700, Jens Axboe wrote:
>> On 1/15/19 7:25 PM, Stephen Rothwell wrote:
>>> Hi all,
>>>
>>> Today's linux-next merge of the block tree got a conflict in:
>>>
>>> fs/ext4/readpage.c
>>>
>>> between commit:
>>>
>>> acc9eb0a6073 ("ext4: add fs-verity read support")
>>>
>>> from the fscrypt tree and commit:
>>>
>>> eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
>>> multi-page bvec")
>>>
>>> from the block tree.
>>>
>>> I fixed it up (see below - the former moved the code modified by the
>>> latter) and can carry the fix as necessary. This is now fixed as far as
>>> linux-next is concerned, but any non trivial conflicts should be mentioned
>>> to your upstream maintainer when your tree is submitted for merging.
>>> You may also want to consider cooperating with the maintainer of the
>>> conflicting tree to minimise any particularly complex conflicts.
>>
>> Ming, I'm pulling this, I thought we agreed none of these bullshit
>> renames? The fact that a patch looks like this:
>>
>> - for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)
>> \
>> + for_each_segment(bv, (it)->bvecs, __cur_iter, __cur_iter)
>> \
>>
>> is SUPER annoying and does NOTHING but to cause merge conflicts.
>>
>> Resend it without that.
>
> We need to differentiate 'segment' with 'bvec' in bvec helpers, which is
> usually seldom used by drivers. For example, only two in-tree users(ceph,
> iov_iter).
> That is why I rename it, and seems Christoph prefers to do it too.
If you want to do a rename, then we do it after. I don't want to deal with
weeks and weeks of fallout from this. Write a rename script that we can
then run at the end of the next merge window. You're going to be playing
catch-up until that happens if we go the current route, and honestly
I'm not at all interested in the fallout from that.
I know exactly what will happen until 5.1-rc opens, and what my tree will
look like from having to deal with this. And then I know exactly what Linus
is going to say, and I can't even argue against it, since he'll be totally
right.
Hence it's not going to happen this way.
--
Jens Axboe
Re: linux-next: manual merge of the block tree with the fscrypt tree
On Tue, Jan 15, 2019 at 07:55:39PM -0700, Jens Axboe wrote:
> On 1/15/19 7:25 PM, Stephen Rothwell wrote:
> > Hi all,
> >
> > Today's linux-next merge of the block tree got a conflict in:
> >
> > fs/ext4/readpage.c
> >
> > between commit:
> >
> > acc9eb0a6073 ("ext4: add fs-verity read support")
> >
> > from the fscrypt tree and commit:
> >
> > eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
> > multi-page bvec")
> >
> > from the block tree.
> >
> > I fixed it up (see below - the former moved the code modified by the
> > latter) and can carry the fix as necessary. This is now fixed as far as
> > linux-next is concerned, but any non trivial conflicts should be mentioned
> > to your upstream maintainer when your tree is submitted for merging.
> > You may also want to consider cooperating with the maintainer of the
> > conflicting tree to minimise any particularly complex conflicts.
>
> Ming, I'm pulling this, I thought we agreed none of these bullshit
> renames? The fact that a patch looks like this:
>
> - for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)
> \
> + for_each_segment(bv, (it)->bvecs, __cur_iter, __cur_iter)
> \
>
> is SUPER annoying and does NOTHING but to cause merge conflicts.
>
> Resend it without that.
We need to differentiate 'segment' with 'bvec' in bvec helpers, which is
usually seldom used by drivers. For example, only two in-tree users(ceph,
iov_iter).
That is why I rename it, and seems Christoph prefers to do it too.
Thanks,
Ming
Re: linux-next: manual merge of the block tree with the fscrypt tree
On 1/15/19 7:25 PM, Stephen Rothwell wrote:
> Hi all,
>
> Today's linux-next merge of the block tree got a conflict in:
>
> fs/ext4/readpage.c
>
> between commit:
>
> acc9eb0a6073 ("ext4: add fs-verity read support")
>
> from the fscrypt tree and commit:
>
> eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
> multi-page bvec")
>
> from the block tree.
>
> I fixed it up (see below - the former moved the code modified by the
> latter) and can carry the fix as necessary. This is now fixed as far as
> linux-next is concerned, but any non trivial conflicts should be mentioned
> to your upstream maintainer when your tree is submitted for merging.
> You may also want to consider cooperating with the maintainer of the
> conflicting tree to minimise any particularly complex conflicts.
Ming, I'm pulling this, I thought we agreed none of these bullshit
renames? The fact that a patch looks like this:
- for_each_bvec(bv, (it)->bvecs, __cur_iter, __cur_iter)\
+ for_each_segment(bv, (it)->bvecs, __cur_iter, __cur_iter) \
is SUPER annoying and does NOTHING but to cause merge conflicts.
Resend it without that.
--
Jens Axboe
linux-next: manual merge of the block tree with the fscrypt tree
Hi all,
Today's linux-next merge of the block tree got a conflict in:
fs/ext4/readpage.c
between commit:
acc9eb0a6073 ("ext4: add fs-verity read support")
from the fscrypt tree and commit:
eb754eb2a953 ("block: allow bio_for_each_segment_all() to iterate over
multi-page bvec")
from the block tree.
I fixed it up (see below - the former moved the code modified by the
latter) and can carry the fix as necessary. This is now fixed as far as
linux-next is concerned, but any non trivial conflicts should be mentioned
to your upstream maintainer when your tree is submitted for merging.
You may also want to consider cooperating with the maintainer of the
conflicting tree to minimise any particularly complex conflicts.
--
Cheers,
Stephen Rothwell
diff --cc fs/ext4/readpage.c
index 0b353a0634f3,e53639784892..
--- a/fs/ext4/readpage.c
+++ b/fs/ext4/readpage.c
@@@ -61,124 -56,6 +61,125 @@@ static inline bool ext4_bio_encrypted(s
#endif
}
+/* postprocessing steps for read bios */
+enum bio_post_read_step {
+ STEP_INITIAL = 0,
+ STEP_DECRYPT,
+ STEP_VERITY,
+};
+
+struct bio_post_read_ctx {
+ struct bio *bio;
+ struct work_struct work;
+ unsigned int cur_step;
+ unsigned int enabled_steps;
+};
+
+static void __read_end_io(struct bio *bio)
+{
+ struct page *page;
+ struct bio_vec *bv;
+ int i;
++ struct bvec_iter_all iter_all;
+
- bio_for_each_segment_all(bv, bio, i) {
++ bio_for_each_segment_all(bv, bio, i, iter_all) {
+ page = bv->bv_page;
+
+ /* PG_error was set if any post_read step failed */
+ if (bio->bi_status || PageError(page)) {
+ ClearPageUptodate(page);
+ SetPageError(page);
+ } else {
+ SetPageUptodate(page);
+ }
+ unlock_page(page);
+ }
+ if (bio->bi_private)
+ mempool_free(bio->bi_private, bio_post_read_ctx_pool);
+ bio_put(bio);
+}
+
+static void bio_post_read_processing(struct bio_post_read_ctx *ctx);
+
+static void decrypt_work(struct work_struct *work)
+{
+ struct bio_post_read_ctx *ctx =
+ container_of(work, struct bio_post_read_ctx, work);
+
+ fscrypt_decrypt_bio(ctx->bio);
+
+ bio_post_read_processing(ctx);
+}
+
+static void verity_work(struct work_struct *work)
+{
+ struct bio_post_read_ctx *ctx =
+ container_of(work, struct bio_post_read_ctx, work);
+
+ fsverity_verify_bio(ctx->bio);
+
+ bio_post_read_processing(ctx);
+}
+
+static void bio_post_read_processing(struct bio_post_read_ctx *ctx)
+{
+ /*
+ * We use different work queues for decryption and for verity because
+ * verity may require reading metadata pages that need decryption, and
+ * we shouldn't recurse to the same workqueue.
+ */
+ switch (++ctx->cur_step) {
+ case STEP_DECRYPT:
+ if (ctx->enabled_steps & (1 << STEP_DECRYPT)) {
+ INIT_WORK(>work, decrypt_work);
+ fscrypt_enqueue_decrypt_work(>work);
+ return;
+ }
+ ctx->cur_step++;
+ /* fall-through */
+ case STEP_VERITY:
+ if (ctx->enabled_steps & (1 << STEP_VERITY)) {
+ INIT_WORK(>work, verity_work);
+ fsverity_enqueue_verify_work(>work);
+ return;
+ }
+ ctx->cur_step++;
+ /* fall-through */
+ default:
+ __read_end_io(ctx->bio);
+ }
+}
+
+static struct bio_post_read_ctx *get_bio_post_read_ctx(struct inode *inode,
+ struct bio *bio,
+ pgoff_t index)
+{
+ unsigned int post_read_steps = 0;
+ struct bio_post_read_ctx *ctx = NULL;
+
+ if (IS_ENCRYPTED(inode) && S_ISREG(inode->i_mode))
+ post_read_steps |= 1 << STEP_DECRYPT;
+#ifdef CONFIG_FS_VERITY
+ if (inode->i_verity_info != NULL &&
+ (index < ((i_size_read(inode) + PAGE_SIZE - 1) >> PAGE_SHIFT)))
+ post_read_steps |= 1 << STEP_VERITY;
+#endif
+ if (post_read_steps) {
+ ctx = mempool_alloc(bio_post_read_ctx_pool, GFP_NOFS);
+ if (!ctx)
+ return ERR_PTR(-ENOMEM);
+ ctx->bio = bio;
+ ctx->enabled_steps = post_read_steps;
+ bio->bi_private = ctx;
+ }
+ return ctx;
+}
+
+static bool bio_post_read_required(struct bio *bio)
+{
+ return bio->bi_private && !bio->bi_status;
+}
+
/*
* I/O completion handler for multipage BIOs.
*
pgpY4AWXN5xU8.pgp
Description: OpenPGP digital signature
