Re: bounce/forward not working due to CR at line end

[David Bremner]

Kai Harries  writes:

> David Bremner  writes:
>
>> Kai Harries  writes:
>>
>>> David Bremner  writes:
>>>
 During off-list discussion, Kai mentioned id:87a7zsm5ol@gmail.com as
 a message that he cannot bounce (and also not forward?). For me this
 forwards fine, but does not bounce because it has a corrupted Cc header,
 and my MTA rejects it.  That doesn't sound related to the original
 problem report (nothing about line endings).
>>>
>>> Can you please evaluate the following on your system:
>>>
>>>   (call-process "notmuch" nil t nil "show" "--format=raw" 
>>> "id:87a7zsm5ol@gmail.com")
>>>
>>> If I do this on my system, then the text that is inserted into the
>>> buffer has ^M (CR) at the line endings. Is this expected?
>>
>> No ^M line endings for me. Can you check the file on disk? Maybe "od -a"
>> lacking a better idea.
>
> `od -a` does also show that the lines are ending with 'cr' and 'nl'
> this is correct according to RFC 2822 [1].

That's not so clear. RFC2822 is explicitely not about disk storage; see
the Scope paragraph of RFC2822 or RFC522. I think most peoples unix mail
delivery agents use line feeds, otherwise more people would be having
problems resending mail.  See e.g. point 2 in

 
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-message_processing.html

Be that as it may, apparently your system is using CRLF.

>
> According to this [2] Emacs normally converts line endings to only 'nl'
> (newline) when opening a DOS file. This works on my system, if I open the
> file from disk no carriage-return (^M) is shown.
>
> Not working is inserting the output of `notmuch show` into a buffer. If
> I do this, then the carriage-return (^M) is shown.

Yes, notmuch-show-resend-message uses notmuch-show-view-raw-message,
which intentionally turns off any conversion

>
> David, I assume your system is also a Linux, does
>
>   notmuch show --format=raw id:1472041345236.7...@de.bosch.com | od -a
>
> also show 'cr' and 'nl' as line ending?

No, I don't have carriage returns in my actual files on disk, and
consequently not in the output of notmuch show --format=raw.

I suspect notmuch is mostly fine with CRLF line endings, due to mainly
passing everything through gmime. But in the case of notmuch show
--format=raw, this is not the case.

I did confirm that adding CRLF line endings to an existing mail file
causes the symptoms you originally described.

I'm still not sure the best place to fix this; I'm pretty sure upstream
emacs won't accept your change (or equivalent) to mime-to-mml.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stashed session keys

[Daniel Kahn Gillmor]

Hi meskio--

On Wed 2017-11-15 23:41:28 +0100, meskio wrote:
> Nice feature. I'm using it and it works fine. I notice some speed up, 
> improving 
> the painfulness of reading long encrypted threads in alot. And I like to 
> don't 
> be able to have around my old private keys.

cool, i'm glad it's working for you!

> I implemented some support for it in alot (using the patch I just sent adding 
> notmuch_message_get_property to the python binding):
> https://github.com/meskio/alot/tree/session-key

very nice :)

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: bounce/forward not working due to CR at line end

[Kai Harries]

David Bremner  writes:

> Kai Harries  writes:
>
>> David Bremner  writes:
>>
>>> During off-list discussion, Kai mentioned id:87a7zsm5ol@gmail.com as
>>> a message that he cannot bounce (and also not forward?). For me this
>>> forwards fine, but does not bounce because it has a corrupted Cc header,
>>> and my MTA rejects it.  That doesn't sound related to the original
>>> problem report (nothing about line endings).
>>
>> Can you please evaluate the following on your system:
>>
>>   (call-process "notmuch" nil t nil "show" "--format=raw" 
>> "id:87a7zsm5ol@gmail.com")
>>
>> If I do this on my system, then the text that is inserted into the
>> buffer has ^M (CR) at the line endings. Is this expected?
>
> No ^M line endings for me. Can you check the file on disk? Maybe "od -a"
> lacking a better idea.

`od -a` does also show that the lines are ending with 'cr' and 'nl'
this is correct according to RFC 2822 [1].

According to this [2] Emacs normally converts line endings to only 'nl'
(newline) when opening a DOS file. This works on my system, if I open the
file from disk no carriage-return (^M) is shown.

Not working is inserting the output of `notmuch show` into a buffer. If
I do this, then the carriage-return (^M) is shown.

David, I assume your system is also a Linux, does

  notmuch show --format=raw id:1472041345236.7...@de.bosch.com | od -a

also show 'cr' and 'nl' as line ending?

If No, then my `notmuch show` behaves different.

If Yes, then my Emacs behaves different when inserting text with DOS
line endings into a buffer.

[1] https://tools.ietf.org/html/rfc2822#section-2.2
[2] 
https://www.gnu.org/software/emacs/manual/html_mono/emacs.html#Coding-Systems
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 15/18] crypto: actually stash session keys when try-decrypt=true

[David Bremner]

Daniel Kahn Gillmor  writes:

> +Be aware that the index is likely sufficient to reconstruct
> +the cleartext of the message itself, so please ensure that the
> +notmuch message index is adequately protected.  DO NOT USE
> +``--try-decrypt=true`` without considering the security of
> +your index.
>  

This is probably just my ignorance, but doesn't stashing session keys
change this from likely to certain? Is it possible we decrypt thing and
don't get session keys.

> +test_begin_subtest "show the message body of the encrypted message"
> +notmuch dump wumpus
> +output=$(notmuch show wumpus | awk '/^\014part}/{ f=0 }; { if (f) { print $0 
> } } /^\014part{ ID: 3/{ f=1 }')
> +expected='This is a test encrypted message with a wumpus.'
> +test_expect_equal \
> +"$output" \
> +"$expected"

I'd be happier if we didn't further entrench the text format in the test
suite. How hard would it be to use json output (+maybe python?) here? 

>   *attempted = true;
>  #if (GMIME_MAJOR_VERSION < 3)
> +#if (GMIME_MAJOR_VERSION == 2 && GMIME_MINOR_VERSION == 6 && 
> GMIME_MICRO_VERSION >= 21)
> +gboolean oldgetsk = g_mime_crypto_context_get_retrieve_session_key 
> (crypto_ctx);
> +gboolean newgetsk = (decrypt_result);
> +if (newgetsk != oldgetsk)
> + /* This could return an error, but we can't do anything about it, so 
> ignore it */
> + g_mime_crypto_context_set_retrieve_session_key (crypto_ctx, newgetsk, 
> NULL);
> +#endif
>  ret = g_mime_multipart_encrypted_decrypt(part, crypto_ctx,
>decrypt_result, err);
> +#if (GMIME_MAJOR_VERSION == 2 && GMIME_MINOR_VERSION == 6 && 
> GMIME_MICRO_VERSION >= 21)
> +if (newgetsk != oldgetsk)
> + g_mime_crypto_context_set_retrieve_session_key (crypto_ctx, oldgetsk, 
> NULL);

I lost track a bit, but now there's at least 2 (maybe 3) repetitions of
this somewhat complicated test, and one more needed for
built_with.session_keys. HAVE_GMIME_SESSION_KEYS is looking better and
better.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 16/18] crypto: add --try-decrypt=nostash to avoid stashing session keys

[David Bremner]

Daniel Kahn Gillmor  writes:

> +DO NOT USE ``index.try_decrypt=true`` or ``index-only``
> +without considering the security of your index.

is index-only a typo there?

As a future improvement it would be nice to reduce some of the
documentation cut and paste for common options, perhaps with include
files? As long as the solution isn't worse than the problem of course.

> +test_expect_equal \
> +"$output" \
> +"$expected"
> +
> +
> +
> +

Is there some reason for all those blank lines?

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 13/18] cli/new, insert, reindex: change index.try_decrypt to "auto" by default

[David Bremner]

Daniel Kahn Gillmor  writes:

> The new "auto" decryption policy is not only good for "notmuch show"
> and "notmuch reindex".  It's also useful for indexing messages --
> there's no good reason to not try to go ahead and index the cleartext
> of a message that we have a stashed session key for.

I'm confused here. You talk about indexing other than reindex, but the
only tests that change are reindex? Is this meant to change "notmuch
new" behaviour?

d

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: bounce/forward not working due to CR at line end

[Tomi Ollila]

On Wed, Nov 15 2017, David Bremner wrote:

> Kai Harries  writes:
>
>> David Bremner  writes:
>>
>>> During off-list discussion, Kai mentioned id:87a7zsm5ol@gmail.com as
>>> a message that he cannot bounce (and also not forward?). For me this
>>> forwards fine, but does not bounce because it has a corrupted Cc header,
>>> and my MTA rejects it.  That doesn't sound related to the original
>>> problem report (nothing about line endings).
>>
>> Can you please evaluate the following on your system:
>>
>>   (call-process "notmuch" nil t nil "show" "--format=raw" 
>> "id:87a7zsm5ol@gmail.com")
>>
>> If I do this on my system, then the text that is inserted into the
>> buffer has ^M (CR) at the line endings. Is this expected?
>
> No ^M line endings for me. Can you check the file on disk? Maybe "od -a"
> lacking a better idea.

$ notmuch search --output=files id:87a7zsm5ol@gmail.com

outputs path to one file in my system. the mail content is base64-encoded
and there is no cr characters in my system 
(I used base64 -d 'edited-mail ' | tr '\r' '$' | less to look)

As Kai sent the mail, his local copy may be encoded differently...

Tomi

(Hoo, I just wondered whether there is base64-decode-region in emacs --
there is a function exactly that name... had to M-x read-only-mode (where
did M-x toggle-read-only disappear) in notmuch-raw-id:... buffer before
trying that (i.e. in emacs mua, in show buffer, first pressed 'V' to see
raw message contents.)

PS2: it is possible this mail doesn't reach Kai, as gmail is somewhat
picky from where it accepts email... we may get this fixed any century
now, though.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch