stitching threads (v3 14/17)

2019-05-27 Thread Daniel Kahn Gillmor
hm, it appears that notmuch-emacs sends duplicate References: headers during reply when i add that manually to the headers field during compose. and then when notmuch indexes a message, it only indexes the first References: header it finds. These are curious things i find as i try to stitch the

Re: [PATCH v3 14/17] test: ensure that protected headers appear in notmuch-emacs search as expected

2019-05-27 Thread Daniel Kahn Gillmor
On Mon 2019-05-27 17:35:44 -0400, Daniel Kahn Gillmor wrote: > We initially test only notmuch-search; tests for other functionality > come in different patchsets later. > > Signed-off-by: Daniel Kahn Gillmor sorry, this patch (v3 14/17) is a minor update to the protected h

[PATCH v3 14/17] test: ensure that protected headers appear in notmuch-emacs search as expected

2019-05-27 Thread Daniel Kahn Gillmor
We initially test only notmuch-search; tests for other functionality come in different patchsets later. Signed-off-by: Daniel Kahn Gillmor --- test/T358-emacs-protected-headers.sh | 36 1 file changed, 36 insertions(+) create mode 100755 test/T358-emacs-protected

Re: [PATCH v3 03/17] test: new test framework to compare json parts

2019-05-27 Thread Daniel Kahn Gillmor
On Mon 2019-05-27 16:34:27 -0400, Daniel Kahn Gillmor wrote: > From: Jameson Graef Rollins > > This makes it easier to write fairly compact, readable tests of json > output, without needing to sanitize away parts that we don't care > about. woops, patches crossed in the et

_notmuch_database_log vs _notmuch_database_log_append [was: Re: [PATCH v2 10/17] indexing: record protected subject when indexing cleartext]

2019-05-27 Thread Daniel Kahn Gillmor
On Mon 2019-05-27 07:24:41 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> +status = _notmuch_message_crypto_potential_payload (msg_crypto, clear, >> GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT); >> +_index_mime_part (m

[PATCH v3 10/17] indexing: record protected subject when indexing cleartext

2019-05-27 Thread Daniel Kahn Gillmor
When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 46 +++--- lib/message.cc

Re: [PATCH v2 06/17] cli/show: add information about which headers were protected

2019-05-27 Thread Daniel Kahn Gillmor
On Mon 2019-05-27 07:12:52 -0300, David Bremner wrote: > I think you also need to add a definition for header_name to schemata > (in the same way that messageid is defined as a string). thanks, done in v3, which you should see shortly. > The name "header-mask" is a bit generic, but I don't have

[PATCH v3 06/17] cli/show: add information about which headers were protected

2019-05-27 Thread Daniel Kahn Gillmor
This allows a clever UI frontend to mark whether a header was protected (or not), and if it was protected, to show the details to an interested user. As before, we only handle Subject for now, but we might be able to handle other headers in the future. Signed-off-by: Daniel Kahn Gillmor

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread Daniel Kahn Gillmor
On Sun 2019-05-26 13:54:29 -0300, David Bremner wrote: > % uncrustify -c devel/uncrustify.cfg --replace $files thanks for this pointer, i will experiment with it. > If we do decide to rip off the bandage, that will cause a certain amount > of rebasing pain for any patch series in flight; now

[PATCH v2 17/17] cli/reply: pull proposed subject line from the message, not the index

2019-05-26 Thread Daniel Kahn Gillmor
that problem, and doesn't cause any additional tests to fail. Signed-off-by: Daniel Kahn Gillmor --- notmuch-reply.c | 2 +- test/T356-protected-headers.sh | 1 - test/T358-emacs-protected-headers.sh | 1 - 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/notmuch

[PATCH v2 02/17] util/crypto: add information about the payload part

2019-05-26 Thread Daniel Kahn Gillmor
://dkg.fifthhorseman.net/blog/e-mail-cryptography.html#cryptographic-envelope for more description of the Cryptographic Payload vs. the Cryptographic Envelope. Signed-off-by: Daniel Kahn Gillmor --- util/crypto.c | 37 + util/crypto.h | 5 + 2 files changed

[PATCH v2 13/17] test: try indexing nested messages and protected headers

2019-05-26 Thread Daniel Kahn Gillmor
We want to make sure that internally-forwarded messages don't end up "bubbling up" when they aren't actually the cryptographic payload. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 6 ...pted-message-with-forwarded-attachmen

[PATCH v2 04/17] cli/show: add tests for viewing protected headers

2019-05-26 Thread Daniel Kahn Gillmor
Here we add several variant e-mail messages, some of which have correctly-structured protected headers, and some of which do not. The goal of the tests is to ensure that the right protected subjects get reported. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh

[PATCH v2 08/17] test: show cryptographic envelope information for signed mails

2019-05-26 Thread Daniel Kahn Gillmor
Make sure that we emit the correct cryptographic envelope status for cleartext signed messages. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 11 ++- .../signed-protected-header.eml | 29 +++ .../protected-headers/simple

[PATCH v2 07/17] test: add test for missing external subject

2019-05-26 Thread Daniel Kahn Gillmor
Adding another test to ensure that we handle protected headers gracefully when no external subject is present. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 6 .../subjectless-protected-header.eml | 29 +++ 2 files changed

[PATCH v2 16/17] test: reply (in cli and emacs) should protect indexed sensitive headers

2019-05-26 Thread Daniel Kahn Gillmor
. We will fix the two broken tests in a subsequent patch. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh | 7 + test/T358-emacs-protected-headers.sh | 45 2 files changed, 52 insertions(+) diff --git a/test/T356-protected-headers.sh b

[PATCH v2 11/17] test: protected headers should work when both encrypted and signed.

2019-05-26 Thread Daniel Kahn Gillmor
t (outside Subject: matches inner Subject:) See the discussion at https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html#protected-headers for more details about the nuances between signed, stripped, and stubbed headers. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-

[PATCH v2 10/17] indexing: record protected subject when indexing cleartext

2019-05-26 Thread Daniel Kahn Gillmor
When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 42 ++ lib/message.cc

[PATCH v2 06/17] cli/show: add information about which headers were protected

2019-05-26 Thread Daniel Kahn Gillmor
age's Subject looked like on the wire" feature in expert mode. As before, we only handle Subject for now, but we might be able to handle other headers in the future. Signed-off-by: Daniel Kahn Gillmor --- devel/schemata | 6 ++ notmuch-show.c

[PATCH v2 03/17] test: new test framework to compare json parts

2019-05-26 Thread Daniel Kahn Gillmor
From: Jameson Graef Rollins This makes it easier to write fairly compact, readable tests of json output, without needing to sanitize away parts that we don't care about. Signed-off-by: Daniel Kahn Gillmor --- test/json_check_nodes.py | 113 +++ test/test

Protected Headers (2nd major revision, more testing!)

2019-05-26 Thread Daniel Kahn Gillmor
Hi all-- Way back in id:20180511055544.13676-1-...@fifthhorseman.net, i proposed support for protected headers (in particular, for being able to read and search for subject lines of encrypted messages which protect the Subject). Although that series was reviewed by Bremner, i never managed to

[PATCH v2 01/17] cli/show: emit headers after emitting body

2019-05-26 Thread Daniel Kahn Gillmor
This paves the way for emitting protected headers after verification and decryption, because it means that the headers will only be emitted after the body has been parsed. Signed-off-by: Daniel Kahn Gillmor --- notmuch-show.c| 6 +++--- test/T170-sexp.sh | 10 +- 2 files changed, 8

[PATCH v2 12/17] test: after reindexing, only legitimate protected subjects are searchable

2019-05-26 Thread Daniel Kahn Gillmor
This test scans for all the possible protected headers (including bogus/broken ones) that are present in the protected-headers corpus, trying to make sure that only the ones that are not broken or malformed show up in a search after re-indexing. Signed-off-by: Daniel Kahn Gillmor --- test/T356

[PATCH v2 05/17] cli/show: emit payload subject instead of outside subject

2019-05-26 Thread Daniel Kahn Gillmor
Correctly fix the two outstanding tests so that the protected (hidden) subject is properly reported. Signed-off-by: Daniel Kahn Gillmor --- notmuch-client.h | 2 +- notmuch-reply.c| 4 +++- notmuch-show.c | 14 +- test/T356-protected

[PATCH v2 15/17] test: emacs/show: ensure that protected headers appear as expected

2019-05-26 Thread Daniel Kahn Gillmor
This tests notmuch-show; headers appear appropriately based on the setting of notmuch-crypto-process-mime. Signed-off-by: Daniel Kahn Gillmor --- test/T358-emacs-protected-headers.sh | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/test/T358-emacs

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread Daniel Kahn Gillmor
On Sun 2019-05-26 09:01:46 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> This is the third revision of the series originally posted at >> id:20190424183113.29242-1-...@fifthhorseman.net (revision 2 was at >> id:20190520032228.27420-1-...@fifthhorsema

[PATCH v2] test: report summary even when aborting

2019-05-26 Thread Daniel Kahn Gillmor
the return code). If we do abort due to this high-level failure, though, we should also announce to the user that we're doing so as close to the end of the process as possible, to make it easier to find the problem. Signed-off-by: Daniel Kahn Gillmor --- test/notmuch-test | 14 +++--- 1 file

Re: [PATCH] test: report summary even when aborting

2019-05-26 Thread Daniel Kahn Gillmor
Thanks for the feedback, Tomi! On Sat 2019-05-25 22:41:58 +0300, Tomi Ollila wrote: > On Sat, May 25 2019, Daniel Kahn Gillmor wrote: > >> In certain cases of test suite failure, the summary report was not >> being printed. In particular, any failure on the parallel test

[PATCH v3 2/4] cli: expose message-wide crypto status from mime-node

2019-05-25 Thread Daniel Kahn Gillmor
The mime node context (a per-message context) gains a cryptographic status object, and the mime_node_t object itself can return a view on that status to an interested party. The status is not yet populated, and for now we can keep that view read-only, so that it can only be populated/modified

revision 3: easing access to the cryptographic envelope

2019-05-25 Thread Daniel Kahn Gillmor
This is the third revision of the series originally posted at id:20190424183113.29242-1-...@fifthhorseman.net (revision 2 was at id:20190520032228.27420-1-...@fifthhorseman.net) This series addresses comments raised by David Bremner in his review. Thanks, Bremner! The most significant change

[PATCH v3 3/4] mime-node: track whole-message crypto state while walking the tree

2019-05-25 Thread Daniel Kahn Gillmor
Deliberately populate the message's cryptographic status while walking the MIME tree from the CLI. Note that the additional numchild argument added to _mime_node_create is a passthrough needed to be able to adequately populate the crypto state object. --- mime-node.c | 23 +--

[PATCH v3 4/4] cli/show: emit new whole-message crypto status output

2019-05-25 Thread Daniel Kahn Gillmor
This allows MUAs that don't want to think about per-mime-part cryptographic status to have a simple high-level overview of the message's cryptographic state. Sensibly structured encrypted and/or signed messages will work fine with this. The only requirement for the simplest encryption + signing

[PATCH v3 1/4] util/crypto: _notmuch_message_crypto: tracks message-wide crypto state

2019-05-25 Thread Daniel Kahn Gillmor
E-mail encryption and signatures reported by notmuch are at the MIME part level. This makes sense in the dirty details, but for users we need to have a per-message conception of the cryptographic state of the e-mail. (see https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html for more

[PATCH] test: report summary even when aborting

2019-05-25 Thread Daniel Kahn Gillmor
the return code). Signed-off-by: Daniel Kahn Gillmor --- test/notmuch-test | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/test/notmuch-test b/test/notmuch-test index 50ed8721..d835e152 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -45,6 +45,8 @@ else fi

Re: [PATCH 3/3] emacs: Drop content-free "Unknown signature status" button

2019-05-25 Thread Daniel Kahn Gillmor
On Fri 2019-05-24 22:38:12 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> On Thu 2019-05-23 22:13:59 -0300, David Bremner wrote: >>> Daniel Kahn Gillmor writes: >>> >>>> diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-cryp

Re: [PATCH v2 4/4] cli/show: emit new whole-message crypto status output

2019-05-24 Thread Daniel Kahn Gillmor
On Fri 2019-05-24 16:09:38 -0400, Daniel Kahn Gillmor wrote: > On Thu 2019-05-23 07:50:43 -0300, David Bremner wrote: >> Daniel Kahn Gillmor writes: >> >>> headers:headers, >>> +crypto?:crypto, # omitted if crypto disabled, or if no

Re: [PATCH v2 1/4] util/crypto: _notmuch_message_crypto: tracks message-wide crypto state

2019-05-24 Thread Daniel Kahn Gillmor
On Wed 2019-05-22 09:18:53 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> +static int >> +_notmuch_message_crypto_cleanup (_notmuch_message_crypto_t *msg_crypto) >> +{ >> +if (!msg_crypto) >> +return 0; >> +if (msg_crypto->

Re: [PATCH v2 4/4] cli/show: emit new whole-message crypto status output

2019-05-24 Thread Daniel Kahn Gillmor
On Thu 2019-05-23 07:50:43 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> headers:headers, >> +crypto?:crypto, # omitted if crypto disabled, or if no part >> was signed or encrypted. >> body?: [part

Re: [PATCH] test: allow disabling timeout with NOTMUCH_TEST_TIMEOUT=0

2019-05-24 Thread Daniel Kahn Gillmor
On Wed 2019-05-22 08:52:55 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> Tests appear to be hanging when run under GNU timeout on debian >> stretch. To aid in diagnosing this or similar problems, it's handy to >> be able to disable timeout from the comman

Re: [PATCH 3/3] emacs: Drop content-free "Unknown signature status" button

2019-05-24 Thread Daniel Kahn Gillmor
On Thu 2019-05-23 22:13:59 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el >> index 353f721e..68171153 100644 >> --- a/emacs/notmuch-crypto.el >> +++ b/emacs/notmuch-crypto.el >>

Re: [PATCH] test: redirect STDIN from /dev/tty

2019-05-21 Thread Daniel Kahn Gillmor
On Tue 2019-05-21 23:17:02 +0300, Tomi Ollila wrote: > Without this stdin may be anything that parent process provided for it. I'm fine with this change -- i can confirm that it avoids the hanging problem on debian stable for me. please merge either this, or

Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel

2019-05-21 Thread Daniel Kahn Gillmor
On Mon 2019-05-20 23:32:20 -0300, David Bremner wrote: > I wonder if most/all of our use of gdb could be replaced by writing > simple shim libraries and using LD_PRELOAD. I would have no objection to such a change -- it seems like a plausible approach to me -- but i'm not prepared to write it (or

Re: [PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel

2019-05-21 Thread Daniel Kahn Gillmor
Hi Tomi-- On Tue 2019-05-21 09:12:12 +0300, Tomi Ollila wrote: > This looks like a good read: > https://ftp.gnu.org/old-gnu/Manuals/gdb/html_node/gdb_21.html > and is used to speculate below. Thanks for this pointer, and for the additional analysis. While i understand this problem much better

[PATCH] test: avoid hanging older gdb under GNU timeout and moreutils parallel

2019-05-20 Thread Daniel Kahn Gillmor
ging problems on the older setup without understanding them exactly. I welcome any improvements or actual understanding. Signed-off-by: Daniel Kahn Gillmor --- test/T050-new.sh| 2 +- test/T060-count.sh | 2 +- test/T070-insert.sh | 4 3 files changed, 6 insertions(+), 2 deletions(

Re: parallel tests broken on Debian stable

2019-05-20 Thread Daniel Kahn Gillmor
On Mon 2019-05-20 19:49:02 -0400, Daniel Kahn Gillmor wrote: > However, i tried selectively upgrading all the versions of all of these > packages *except for gdb* to the version in buster (or to the version > from backports, in the case of the kernel). and i'm *still* seeing the

Re: parallel tests broken on Debian stable

2019-05-20 Thread Daniel Kahn Gillmor
On Mon 2019-05-20 13:27:03 -0400, Daniel Kahn Gillmor wrote: > c) we should avoid the timeout hanging :) I dug into this today, and i'm reporting back my findings. I have what appears to be a fix (see below), but i don't understand it, so i'm not advocating for it. To be clear: my two t

[PATCH] test: allow disabling timeout with NOTMUCH_TEST_TIMEOUT=0

2019-05-20 Thread Daniel Kahn Gillmor
Tests appear to be hanging when run under GNU timeout on debian stretch. To aid in diagnosing this or similar problems, it's handy to be able to disable timeout from the command line at will. Signed-off-by: Daniel Kahn Gillmor --- test/notmuch-test | 9 +++-- 1 file changed, 7 insertions

Re: parallel tests broken on Debian stable

2019-05-20 Thread Daniel Kahn Gillmor
On Mon 2019-05-20 21:55:05 +0300, Tomi Ollila wrote: > On Mon, May 20 2019, Daniel Kahn Gillmor wrote: >> b) we should have a clearer sense of which tests completed and which >> did not >> […] >> if someone else wants to >> improve the test suite

Re: [PATCH v2] configure: Ensure that GMime can extract session keys

2019-05-20 Thread Daniel Kahn Gillmor
On Mon 2019-05-20 16:46:23 -0300, David Bremner wrote: > I pushed this, and broke travis. Can you please have a look at > > https://travis-ci.org/notmuch/notmuch/builds/534979532 Sorry about this, the flaw was a difference between the ubuntu and debian linker default behavior. I've responded

[PATCH 2/3] configure: handle TEMP_GPG more robustly

2019-05-20 Thread Daniel Kahn Gillmor
We never want ./configure to try to do something with an unassigned variable. So, make the directory $TEMP_GPG at the start of the testing of session-key handling, and clean it up afterwards as long as the directory exists. Signed-off-by: Daniel Kahn Gillmor --- configure | 10 +++--- 1

[PATCH 3/3] configure: make _check_session_keys work with an as-needed linker

2019-05-20 Thread Daniel Kahn Gillmor
is needed, that library will no longer be linked in the final outcome. _check_session_keys.c was failing on those systems. Signed-off-by: Daniel Kahn Gillmor --- configure | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure b/configure index 2531cc39..9e8a21bf 100755 --- a/confi

[PATCH 1/3] configure: better error handling on session key check.

2019-05-20 Thread Daniel Kahn Gillmor
ion_keys.c around if ./configure fails, so that the user can play with it more easily for debugging. * let error messages show when _check_session_keys.c is built. Signed-off-by: Daniel Kahn Gillmor --- configure | 16 ++-- 1 file changed, 10 insertions(+), 6 deletions(-)

clean up session-key configure checks

2019-05-20 Thread Daniel Kahn Gillmor
The Ubuntu linker prefers --as-needed. This is nice, but it means that our _check_session_key.c configure-time script (introduced by me in 30c4fa3702f90572afcd1984dbd7aba70f2b4fd9) fails on xenial. In the course of debugging the reason, i found several other infelicities in the error cases in

Re: parallel tests broken on Debian stable

2019-05-20 Thread Daniel Kahn Gillmor
Thanks for catching this, Bremner. On Fri 2019-05-10 07:45:18 -0300, David Bremner wrote: > In a debian stretch neither gnu parallel nor moreutils parallel > successfully completes the tests. I have confirmed this misbehavior on debian stretch with moreutils parallel. on stretch (with gmime 3.0

Re: [PATCH] test/emacs: revert invalid-from test to pre-86f89385 behaviour

2019-05-20 Thread Daniel Kahn Gillmor
On Sat 2019-05-11 20:45:59 -0600, David Bremner wrote: > To the best of my understanding, this original behaviour was what > Carl's homebrew parser produced. With commit 86f89385 Austin switched > to using GMime (2.6). This produced arguably worse results, but since > the input was bad, we could

[PATCH 2/2] tests: fail and report when a parallel build fails (or times out)

2019-05-20 Thread Daniel Kahn Gillmor
When a parallel build fails (or when it times out, if timeout is present), the test suite should not blithely succeed. Catch these failures and at least report them. Signed-off-by: Daniel Kahn Gillmor --- test/notmuch-test | 5 + 1 file changed, 5 insertions(+) diff --git a/test/notmuch

[PATCH 1/2] tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)

2019-05-20 Thread Daniel Kahn Gillmor
The current 2 minute timeout is reasonable, but to exercise the test suite or induce timeout failures, we might want to make it shorter. This makes it configurable so you can run (for example): make check NOTMUCH_TEST_TIMEOUT=10s We stick with the default of 2m. Signed-off-by: Daniel Kahn

[PATCH v2 3/4] mime-node: track whole-message crypto state while walking the tree

2019-05-19 Thread Daniel Kahn Gillmor
Deliberately populate the message's cryptographic status while walking the MIME tree from the CLI. Note that the additional numchild argument added to _mime_node_create is a passthrough needed to be able to adequately populate the crypto state object. --- mime-node.c | 23 +--

revision 2: easing access to the cryptographic envelope

2019-05-19 Thread Daniel Kahn Gillmor
This is the second revision of the series originally posted at id:20190424183113.29242-1-...@fifthhorseman.net (that series no longer applies directly to master due to all the cleanup that has been merged recently) This series is an important baseline for my work on protected headers, which i

[PATCH v2 4/4] cli/show: emit new whole-message crypto status output

2019-05-19 Thread Daniel Kahn Gillmor
This allows MUAs that don't want to think about per-mime-part cryptographic status to have a simple high-level overview of the message's cryptographic state. Sensibly structured encrypted and/or signed messages will work fine with this. The only requirement for the simplest encryption + signing

[PATCH v2 2/4] cli: expose message-wide crypto status from mime-node

2019-05-19 Thread Daniel Kahn Gillmor
The mime node context (a per-message context) gains a cryptographic status object, and the mime_node_t object itself can return a view on that status to an interested party. The status is not yet populated, and for now we can keep that view read-only, so that it can only be populated/modified

Re: [PATCH 3/3] emacs: Drop content-free "Unknown signature status" button

2019-05-19 Thread Daniel Kahn Gillmor
On Mon 2019-04-22 13:18:14 -0400, Daniel Kahn Gillmor wrote: > When we have not been able to evaluate the signature status of a given > MIME part, showing a content-free (and interaction-free) "[ Unknown > signature status ]" button doesn't really help the user at all, and

[PATCH v2 1/4] util/crypto: _notmuch_message_crypto: tracks message-wide crypto state

2019-05-19 Thread Daniel Kahn Gillmor
E-mail encryption and signatures reported by notmuch are at the MIME part level. This makes sense in the dirty details, but for users we need to have a per-message conception of the cryptographic state of the e-mail. (see https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html for more

Re: [PATCH v2] configure: Ensure that GMime can extract session keys

2019-05-19 Thread Daniel Kahn Gillmor
On Mon 2019-05-06 16:16:55 -0400, Daniel Kahn Gillmor wrote: > GMime 3.0 and higher can extract session keys, but it will *not* > extract session keys if it was built with --disable-crypto, or if it > was built against GPGME version < 1.8.0. > > Notmuch currently expects to

Re: [PATCH 3/3] test/crypto: add_gnupg_home should have ultimate trust on "its own" key

2019-05-07 Thread Daniel Kahn Gillmor
On Tue 2019-05-07 06:50:29 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> The typical use case for gpg is that if you control a secret key, you >> mark it with "ultimate" ownertrust. >> >> The opaque --import-ownertrust mechanism is GnuPG's

[PATCH] test: avoid unnecessary extraction of the test fingerprint

2019-05-07 Thread Daniel Kahn Gillmor
valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u Signed-off-by: Daniel Kahn Gillmor --- test/T357-index-decryption.sh | 2 -- 1 file changed, 2 deletions(-) diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh index c9cd5e30..8a2d4c02 100755 --- a/test/T357-index

Re: [PATCH] test: let the OS choose a port for smtp-dummy

2019-05-07 Thread Daniel Kahn Gillmor
On Tue 2019-05-07 07:20:49 -0300, David Bremner wrote: > This should avoid potential collisions if we start running multiple > smtp-dummy processes in parallel. This is excellent, simple, and clearly the right thing to do. I've reviewed it, and am running it on my own development branch with no

Re: [PATCH] travis: use ppa:notmuch/notmuch

2019-05-06 Thread Daniel Kahn Gillmor
On Sun 2019-05-05 19:48:47 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: >> This should allow us to continue to use the Travis continuous >> integration build on the release branch. > > And on master as well, I guess. yes, it should work on both master and re

Re: parallelize test suite

2019-05-06 Thread Daniel Kahn Gillmor
On Sun 2019-05-05 19:44:01 +0300, Tomi Ollila wrote: >> I actually think that CI and autobuilders *should* exercise the parallel >> tests, as annoying as that might be initially, because it seems likely >> to catch any other potential entanglements. > > ... after it has been proven a bit more to

Re: decryption error for signed+encrypted messages

2019-05-06 Thread Daniel Kahn Gillmor
blast from the past (cleaning up the bugtracker a bit). On Thu 2014-01-23 09:37:47 -0400, David Bremner wrote: > This bug seems to be (some time ago) marked fixed upstream > > https://bugzilla.gnome.org/show_bug.cgi?id=677088 This was definitely a bug in gmime, and it was resolved by gmime

Re: [PATCH] test: add configurable port to smtp-dummy

2019-05-06 Thread Daniel Kahn Gillmor
On Mon 2019-05-06 22:39:26 +0300, Tomi Ollila wrote: > we run smtp-dummy as: > >smtp_dummy_pid= >eval `$TEST_DIRECTORY/smtp-dummy --background sent_message` > > if process binds using port `0` system fill find available port for it... > > ... which smtp-dummy could return in a variable to

Re: [PATCH] tests: environment variable to specify that tests should be serialized

2019-05-06 Thread Daniel Kahn Gillmor
On Mon 2019-05-06 22:15:49 +0300, Tomi Ollila wrote: > While I like this parallelization option, and hope a version (could be even > David's smtp_dummy change) of it could be available in notmuch repository > as soon as possible, I would not like it being default -- just like make -J > is not

Re: [PATCH v2] configure: Ensure that GMime can extract session keys

2019-05-06 Thread Daniel Kahn Gillmor
On Mon 2019-05-06 16:16:55 -0400, Daniel Kahn Gillmor wrote: > GMime 3.0 and higher can extract session keys, but it will *not* > extract session keys if it was built with --disable-crypto, or if it > was built against GPGME version < 1.8.0. > > Notmuch currently expects to

[PATCH v2] configure: Ensure that GMime can extract session keys

2019-05-06 Thread Daniel Kahn Gillmor
sic-encrypted.eml used in it. Signed-off-by: Daniel Kahn Gillmor --- configure | 54 ++ 1 file changed, 54 insertions(+) diff --git a/configure b/configure index 9140026a..e157aadf 100755 --- a/configure +++ b/configure @@ -497,6 +497,60 @@ if pkg-con

[PATCH] util/crypto: improve comment

2019-05-06 Thread Daniel Kahn Gillmor
The comment line here lingers from when we were using some fancy version checking about session keys. Correct it to match the current state. Signed-off-by: Daniel Kahn Gillmor --- util/crypto.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/util/crypto.c b/util/crypto.c

[PATCH] configure: Ensure that GMime can extract session keys

2019-05-06 Thread Daniel Kahn Gillmor
sic-encrypted.eml used in it. Signed-off-by: Daniel Kahn Gillmor --- configure | 54 ++ 1 file changed, 54 insertions(+) diff --git a/configure b/configure index 9140026a..805292be 100755 --- a/configure +++ b/configure @@ -497,6 +497,60 @@ if pkg-con

[PATCH v3 1/3] emacs: test notmuch-show during message decryption

2019-05-06 Thread Daniel Kahn Gillmor
readdir()s the underlying maildir. Signed-off-by: Daniel Kahn Gillmor --- test/T357-index-decryption.sh | 13 + test/T450-emacs-show.sh | 15 ++ test/corpora/crypto/basic-encrypted.eml | 28 +++ .../notmuch-show-decrypted

Re: [PATCH v2 1/3] emacs: test notmuch-show during message decryption

2019-05-06 Thread Daniel Kahn Gillmor
On Fri 2019-05-03 17:59:49 +, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> We did not have a test showing what message decryption looks like >> within notmuch-emacs. This change gives us a baseline for future work >> on the notmuch-emacs interface. >

[PATCH] travis: use ppa:notmuch/notmuch

2019-05-05 Thread Daniel Kahn Gillmor
. This should allow us to continue to use the Travis continuous integration build on the release branch. Signed-off-by: Daniel Kahn Gillmor --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 90a1cc56..f9516bde 100644 --- a/.travis.yml +++ b/.travis.yml

Re: parallelize test suite

2019-05-05 Thread Daniel Kahn Gillmor
On Sat 2019-05-04 19:53:23 -0300, David Bremner wrote: > Last time we discussed parallel test running, there we concerns about > multiple versions of certain servers colliding with each other. This > still seems to be at least a theoretical issue with smtp-dummy, although > a glance suggests that

Re: [PATCH 1/2] test/raw: add some messages likely to be multiples of buffer size

2019-05-05 Thread Daniel Kahn Gillmor
On Sun 2019-05-05 07:49:16 -0300, David Bremner wrote: > My take away from that is that I should probably squash these two > commits into one. I've read this series and it seems reasonable to me. i agree that we should be testing messages smaller than 4096 octets, either by squashing the two

[PATCH 1/3] test/crypto: clarify the difference between ownertrust and validity

2019-05-04 Thread Daniel Kahn Gillmor
; validity, so the test is correct, but just misnamed. Signed-off-by: Daniel Kahn Gillmor --- test/T350-crypto.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh index 6045a7dc..bd369f8f 100755 --- a/test/T350-crypto.sh +++ b

[PATCH 3/3] test/crypto: add_gnupg_home should have ultimate trust on "its own" key

2019-05-04 Thread Daniel Kahn Gillmor
adjust the test suite to match this change, inverting the sense of one test: since the default is now that the user ID of the suite's own key is valid, we change the test to make sure that the user ID is not emitted when it is *not* valid. Signed-off-by: Daniel Kahn Gillmor --- test/T350-crypto.sh | 1

Re: parallelize test suite

2019-05-04 Thread Daniel Kahn Gillmor
On Sat 2019-05-04 20:57:43 +, Rollins, Jameson wrote: > This is a simple patch series that will run the entire test suite in > parallel if either the moreutils or GNU parallel utility is > available. On my 8-core machine the full test suite will now run in > under 20 seconds, which is a

[PATCH 2/3] test: simplify user ID handling

2019-05-04 Thread Daniel Kahn Gillmor
The user ID on the self-test is a little bit clunky-looking. It also may end up showing up elsewhere in the test suite. Centralizing the user ID in one place should make it easier to handle if it ever changes, and should make tests easier to read. Signed-off-by: Daniel Kahn Gillmor --- test

Cleaning up GnuPG User ID validity in the test suite

2019-05-04 Thread Daniel Kahn Gillmor
ate" ownertrust. This is a very nit-picky series without much of a functional difference, but it makes the test suite more conceptually coherent cryptographically, and should make future changes cleaner and more sensible. Signed-off-by: Daniel Kahn Gillmor ___

[PATCH v2 04/15] gmime-cleanup: remove obsolete gpg_path configuration option and crypto contexts

2019-05-02 Thread Daniel Kahn Gillmor
to drop. signed-off-by: Daniel Kahn Gillmor --- mime-node.c | 16 -- notmuch-client.h| 9 notmuch-config.c| 29 --- notmuch-reply.c | 4 -- notmuch-show.c | 4 -- notmuch.c | 8 --- test/T030-config.sh | 7 +-- test/T040-setup.sh

[PATCH v2 13/15] gmime-cleanup: pass NULL arguments explicitly where GMime 3.0 expects it

2019-05-02 Thread Daniel Kahn Gillmor
, and explicitly use the 3.0 argument lists. Signed-off-by: Daniel Kahn Gillmor --- lib/message-file.c | 4 ++-- lib/thread.cc | 2 +- mime-node.c| 2 +- notmuch-reply.c| 10 +- notmuch-search.c | 4 ++-- notmuch-show.c | 4 ++-- util/gmime-extra.h | 7 --- 7

[PATCH v2 07/15] gmime-cleanup: simplify T355-smime.sh

2019-05-02 Thread Daniel Kahn Gillmor
GMime 3.0 and later can handle User ID as expected. signed-off-by: Daniel Kahn Gillmor --- test/T355-smime.sh | 9 ++--- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/test/T355-smime.sh b/test/T355-smime.sh index be45e3b1..e410286b 100755 --- a/test/T355-smime.sh +++ b/test

[PATCH v2 03/15] gmime-cleanup: remove GMime 2.6 variant codeblocks

2019-05-02 Thread Daniel Kahn Gillmor
signed-off-by: Daniel Kahn Gillmor --- lib/message-file.c | 42 - notmuch-show.c | 21 --- util/gmime-extra.c | 93 -- util/gmime-extra.h | 12 -- 4 files changed, 168 deletions(-) diff --git a/lib/message-file.c b

[PATCH v2 02/15] gmime-cleanup: drop unused gmime 2.6 content_type from _index_encrypted_mime_part

2019-05-02 Thread Daniel Kahn Gillmor
In _index_mime_part, we don't need to extract the content-type from the part until just before we use it, so we also defer it lazily. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 22 +- 1 file changed, 1 insertion(+), 21 deletions(-) diff --git a/lib/index.cc b/lib

[PATCH v2 12/15] gmime-cleanup: use GMime 3.0 function names

2019-05-02 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 4 ++-- notmuch-reply.c| 8 notmuch-show.c | 16 util/gmime-extra.h | 4 4 files changed, 14 insertions(+), 18 deletions(-) diff --git a/lib/index.cc b/lib/index.cc index f21761d0..41822488

[PATCH v2 05/15] gmime-cleanup: always support session keys

2019-05-02 Thread Daniel Kahn Gillmor
Our minimum version of GMime 3.0 always supports good session key handling. signed-off-by: Daniel Kahn Gillmor --- configure | 9 - lib/built-with.c | 2 +- lib/index.cc | 4 +--- mime-node.c | 2 -- notmuch-show.c

[PATCH v2 01/15] build: drop support for gmime-2.6

2019-05-02 Thread Daniel Kahn Gillmor
From: David Bremner GMime 3.0 is over 2 years old now, and 2.6 has been deprecated in notmuch for about 1.5 years. Comments and documentation no longer need to refer to GMime 2.6, so clean them all up. Signed-off-by: Daniel Kahn Gillmor --- INSTALL | 2 +- configure | 24

[PATCH v2 06/15] gmime-cleanup: tests should only care about gmime 3

2019-05-02 Thread Daniel Kahn Gillmor
note that "notmuch-show for message with invalid From" is still broken in T310-emacs.sh. It would be good to debug what's going on there and try to get it fixed! signed-off-by: Daniel Kahn Gillmor --- test/T190-multipart.sh | 2 -- test/T310-emacs.sh | 2 +- test/T350-crypto.

[PATCH v2 11/15] gmime-cleanup: use GMime 3.0 data types

2019-05-02 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor --- notmuch-reply.c| 4 ++-- notmuch-show.c | 8 util/gmime-extra.h | 11 +++ 3 files changed, 9 insertions(+), 14 deletions(-) diff --git a/notmuch-reply.c b/notmuch-reply.c index fa91c5de..48fdbc92 100644 --- a/notmuch-reply.c

[PATCH v2 08/15] gmime-cleanup: drop g_mime_2_6_unref

2019-05-02 Thread Daniel Kahn Gillmor
signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 1 - notmuch-reply.c| 2 -- util/gmime-extra.h | 1 - 3 files changed, 4 deletions(-) diff --git a/lib/index.cc b/lib/index.cc index 6b6fbb8f..76830921 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -591,7 +591,6

[PATCH v2 14/15] gmime-cleanup: pass NULL as default GMimeParserOptions

2019-05-02 Thread Daniel Kahn Gillmor
eam commit d0ebdd2ea3e6fa635a2a551c846e9bc8b6040353 (which itself precedes GMime 3.0). Signed-off-by: Daniel Kahn Gillmor --- lib/message-file.c | 2 +- mime-node.c| 2 +- util/gmime-extra.h | 2 -- 3 files changed, 2 insertions(+), 4 deletions(-) diff --git a/lib/message-file.c b/lib/message-file.c inde

[PATCH v2 09/15] gmime-cleanup: drop all arguments unused in GMime 3

2019-05-02 Thread Daniel Kahn Gillmor
This means dropping GMimeCryptoContext and notmuch_config arguments. All the argument changes are to internal functions, so this is not an API or ABI break. We also get to drop the #define for g_mime_3_unused. signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 3 +-- mime-node.c

[PATCH v2 10/15] gmime-cleanup: drop unused gmime #defines and simplify g_mime_init ()

2019-05-02 Thread Daniel Kahn Gillmor
Several of these #defines were not actually used in the notmuch codebase any longer. And as of GMime 3.0, g_mime_init takes no arguments, so we can also drop the bogus RFC2047 argument that we were passing and then #defining away. signed-off-by: Daniel Kahn Gillmor --- lib/database.cc| 2

<    1   2   3   4   5   6   7   8   9   10   >