[PATCH 11/18] cli/new, insert, reindex: update documentation for --try-decrypt=auto

2017-10-25 Thread Daniel Kahn Gillmor
we also include --try-decrypt=auto in the tab completion. --- completion/notmuch-completion.bash | 6 +++--- doc/man1/notmuch-insert.rst| 16 ++-- doc/man1/notmuch-new.rst | 10 +++--- doc/man1/notmuch-reindex.rst | 23 ++- 4 files

Re: [PATCH 12/12] python: add try_decrypt argument to Database.index_file()

2017-10-24 Thread Daniel Kahn Gillmor
On Fri 2017-10-20 22:25:49 -0400, Daniel Kahn Gillmor wrote: > We adopt a pythonic idiom here with an optional argument, rather than > exposing the user to the C indexopts object directly. Please ignore this patch to the python bindings -- i plan to publish a new series shortly ("stas

Bug in notmuch-emacs: inconsistent rendering for messages with multiple From: headers

2017-10-23 Thread Daniel Kahn Gillmor
of search results. --- But the first message in notmuch-show doesn't show Bob at all: --- Alice <al...@fifthhorseman.net> (8 mins. ago) () Subject: a test from alice and bob To: Daniel Kahn Gillmor <d...@fifthhorseman.net> Date: Mon, 23 Oct 201

BUG: emacs: fcc duplicates messages with variant bodies due to mml security

2017-10-22 Thread Daniel Kahn Gillmor
I'm concerned that notmuch-emacs is causing variant messages of encrypted/signed messages to be generated. In particular, if notmuch-fcc-dirs is not nil, two variants of the message bodies will be created: one will be sent through "notmuch insert" and the other will be sent to sendmail. the mml

[PATCH] NEWS: document notmuch reindex

2017-10-22 Thread Daniel Kahn Gillmor
Describe the introduction of the "reindex" subcommand. This blurb acknowledges Subject: instability under reindexing when multiple copies exist (suggesting that this is something that needn't necessarily hold up a release). --- NEWS | 13 + 1 file changed, 13 insertions(+) diff

[PATCH] NEWS: cleartext indexing

2017-10-22 Thread Daniel Kahn Gillmor
give a short summary of cleartext indexing of encrypted e-mail messages. --- NEWS | 18 ++ 1 file changed, 18 insertions(+) diff --git a/NEWS b/NEWS index c2ebb644..895b7627 100644 --- a/NEWS +++ b/NEWS @@ -11,6 +11,24 @@ Out-of-tree builds has been dropped. The same can now

Re: web interface to notmuch

2017-10-22 Thread Daniel Kahn Gillmor
On Sat 2017-10-21 23:00:00 +0300, Jani Nikula wrote: > For the list archive, we could restrict to displaying text/plain only. and text/x-diff, surely :) But yeah, good point. Brian, what do you think about such a constraint? would that make your implementation safe enough to put on the public

nmbug.notmuchmail.org is now HTTPS and IPv6-enabled

2017-10-21 Thread Daniel Kahn Gillmor
hey all-- bremner has been maintaining the nmbug.notmuchmail.org webservice for the benefit of the community for quite a while now. I've just offered to help co-maintain, because i wanted to see the network service move to encrypted transport. With bremner's permission and assistance, that's

[PATCH 10/12] cli/insert: add --try-decrypt=(true|false)

2017-10-20 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion, and add more tests. --- completion/notmuch-completion.bash | 6 +++- doc/man1/notmuch-insert.rst| 14 ++ notmuch-insert.c

[PATCH 09/12] cli/new: add --try-decrypt=(true|false)

2017-10-20 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting during "notmuch new" on a per-invocation basis. We update the documentation and tab completion, and also add a test. --- completion/notmuch-completion.bash | 13 -- doc/man1/notmuch-new.rst | 12 + notmuch-new.c

[PATCH 07/12] cli: set up shared command-line arguments for indexing

2017-10-20 Thread Daniel Kahn Gillmor
We have an indexopts structure for manipulating indexing in different ways, but we also have three command-line invocations that can trigger indexing: new, insert, and reindex. This changeset prepares a common parser that these subcommands can share. Note: if the deprecated crypto.gpg_path

[PATCH 01/12] index: implement notmuch_indexopts_t with try_decrypt

2017-10-20 Thread Daniel Kahn Gillmor
This is currently mostly a wrapper around _notmuch_crypto_t that keeps its internals private and doesn't expose any of the GMime API. However, non-crypto indexing options might also be added later (e.g. filters or other transformations). --- lib/add-message.cc| 11 ++- lib/indexopts.c

[PATCH 06/12] config: define new option index.try_decrypt

2017-10-20 Thread Daniel Kahn Gillmor
By default, notmuch won't try to decrypt on indexing. With this patch, we make it possible to indicate a per-database preference using the config variable "index.try_decrypt", which by default will be false. At indexing time, the database needs some way to know its internal defaults for how to

[PATCH 08/12] tests: emacs_fcc_message: allow passing --arguments to notmuch new

2017-10-20 Thread Daniel Kahn Gillmor
Subsequent patches may want to send GNU-style --long-arguments to notmuch new in the test suite, in particular when invoking emacs_fcc_message. This changeset makes that possible. --- test/test-lib.sh | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git

[PATCH 05/12] config: test whether an item is stored in the database by name

2017-10-20 Thread Daniel Kahn Gillmor
QUERY_STRING was only used in two places, both to test whether a variable should be stored in (or retrieved from) the database. Since other configuration variables might be stored in the database in the future, consolidate that test into a single function. We also document that these

cleartext indexing, revision 7

2017-10-20 Thread Daniel Kahn Gillmor
What follows is the 7th revision of the cleartext indexing series. As mentioned in the previous discussions, there are a handful of minor cleanups (spelling, formatting). Beyond those, the main differences between this series and the 6th are: * changed names of variables that support

[PATCH 02/12] doc: add notmuch-properties(7)

2017-10-20 Thread Daniel Kahn Gillmor
We will want a user-facing place to record details about the use of notmuch properties shortly. This establishes a new manual page for that purpose. --- doc/conf.py | 4 +++ doc/index.rst | 1 + doc/man1/notmuch-dump.rst | 5 ++--

[PATCH 04/12] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-20 Thread Daniel Kahn Gillmor
If we see index options that ask us to decrypt when indexing a message, and we encounter an encrypted part, we'll try to descend into it. If we can decrypt, we add the property index.decryption=success. If we can't decrypt (or recognize the encrypted type of mail), we add the property

[PATCH 11/12] cli/reindex: add --try-decrypt=(true|false)

2017-10-20 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables

[PATCH 03/12] reindex: drop all properties named with prefix "index."

2017-10-20 Thread Daniel Kahn Gillmor
This allows us to create new properties that will be automatically set during indexing, and cleared during re-indexing, just by choice of property name. --- doc/man7/notmuch-properties.rst | 6 ++ lib/message.cc | 4 lib/notmuch.h | 3 +++ 3 files

[PATCH 12/12] python: add try_decrypt argument to Database.index_file()

2017-10-20 Thread Daniel Kahn Gillmor
We adopt a pythonic idiom here with an optional argument, rather than exposing the user to the C indexopts object directly. --- bindings/python/notmuch/database.py | 37 ++--- bindings/python/notmuch/globals.py | 5 + 2 files changed, 39 insertions(+), 3

Re: [PATCH v6 06/14] reindex: drop all properties named with prefix "index."

2017-10-19 Thread Daniel Kahn Gillmor
On Thu 2017-10-19 08:52:46 -0300, David Bremner wrote: > Daniel Kahn Gillmor <d...@fifthhorseman.net> writes: > >> +ret = notmuch_message_remove_all_properties_with_prefix (message, >> "index."); >> +if (ret) { >> +INTERNAL_ERROR ("

Re: [PATCH v6 06/14] reindex: drop all properties named with prefix "index."

2017-10-19 Thread Daniel Kahn Gillmor
On Thu 2017-10-19 08:54:25 -0300, David Bremner wrote: > Daniel Kahn Gillmor <d...@fifthhorseman.net> writes: > >> * >> + * Notmuch has some conventions about how certain properties are >> + * treated. Those conventions include: >> + * >> + *

Re: web interface to notmuch

2017-10-19 Thread Daniel Kahn Gillmor
On Thu 2017-10-19 16:00:33 -0400, Brian Sniffen wrote: > I don’t think they can be sanitized. Web tech moves so fast. well, there are at least a handful of python modules that claim to do some sort of sanitization. in debian alone, we have at least: python3-django-html-sanitizer

Re: web interface to notmuch

2017-10-19 Thread Daniel Kahn Gillmor
On Thu 2017-10-19 11:01:53 -0400, Brian Sniffen wrote: > I put together something like this, visible at > https://github.com/briansniffen/notmuch/tree/nmweb/contrib/notmuch-web > > It's not much of a service. I am pretty sure it is exploitable---that > content in text/html parts of messages can

Re: [PATCH v6 04/14] doc: add notmuch-properties(7)

2017-10-19 Thread Daniel Kahn Gillmor
On Tue 2017-10-17 15:09:58 -0400, Daniel Kahn Gillmor wrote: > diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst > new file mode 100644 [...] > +**notmuch** **count** **property:**<*key*>=<*value*> > + > +**notmuch** **search** *

[PATCH v6 03/14] index: implement notmuch_indexopts_t with try_decrypt

2017-10-17 Thread Daniel Kahn Gillmor
This is currently mostly a wrapper around _notmuch_crypto_t that keeps its internals private and doesn't expose any of the GMime API. However, non-crypto indexing options might also be added later (e.g. filters or other transformations). --- lib/add-message.cc| 11 ++- lib/indexopts.c

[PATCH v6 04/14] doc: add notmuch-properties(7)

2017-10-17 Thread Daniel Kahn Gillmor
We will want a user-facing place to record details about the use of notmuch properties shortly. This establishes a new manual page for that purpose. --- doc/conf.py | 4 +++ doc/index.rst | 1 + doc/man1/notmuch-dump.rst | 5 ++--

[PATCH v6 05/14] properties: add notmuch_message_remove_all_properties_with_prefix()

2017-10-17 Thread Daniel Kahn Gillmor
Subsequent patches will introduce a convention that properties whose name starts with "index." will be stripped (and possibly re-added) during re-indexing. This patch lays the groundwork for doing that. --- lib/message-property.cc | 18 -- lib/notmuch.h | 16

[PATCH v6 14/14] cli/reindex: add --try-decrypt=(true|false)

2017-10-17 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting on a per-run basis when invoking "notmuch reindex". This allows the possibility of (for example) an emacs keybinding that adds the cleartext of the currently shown decrypted message to the index, making it searchable in the future. It also enables

[PATCH v6 01/14] crypto: move into libnotmuch_util

2017-10-17 Thread Daniel Kahn Gillmor
This prepares us for using the crypto object in both libnotmuch and the client. --- Makefile.local| 1 - notmuch-client.h | 22 +- util/Makefile.local | 2 +- crypto.c => util/crypto.c | 7 ++- util/crypto.h | 28

[PATCH v6 13/14] cli/insert: add --try-decrypt=(true|false)

2017-10-17 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting on a per-message basis when invoking "notmuch insert". We also update the documentation and tab completion. --- completion/notmuch-completion.bash | 6 +- doc/man1/notmuch-insert.rst| 14 ++ notmuch-insert.c

[PATCH v6 12/14] cli/new: add --try-decrypt=(true|false)

2017-10-17 Thread Daniel Kahn Gillmor
Enable override of the index.try_decrypt setting during "notmuch new" on a per-invocation basis. We update the documentation and tab completion, and also add a test. --- completion/notmuch-completion.bash | 13 -- doc/man1/notmuch-new.rst | 12 + notmuch-new.c

[PATCH v6 02/14] crypto: make shared crypto code behave library-like

2017-10-17 Thread Daniel Kahn Gillmor
If we're going to reuse the crypto code across both the library and the client, then it needs to report error states properly and not write to stderr. --- lib/database.cc | 6 lib/notmuch.h | 17 +++ mime-node.c | 7 - util/crypto.c | 92

[PATCH v6 06/14] reindex: drop all properties named with prefix "index."

2017-10-17 Thread Daniel Kahn Gillmor
This allows us to create new properties that will be automatically set during indexing, and cleared during re-indexing, just by choice of property name. --- doc/man7/notmuch-properties.rst | 6 ++ lib/message.cc | 6 ++ lib/notmuch.h | 6 ++ 3 files

[PATCH v6 07/14] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-17 Thread Daniel Kahn Gillmor
If we see index options that ask us to decrypt when indexing a message, and we encounter an encrypted part, we'll try to descend into it. If we can decrypt, we add the property index.decryption=success. If we can't decrypt (or recognize the encrypted type of mail), we add the property

cleartext indexing, revision 6

2017-10-17 Thread Daniel Kahn Gillmor
This is the 6th revision of the cleartext indexing series. It differs from the 5th mainly in minor edits: fixing typos and improving comments, documentation, and commit messages. It also has the following (slightly more) significant changes: * improvement of the test suite to permit passing

[PATCH v6 08/14] config: store index.try_default in the database.

2017-10-17 Thread Daniel Kahn Gillmor
At indexing time, the database needs some way to know its internal defaults for how to index encrypted parts. It shouldn't be contingent on an external config file (since that can't be retrieved from the database object itself). This behaves similarly to the query.* configurations, which are

[PATCH v6 10/14] cli: set up shared command-line arguments for indexing

2017-10-17 Thread Daniel Kahn Gillmor
We have an indexopts structure for manipulating indexing in different ways, but we also have three command-line invocations that can trigger indexing: new, insert, and reindex. This changeset prepares a common parser that these subcommands can share. Note: if the deprecated crypto.gpg_path

[PATCH v6 11/14] tests: emacs_fcc_message: allow passing --arguments to notmuch new

2017-10-17 Thread Daniel Kahn Gillmor
Subsequent patches may want to send GNU-style --long-arguments to notmuch new in the test suite, in particular when invoking emacs_fcc_message. This changeset makes that possible. --- test/test-lib.sh | 11 ++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git

[PATCH v6 09/14] config: define new option index.try_decrypt

2017-10-17 Thread Daniel Kahn Gillmor
By default, notmuch won't try to decrypt on indexing. With this patch, we make it possible to indicate a per-database preference using the config variable "index.try_decrypt", which by default will be false. --- doc/man1/notmuch-config.rst | 12 doc/man7/notmuch-properties.rst |

Re: [PATCH] cli/crypto: fix segfault on failed gmime2 crypto context creation

2017-10-17 Thread Daniel Kahn Gillmor
On Mon 2017-10-16 18:40:44 +0300, Jani Nikula wrote: > Commit 1fdc08d0ffab ("cli/crypto: treat failure to create a crypto > context as fatal.") started treating crypto context creation failures > "as fatal", returning NULL from _mime_node_create(). > > Unfortunately, we do not have NULL checks for

Re: [PATCH] lib: add support for thread: queries

2017-10-17 Thread Daniel Kahn Gillmor
On Mon 2017-10-16 22:07:54 +0300, Jani Nikula wrote: > Add support for querying threads using message-ids in addition to > thread-ids. The main benefit is that thread queries via message-ids > are portable across databases, re-indexing, and thread joining, while > thread ids can be somewhat

[PATCH 2/2] doc: make SEE ALSO references one-per-line

2017-10-17 Thread Daniel Kahn Gillmor
This will make future diffs cleaner, make it easier to keep them alphabetical, and make it easier to scan and search the documentation sources. --- doc/man1/notmuch-address.rst | 16 doc/man1/notmuch-compact.rst | 16 doc/man1/notmuch-config.rst |

[PATCH 1/2] doc: fix documentation typos and copy/paste-os

2017-10-17 Thread Daniel Kahn Gillmor
--- doc/man1/notmuch-dump.rst| 2 +- doc/man1/notmuch-restore.rst | 8 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst index f3f2b394..1e4468e6 100644 --- a/doc/man1/notmuch-dump.rst +++ b/doc/man1/notmuch-dump.rst

[PATCH v5 06/12] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-15 Thread Daniel Kahn Gillmor
If we see index options that ask us to decrypt when indexing a message, and we encounter an encrypted part, we'll try to descend into it. If we can decrypt, we add the property index.decryption=success. If we can't decrypt (or recognize the encrypted type of mail), we add the property

[PATCH v5 10/12] cli/new: add --try-decrypt=(true|false)

2017-10-15 Thread Daniel Kahn Gillmor
Try to decrypt any encrypted parts of newly-discovered messages while indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option is

[PATCH v5 11/12] cli/insert: add --try-decrypt=(true|false)

2017-10-15 Thread Daniel Kahn Gillmor
Allow an incoming message to be delivered while indexing the cleartext, on a per-message basis. This requires the secret keys for the message to be available. For the moment, the most functional approach is to ensure that gpg-agent is running and knows about any secret keys that might be useful

Re: [PATCH v3 11/15] config: indexing defaults will be stored in the database.

2017-10-15 Thread Daniel Kahn Gillmor
On Sat 2017-10-14 15:08:51 -0300, David Bremner wrote: > Daniel Kahn Gillmor <d...@fifthhorseman.net> writes: > >> static int >> _print_db_config(notmuch_config_t *config, const char *name) >> @@ -859,6 +860,8 @@ notmuch_config_command_get (notmuch_co

cleartext indexing version 5

2017-10-15 Thread Daniel Kahn Gillmor
Thanks to everyone for their reviews of this series that provides indexing of the cleartext of encrypted messages (and thanks to bremner for already pushing some of the less controversial patches of the previous revisions of this series). I hope you'll agree that your suggestions have improved

[PATCH v5 04/12] properties: add notmuch_message_remove_all_properties_with_prefix()

2017-10-15 Thread Daniel Kahn Gillmor
Subsequent patches will introduce a convention that properties whose name starts with "index." will be stripped (and possibly re-added) during re-indexing. This patch lays the groundwork for doing that. --- lib/message-property.cc | 18 -- lib/notmuch.h | 16

[PATCH v5 05/12] reindex: drop all properties named with prefix "index."

2017-10-15 Thread Daniel Kahn Gillmor
This allows us to create new properties that will be automatically set during indexing, and cleared during re-indexing, just by choice of property name. --- lib/message.cc | 6 ++ lib/notmuch.h | 6 ++ 2 files changed, 12 insertions(+) diff --git a/lib/message.cc b/lib/message.cc index

[PATCH v5 01/12] crypto: move into libnotmuch_util

2017-10-15 Thread Daniel Kahn Gillmor
This prepares us for using the crypto object in both libnotmuch and the client. --- Makefile.local| 1 - notmuch-client.h | 22 +- util/Makefile.local | 2 +- crypto.c => util/crypto.c | 7 ++- util/crypto.h | 28

[PATCH v5 09/12] cli: set up shared command-line arguments for indexing

2017-10-15 Thread Daniel Kahn Gillmor
We have an indexopts structure for manipulating indexing in different ways, but we also have three command-line invocations that can trigger indexing: new, insert, and reindex. This changeset prepares a common parser that these subcommands can share. At the moment, it's just --try-decrypt, but

[PATCH v5 02/12] crypto: make shared crypto code behave library-like

2017-10-15 Thread Daniel Kahn Gillmor
If we're going to reuse the crypto code across both the library and the client, then it needs to report error states properly and not write to stderr. --- lib/database.cc | 6 lib/notmuch.h | 17 +++ mime-node.c | 7 - util/crypto.c | 92

[PATCH v5 12/12] cli/reindex: add --try-decrypt=(true|false)

2017-10-15 Thread Daniel Kahn Gillmor
Try to decrypt any encrypted parts of newly-discovered messages while re-indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option

[PATCH v5 07/12] config: indexing defaults will be stored in the database.

2017-10-15 Thread Daniel Kahn Gillmor
At indexing time, the database needs to know its internal defaults. It shouldn't be contingent on an external config file (since that can't be retrieved from the database object itself). This behaves the same as the query.* configurations, which are also stored in the database itself, so we're

[PATCH v5 08/12] config: define new option index.try_decrypt

2017-10-15 Thread Daniel Kahn Gillmor
By default, notmuch won't try to decrypt on indexing. With this patch, we make it possible to indicate a per-database preference using the config variable "index.try_decrypt", which by default will be false. --- doc/man1/notmuch-config.rst | 12 lib/indexopts.c | 18

[PATCH v5 03/12] index: implement notmuch_indexopts_t with try_decrypt

2017-10-15 Thread Daniel Kahn Gillmor
This is currently mostly a wrapper around _notmuch_crypto_t that keeps its internals private and doesn't expose any of the GMime API. However, non-crypto indexing options might also be added later (e.g. filters or other transformations). --- lib/add-message.cc| 11 ++- lib/indexopts.c

Re: [PATCH v3 10/15] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-13 Thread Daniel Kahn Gillmor
On Thu 2017-10-12 22:08:26 -0300, David Bremner wrote: > The second _notmuch_database_log will override the first > here. You can use _notmuch_database_log_append if you don't want to > clear the existing log (e.g. at least for the second _log > here). good catch, thanks! I'll include that in my

Re: [PATCH v4 08/15] index: implement notmuch_indexopts_t with try_decrypt

2017-10-12 Thread Daniel Kahn Gillmor
On Thu 2017-10-12 08:18:15 -0300, David Bremner wrote: > Daniel Kahn Gillmor <d...@fifthhorseman.net> writes: > >> >> +if (!indexopts) >> +indexopts = def_indexopts = notmuch_database_get_default_indexopts >> (notmuch); >> + > &

Re: [PATCH v3 04/15] crypto: move into libutil

2017-10-12 Thread Daniel Kahn Gillmor
Hi Bremner-- Thanks for the review! On Thu 2017-10-12 07:54:33 -0300, David Bremner wrote: > Daniel Kahn Gillmor <d...@fifthhorseman.net> writes: > >> This prepares us for using the crypto object in both the library and >> the client. > > I think we could be

Re: Public notmuch.mbox missing some messages which are tagged in nmbug

2017-10-11 Thread Daniel Kahn Gillmor
On Wed 2017-10-11 08:04:05 -0300, David Bremner wrote: > "W. Trevor King" writes: > >> If somebody has copies of those messages, can they push them up into >> the mbox? Maybe they went to the freelists list [3]? Maybe they were >> off-list? Maybe they just missed inclusion in

[PATCH v4 08/15] index: implement notmuch_indexopts_t with try_decrypt

2017-10-10 Thread Daniel Kahn Gillmor
This is currently mostly a wrapper around _notmuch_crypto_t that keeps its internals private and doesn't expose any of the GMime API. However, non-crypto indexing options might also be added later (e.g. filters or other transformations). --- lib/add-message.cc| 9 - lib/indexopts.c

avoid double typedef

2017-10-10 Thread Daniel Kahn Gillmor
On IRC, Domo pointed out that older gcc complains when a typedef gets repeated. So i'm updating patch 8 in this series to avoid double typedefs. The rest of the series should be unchanged, so i'm avoiding re-flooding the lst with them, but i'm happy to send along a full round of v4 if folks

Re: [PATCH 0/3] nmbug:

2017-10-10 Thread Daniel Kahn Gillmor
On Tue 2017-10-10 15:49:48 -0700, W. Trevor King wrote: > Two changes and a bugfix spun off from today's IRC disussion. This series looks reasonable to me, from what little i understand of nmbug. Thanks for proposing the changes, Trevor. --dkg signature.asc Description: PGP signature

Re: cleartext indexing, round 3

2017-10-10 Thread Daniel Kahn Gillmor
On Tue 2017-10-10 08:50:17 -0700, Jameson Graef Rollins wrote: > On Tue, Oct 10 2017, Daniel Kahn Gillmor <d...@fifthhorseman.net> wrote: >> I've also pushed this series to the "cleartext-indexing" branch at >> https://gitlab.com/dkg/notmuch for tho

[PATCH v3 06/15] crypto: make shared crypto code behave library-like

2017-10-09 Thread Daniel Kahn Gillmor
If we're going to reuse the crypto code across both the library and the client, then it needs to report error states properly and not write to stderr. --- lib/database.cc | 6 lib/notmuch.h | 17 +++ mime-node.c | 7 - util/crypto.c | 92

[PATCH v3 15/15] cli/reindex: add --try-decrypt=(true|false)

2017-10-09 Thread Daniel Kahn Gillmor
Try to decrypt any encrypted parts of newly-discovered messages while re-indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option

[PATCH v3 10/15] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-09 Thread Daniel Kahn Gillmor
If we see index options that ask us to decrypt when indexing a message, and we encounter an encrypted part, we'll try to descend into it. If we can decrypt, we add the property index-decryption=success. If we can't decrypt (or recognize the encrypted type of mail), we add the property

[PATCH v3 11/15] config: indexing defaults will be stored in the database.

2017-10-09 Thread Daniel Kahn Gillmor
At indexing time, the database needs to know its internal defaults. It shouldn't be contingent on an external config file (since that can't be retrieved from the database object itself). This behaves the same as the query.* configurations, which are also stored in the database itself, so we're

[PATCH v3 07/15] tests: prepare for more crypto tests (using add_gnupg_home)

2017-10-09 Thread Daniel Kahn Gillmor
Move add_gnupg_home to test-lib.sh to prepare it for reuse. --- test/T350-crypto.sh | 17 - test/test-lib.sh| 17 + 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh index 53bf4113..401a22c9 100755 ---

[PATCH v3 04/15] crypto: move into libutil

2017-10-09 Thread Daniel Kahn Gillmor
This prepares us for using the crypto object in both the library and the client. --- Makefile.local| 1 - notmuch-client.h | 22 +- util/Makefile.local | 2 +- crypto.c => util/crypto.c | 6 +- util/crypto.h | 28

[PATCH v3 02/15] crypto: drop pretense of notmuch_crypto_context_t

2017-10-09 Thread Daniel Kahn Gillmor
notmuch_crypto_context_t was introduced (i think) as some sort of abstraction layer to make notmuch somewhat independent of GMime. But it isn't even useful for GMime 3.0 or later -- we can drop the pretense that it's some sort of abstraction in this case, and just call it what it is,

cleartext indexing, round 3

2017-10-09 Thread Daniel Kahn Gillmor
What follows is the third round of the latest revision of cleartext indexing patches. This series is rebased against current master, with the improved option-handling code, and takes into consideration the review that Jani did on round 2 of this series. I welcome additional review and feedback.

[PATCH v3 09/15] gmime-extra: drop compat layer for g_mime_multipart_encrypted_decrypt

2017-10-09 Thread Daniel Kahn Gillmor
In practice, we're going to see this function invoked differently depending on which gmime we build against. The compatibility layer forces our code into the lowest-common-denominator -- unable to make use of new features even when built against a newer version. Dropping the compatibility layer

[PATCH v3 14/15] cli/insert: add --try-decrypt=(true|false)

2017-10-09 Thread Daniel Kahn Gillmor
Allow an incoming message to be delivered while indexing the cleartext, on a per-message basis. This requires the secret keys for the message to be available. For the moment, the most functional approach is to ensure that gpg-agent is running and knows about any secret keys that might be useful

[PATCH v3 01/15] crypto: rename notmuch_crypto_t to _notmuch_crypto_t

2017-10-09 Thread Daniel Kahn Gillmor
The notmuch_crypto_t struct isn't used externally, and we have no plans to explicitly export it. Prefix its name (and associated functions) with _ to make that intent clear. --- crypto.c | 12 ++-- mime-node.c | 6 +++--- notmuch-client.h | 12 ++-- notmuch-reply.c

Re: [PATCH v2 04/10] index: implement notmuch_indexopts_t with try_decrypt

2017-10-09 Thread Daniel Kahn Gillmor
On Sat 2017-09-23 19:10:18 +0300, Jani Nikula wrote: >> --- a/lib/indexopts.c >> +++ b/lib/indexopts.c >> @@ -21,9 +21,27 @@ >> #include "notmuch-private.h" >> >> notmuch_indexopts_t * >> -notmuch_database_get_default_indexopts (notmuch_database_t unused (*db)) >>

Re: [PATCH v2 05/10] crypto: index encrypted parts when indexopts try_decrypt is set.

2017-10-09 Thread Daniel Kahn Gillmor
On Sat 2017-09-23 19:05:40 +0300, Jani Nikula wrote: > I'd like this #if block to be abstracted to separate functions for gmime > 2 vs. 3. Adding conditional compilation within functions is ugly and > hard to track for both branches. That abstraction has already happened -- it's in util/crypto.c;

[PATCH v3 05/15] gmime-extra: remove duplicate GMimeAddressType typedef

2017-10-09 Thread Daniel Kahn Gillmor
"typedef GMimeAddressType GMimeRecipientType" is already present further down in the compatibility wrapper (with other typedefs). We don't need it twice. --- util/gmime-extra.h | 1 - 1 file changed, 1 deletion(-) diff --git a/util/gmime-extra.h b/util/gmime-extra.h index de275bc1..e060bcc2

[PATCH v3 12/15] config: define new option index.try_decrypt

2017-10-09 Thread Daniel Kahn Gillmor
By default, notmuch won't try to decrypt on indexing. With this patch, we make it possible to indicate a per-database preference using the config variable "index.try_decrypt", which by default will be false. --- doc/man1/notmuch-config.rst | 12 lib/indexopts.c | 18

[PATCH v3 03/15] crypto: _notmuch_crypto_cleanup should return void

2017-10-09 Thread Daniel Kahn Gillmor
There's no chance that _notmuch_crypto_cleanup() will ever return anything other than 0, and no one ever checks its return value anyway. So make it return void instead of int. --- crypto.c | 7 ++- notmuch-client.h | 2 +- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git

[PATCH v3 13/15] cli/new: add --try-decrypt=(true|false)

2017-10-09 Thread Daniel Kahn Gillmor
Try to decrypt any encrypted parts of newly-discovered messages while indexing them. The cleartext of any successfully-decrypted messages will be indexed, with tags applied in the same form as from notmuch insert --try-decrypt=true. Note: if the deprecated crypto.gpg_path configuration option is

[PATCH v3 08/15] index: implement notmuch_indexopts_t with try_decrypt

2017-10-09 Thread Daniel Kahn Gillmor
This is currently mostly a wrapper around _notmuch_crypto_t that keeps its internals private and doesn't expose any of the GMime API. However, non-crypto indexing options might also be added later (e.g. filters or other transformations). --- lib/add-message.cc| 9 - lib/indexopts.c

Re: [PATCH v2 02/10] crypto: make shared crypto code behave library-like

2017-10-09 Thread Daniel Kahn Gillmor
On Sat 2017-09-23 18:36:18 +0300, Jani Nikula wrote: >> if (!cryptoctx) >> return NULL; > > I guess this will work because we initialize cryptoctx to NULL, but if > we return the status, I think we should trust status == success means > cryptoctx is fine, and otherwise we shouldn't

Re: [PATCH 1/2] cli: convert notmuch_bool_t to stdbool

2017-10-09 Thread Daniel Kahn Gillmor
On Sat 2017-10-07 11:44:04 +0300, Jani Nikula wrote: > C99 stdbool turned 18 this year. There really is no reason to use our > own, except in the library interface for backward > compatibility. Convert the cli and test binaries to stdbool. +1 LGTM. I'd also be interested in contemplating a

Re: [PATCH] doc: fix typos

2017-09-27 Thread Daniel Kahn Gillmor
On Tue 2017-09-26 19:23:09 +0200, Jakub Wilk wrote: > --- > doc/man7/notmuch-search-terms.rst | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/doc/man7/notmuch-search-terms.rst > b/doc/man7/notmuch-search-terms.rst > index dd76972e..b27f31f7 100644 > ---

Re: [PATCH 0/9] argument parsing fixes and improvements

2017-09-25 Thread Daniel Kahn Gillmor
On Mon 2017-09-25 08:34:13 -0300, David Bremner wrote: > I think there is two different discussions one could be having here; one > about the UI, the other about the implimentation. > > From the UI point of view, Are you using the term "UI" to mean "API" here? i tend to think of "UI" as the CLI

[PATCH] fix reference to notmuch_message_get_properties

2017-09-23 Thread Daniel Kahn Gillmor
--- lib/notmuch.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/notmuch.h b/lib/notmuch.h index 34bf5899..cbde6a93 100644 --- a/lib/notmuch.h +++ b/lib/notmuch.h @@ -1873,7 +1873,7 @@ notmuch_message_get_properties (notmuch_message_t *message, const char *key, not *

Re: [PATCH 0/9] argument parsing fixes and improvements

2017-09-21 Thread Daniel Kahn Gillmor
On Thu 2017-09-21 20:07:30 +0300, Jani Nikula wrote: > So I think I'd prefer either strict booleans where the default is one or > the other, or an explicit tristate true|false|default. In your case, > perhaps --try-decrypt=(true|false|database), where the default can be > requested and clearly

Re: [RFC PATCH] emacs: require minimum version to be 24.1 c-b c-b c-b c-b c-t c-e

2017-09-20 Thread Daniel Kahn Gillmor
On Wed 2017-09-20 13:18:44 -0300, David Bremner wrote: > Tomi Ollila writes: >> >> +(defvar notmuch--minimal-version "42.1") ;; almost indistinguishable from >> magit > > typo? I think that's what the "c-b c-b c-b c-b c-t c-e" in the Subject: is about. if you do that, you

Re: [PATCH 0/9] argument parsing fixes and improvements

2017-09-20 Thread Daniel Kahn Gillmor
Hi Jani-- On Wed 2017-09-20 12:16:33 +0300, Jani Nikula wrote: > Those are good questions, and up for debate. I was more interested in > getting the mask out behaviour for keyword flag arguments, and the > boolean part came practically free. Or, there was certain synergy in > having them behave

Re: [PATCH 0/9] argument parsing fixes and improvements

2017-09-19 Thread Daniel Kahn Gillmor
On Tue 2017-09-19 23:39:20 +0300, Jani Nikula wrote: > I had some other things in mind, but ended up with this prep series > instead... The main thing is patch 6 adding --no-argument style negating > arguments for boolean and keyword flag args. The rest is mostly just > refactoring and tests to

Re: [PATCH v2 2/3] database: add n_d_index_file (deprecates n_d_add_message)

2017-09-17 Thread Daniel Kahn Gillmor
On Wed 2017-08-23 08:01:22 -0300, David Bremner wrote: > Pushed 2 and 3, but with both amended to fix whitespace errors. Please > _do_ adjust your settings, this is not a test ;). thanks for this push, bremner. if there are settings that need to be adjusted, i believe .dir-locals.el is the place

Re: [PATCH 4/4] lib: index the content-type of the parts of encrypted messages

2017-09-17 Thread Daniel Kahn Gillmor
this 4-patch series LGTM. Thanks, Jani! --dkg ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: index the content-type of the parts of encrypted messages

2017-09-15 Thread Daniel Kahn Gillmor
On Fri 2017-09-15 10:48:08 +0300, Jani Nikula wrote: > Nitpick, the extra braces aren't needed here. But the patch does what > it says on the box. Thanks for the review! I prefer to keep the braces, i think they make it clearer what's happening. If whoever's merging prefers to remove the

[PATCH v2 01/10] crypto: Move crypto.c into libutil

2017-09-14 Thread Daniel Kahn Gillmor
ypto.c b/util/crypto.c similarity index 79% rename from crypto.c rename to util/crypto.c index cc45b885..97e8c8f4 100644 --- a/crypto.c +++ b/util/crypto.c @@ -16,18 +16,26 @@ * along with this program. If not, see https://www.gnu.org/licenses/ . * * Authors: Jameson Rollins <jroll...@fines

[PATCH v2 09/10] cli/insert: add --try-decrypt=(true|false)

2017-09-14 Thread Daniel Kahn Gillmor
Allow an incoming message to be delivered while indexing the cleartext, on a per-message basis. This requires the secret keys for the message to be available. For the moment, the most functional approach is to ensure that gpg-agent is running and knows about any secret keys that might be useful

[PATCH v2 02/10] crypto: make shared crypto code behave library-like

2017-09-14 Thread Daniel Kahn Gillmor
If we're going to reuse the crypto code across both the library and the client, then it needs to report error states properly and not write to stderr. --- lib/database.cc | 6 lib/notmuch.h | 17 +++ mime-node.c | 7 - util/crypto.c | 89

<    5   6   7   8   9   10   11   12   13   14   >