from:"Daniel Kahn Gillmor"

Encourage explicit arguments for --decrypt in "show" and "reply"

2017-12-11 Thread Daniel Kahn Gillmor

The notmuch indexing subcommands ("new", "insert", and "reindex") now
have a --decrypt option that takes an argument (the decryption
policy), since the session-keys patches have landed.

But the viewing subcommands ("show" and "reply") have their
traditional --decrypt option that (as a boolean) need not take an
argument, having --decrypt not present means something different from
either --decrypt=true or --decrypt=false.

This series allows the user to explicitly choose --decrypt=auto for
the viewing subcommands, while allowing people to use the
argument-free form (as an alias for --decrypt=true), but warns the
user to encourage them to switch to using an explicit argument
instead.

This is useful normalizing work for the interface, so it's worthwhile
on its own. It is also necessary preparation in the event that we
decide we want to:

 * set up a notmuch configuration option that changes the default for
   --decrypt for the viewing subcommands
 
 * allow "notmuch show" to actually index encrypted messages upon
   their first encounter (e.g., via a new decryption policy, which
   i'll propose separately)

As always, review and feedback welcome!

   --dkg


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor

We also expand tab completion for it, and update T357 to match.

Make use of the bool-to-keyword backward-compatibility feature.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-show.rst  | 37 +
 notmuch-show.c | 27 +--
 test/T357-index-decryption.sh  |  6 +++---
 4 files changed, 42 insertions(+), 34 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index fb093de8..4ab2e5f6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,10 +517,14 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body|--decrypt)
+   --exclude|--body)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
+--decrypt)
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 64caa7a6..7d2b38cb 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -115,22 +115,27 @@ Supported options for **show** include
 supported with --format=json and --format=sexp), and the
 multipart/signed part will be replaced by the signed data.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
-
-Implies --verify.
+``--decrypt=(false|auto|true)``
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e. "multipart/encrypted" parts). Status of
+the decryption will be reported (currently only supported
+with --format=json and --format=sexp) and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will fail.
+
+Note: ``true`` implies --verify.
+
+Default: ``auto``
 
 ``--exclude=(true|false)``
 Specify whether to omit threads only matching
diff --git a/notmuch-show.c b/notmuch-show.c
index d5adc370..ddd3c8c5 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1085,8 +1085,6 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -1101,7 +1099,12 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
{ .opt_bool = ¶ms.entire_thread, .name = "entire-thread",
  .present = &entire_thread_set },
{ .opt_int = ¶ms.part, .name = "part" },
-   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
+   { .opt_keyword = (int*)(¶ms.crypto.decrypt), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_bool = ¶ms.crypto.verify, .name = "verify" },
{ .opt_bool = ¶ms.output_body, .name = "body" },
{ .opt_bool = ¶ms.include_html, .name = "include-html" },
@@ -1115,16 +1118,9 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt_set) {
-   if (decrypt) {
-   /* we do not need or want to ask for session keys */
-   params.crypto.decrypt = NOTMUCH_DECRYPT_NOSTASH;
-   /* decryption implies verification */
-   params.crypto.verify = true;
-   } else {
-   params.crypto.decrypt = NOTMUCH_DECRYP

[PATCH 3/3] cli/reply: make --decrypt take a keyword

2017-12-11 Thread Daniel Kahn Gillmor

This brings the --decrypt argument to "notmuch reply" into line with
the other --decrypt arguments (in "show", "new", "insert", and
"reindex").  This patch is really just about bringing consistency to
the user interface.
---
 completion/notmuch-completion.bash |  2 +-
 doc/man1/notmuch-reply.rst | 34 --
 notmuch-reply.c| 11 ++-
 3 files changed, 27 insertions(+), 20 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 4ab2e5f6..a24b8a08 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -351,7 +351,7 @@ _notmuch_reply()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index ede77930..1b62e075 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -72,20 +72,26 @@ Supported options for **reply** include
 in this order, and copy values from the first that contains
 something other than only the user's addresses.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will likely fail.
+``--decrypt=(false|auto|true)``
+
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e., "multipart/encrypted" parts). Status
+of the decryption will be reported (currently only supported
+with --format=json and --format=sexp), and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's secret keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will likely fail.
+
+Default: ``auto``
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 5cdf642b..75cf7ecb 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -704,8 +704,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -719,7 +717,12 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
+   { .opt_keyword = (int*)(¶ms.crypto.decrypt), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -729,8 +732,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt_set)
-   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_NOSTASH : 
NOTMUCH_DECRYPT_FALSE;
 
 notmuch_exit_if_unsupported_format ();
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/3] cli: some keyword options can be supplied with no argument

2017-12-11 Thread Daniel Kahn Gillmor

We might change some notmuch command line tools that used to be
booleans into keyword arguments.

In that case, there are some legacy tools that will expect to be able
to do "notmuch foo --bar" instead of "notmuch foo --bar=baz".

This patch makes it possible to support that older API, while
providing a warning and an encouragement to upgrade.
---
 command-line-arguments.c  | 59 +++
 command-line-arguments.h  |  4 +++
 test/T410-argument-parsing.sh | 24 ++
 test/arg-test.c   | 12 -
 4 files changed, 82 insertions(+), 17 deletions(-)

diff --git a/command-line-arguments.c b/command-line-arguments.c
index db73ca5e..d0e21693 100644
--- a/command-line-arguments.c
+++ b/command-line-arguments.c
@@ -4,13 +4,19 @@
 #include "error_util.h"
 #include "command-line-arguments.h"
 
+typedef enum {
+OPT_FAILED, /* false */
+OPT_OK, /* good */
+OPT_GIVEBACK, /* pop one of the arguments you thought you were getting off 
the stack */
+} opt_handled;
+
 /*
   Search the array of keywords for a given argument, assigning the
   output variable to the corresponding value.  Return false if nothing
   matches.
 */
 
-static bool
+static opt_handled
 _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
 
 const notmuch_keyword_t *keywords;
@@ -29,16 +35,32 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
else
*arg_desc->opt_keyword = keywords->value;
 
-   return true;
+   return OPT_OK;
 }
+
+if (arg_desc->opt_keyword && arg_desc->keyword_no_arg_value && next != ':' 
&& next != '=') {
+   for (keywords = arg_desc->keywords; keywords->name; keywords++) {
+   if (strcmp (arg_desc->keyword_no_arg_value, keywords->name) != 0)
+   continue;
+
+   *arg_desc->opt_keyword = keywords->value;
+   fprintf (stderr, "Warning: No known keyword option given for 
\"%s\", choosing value \"%s\"."
+"  Please specify the argument explicitly!\n", 
arg_desc->name, arg_desc->keyword_no_arg_value);
+
+   return OPT_GIVEBACK;
+   }
+   fprintf (stderr, "No matching keyword for option \"%s\" and default 
value \"%s\" is invalid.\n", arg_str, arg_desc->name);
+   return OPT_FAILED;
+}
+
 if (next != '\0')
fprintf (stderr, "Unknown keyword argument \"%s\" for option 
\"%s\".\n", arg_str, arg_desc->name);
 else
fprintf (stderr, "Option \"%s\" needs a keyword argument.\n", 
arg_desc->name);
-return false;
+return OPT_FAILED;
 }
 
-static bool
+static opt_handled
 _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
 bool value;
 
@@ -48,45 +70,45 @@ _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
value = false;
 } else {
fprintf (stderr, "Unknown argument \"%s\" for (boolean) option 
\"%s\".\n", arg_str, arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 
 *arg_desc->opt_bool = value;
 
-return true;
+return OPT_OK;
 }
 
-static bool
+static opt_handled
 _process_int_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 char *endptr;
 if (next == '\0' || arg_str[0] == '\0') {
fprintf (stderr, "Option \"%s\" needs an integer argument.\n", 
arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 
 *arg_desc->opt_int = strtol (arg_str, &endptr, 10);
 if (*endptr == '\0')
-   return true;
+   return OPT_OK;
 
 fprintf (stderr, "Unable to parse argument \"%s\" for option \"%s\" as an 
integer.\n",
 arg_str, arg_desc->name);
-return false;
+return OPT_FAILED;
 }
 
-static bool
+static opt_handled
 _process_string_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 if (next == '\0') {
fprintf (stderr, "Option \"%s\" needs a string argument.\n", 
arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 if (arg_str[0] == '\0' && ! arg_desc->allow_empty) {
fprintf (stderr, "String argument for option \"%s\" must be 
non-empty.\n", arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 *arg_desc->opt_string = arg_str;
-return true;
+return OPT_OK;
 }
 
 /* Return number of non-NULL opt_* fields in opt_desc. */
@@ -186,13 +208,15 @@ parse_option (int argc, char **argv, const 
notmuch_opt_desc_t *options, int opt_
if (next != '=' && next != ':' && next != '\0')
continue;
 
+   bool incremented = false;
if (next == '\0' && next_arg != NULL && ! try->opt_bool) {
next = ' ';
value = next_arg;
+   incremented = true;
opt_index ++;
}
 
-   bool opt_status = false;
+   opt_handled opt_status = OPT_FAILED;
if (try->opt_keyword || try-

[PATCH] Standards-Version: bumped to 4.1.2 (no changes needed)

2017-12-11 Thread Daniel Kahn Gillmor

---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index f644695b..bf7eb647 100644
--- a/debian/control
+++ b/debian/control
@@ -28,7 +28,7 @@ Build-Depends:
  gpgsm ,
  gnupg ,
  bash-completion (>=1.9.0~)
-Standards-Version: 4.1.1
+Standards-Version: 4.1.2
 Homepage: https://notmuchmail.org/
 Vcs-Git: git://notmuchmail.org/git/notmuch
 Vcs-Browser: https://git.notmuchmail.org/git/notmuch
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4] python: add decrypt_policy argument to Database.index_file()

2017-12-11 Thread Daniel Kahn Gillmor

We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.

This now includes a simple test to ensure that the decrypt_policy
argument works as expected.
---
 bindings/python/notmuch/database.py | 45 +++--
 bindings/python/notmuch/globals.py  |  5 +
 test/T390-python.sh | 39 
 3 files changed, 87 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 1279804a..2a07e346 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
 _str,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
+NotmuchIndexoptsP,
 NotmuchMessageP,
 NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
 MODE = Enum(['READ_ONLY', 'READ_WRITE'])
 """Constants: Mode in which to open the database"""
 
+DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+"""Constants: policies for decrypting messages during indexing"""
+
 """notmuch_database_get_directory"""
 _get_directory = nmlib.notmuch_database_get_directory
 _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
 # return the Directory, init it with the absolute path
 return Directory(abs_dirpath, dir_p, self)
 
+_get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+_get_default_indexopts.argtypes = [NotmuchDatabaseP]
+_get_default_indexopts.restype = NotmuchIndexoptsP
+
+_indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+_indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+_indexopts_set_decrypt_policy.restype = None
+
+_indexopts_destroy = nmlib.notmuch_indexopts_destroy
+_indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+_indexopts_destroy.restype = None
+
 _index_file = nmlib.notmuch_database_index_file
 _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
  c_void_p,
  POINTER(NotmuchMessageP)]
 _index_file.restype = c_uint
 
-def index_file(self, filename, sync_maildir_flags=False):
+def index_file(self, filename, sync_maildir_flags=False, 
decrypt_policy=None):
 """Adds a new message to the database
 
 :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
 API. You might want to look into the underlying method
 :meth:`Message.maildir_flags_to_tags`.
 
+:param decrypt_policy: If the message contains any encrypted
+parts, and decrypt_policy is set to
+:attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+decrypt the message and index the cleartext, stashing any
+discovered session keys.  If it is set to
+:attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+decrypt during indexing.  If it is set to
+:attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+any stashed session keys it knows about, but will not try
+to access the user's secret keys.
+:attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+:attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+are stashed in the database.  If decrypt_policy is set to
+None (the default), then the database itself will decide
+whether to decrypt, based on the `index.decrypt`
+configuration setting (see notmuch-config(1)).
+
 :returns: On success, we return
 
1) a :class:`Message` object that can be used for things
@@ -457,7 +490,15 @@ class Database(object):
 """
 self._assert_db_is_initialized()
 msg_p = NotmuchMessageP()
-status = self._index_file(self._db, _str(filename), c_void_p(None), 
byref(msg_p))
+indexopts = c_void_p(None)
+if decrypt_policy is not None:
+indexopts = self._get_default_indexopts(self._db)
+self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+status = self._index_file(self._db, _str(filename), indexopts, 
byref(msg_p))
+
+if indexopts:
+self._indexopts_destroy(indexopts)
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
 raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
 pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+pass
+NotmuchIndexopt

[PATCH v3] crypto: signature verification reports valid User IDs

2017-12-08 Thread Daniel Kahn Gillmor

When i'm trying to understand a message signature, i care that i know
who it came from (the "validity" of the identity associated with the
key), *not* whether i'm willing to accept the keyholder's other
identity assertions (the "trust" associated with the certificate).

We've been reporting User ID information based on the "trust"
associated with the certificate, because GMime didn't clearly expose
the validity of the User IDs.

This change relies on fixes made in GMime 3.0.3 and later which
include https://github.com/jstedfast/gmime/pull/18.
---
 configure  |  3 ++-
 notmuch-show.c | 10 +++---
 test/T355-smime.sh |  8 +++-
 util/gmime-extra.c | 28 
 util/gmime-extra.h | 17 ++---
 5 files changed, 46 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index c5e2ffed..daee5a6e 100755
--- a/configure
+++ b/configure
@@ -482,9 +482,10 @@ fi
 # we need to have a version >= 2.6.5 to avoid a crypto bug. We need
 # 2.6.7 for permissive "From " header handling.
 GMIME_MINVER=2.6.7
+GMIME3_MINVER=3.0.3
 
 printf "Checking for GMime development files... "
-if pkg-config --exists "gmime-3.0"; then
+if pkg-config --exists "gmime-3.0 > $GMIME3_MINVER"; then
 printf "Yes (3.0).\n"
 have_gmime=1
 gmime_cflags=$(pkg-config --cflags gmime-3.0)
diff --git a/notmuch-show.c b/notmuch-show.c
index 74e77249..bb44d938 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -446,15 +446,11 @@ format_part_sigstatus_sprinter (sprinter_t *sp, 
mime_node_t *node)
sp->map_key (sp, "expires");
sp->integer (sp, expires);
}
-   /* output user id only if validity is FULL or ULTIMATE. */
-   /* note that gmime is using the term "trust" here, which
-* is WRONG.  It's actually user id "validity". */
if (certificate) {
-   const char *name = g_mime_certificate_get_uid (certificate);
-   GMimeCertificateTrust trust = g_mime_certificate_get_trust 
(certificate);
-   if (name && (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)) {
+   const char *uid = g_mime_certificate_get_valid_userid 
(certificate);
+   if (uid) {
sp->map_key (sp, "userid");
-   sp->string (sp, name);
+   sp->string (sp, uid);
}
}
} else if (certificate) {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 03d24581..532cd84b 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -48,6 +48,12 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "signature verification (notmuch CLI)"
+if [ "${NOTMUCH_GMIME_MAJOR}" -lt 3 ]; then
+# gmime 2 can't report User IDs properly for S/MIME
+USERID=''
+else
+USERID='"userid": "CN=Notmuch Test Suite",'
+fi
 output=$(notmuch show --format=json --verify subject:"test signed message 001" 
\
 | notmuch_json_show_sanitize \
 | sed -e 's|"created": [-1234567890]*|"created": 946728000|' \
@@ -65,7 +71,7 @@ expected='[[[{"id": "X",
  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"fingerprint": "'$FINGERPRINT'",
- "status": "good",
+ "status": "good",'$USERID'
  "expires": 424242424,
  "created": 946728000}],
  "content-type": "multipart/signed",
diff --git a/util/gmime-extra.c b/util/gmime-extra.c
index 901d4d56..bc1e3c4d 100644
--- a/util/gmime-extra.c
+++ b/util/gmime-extra.c
@@ -33,6 +33,21 @@ g_string_talloc_strdup (void *ctx, char *g_string)
 
 #if (GMIME_MAJOR_VERSION < 3)
 
+const char *
+g_mime_certificate_get_valid_userid (GMimeCertificate *cert)
+{
+/* output user id only if validity is FULL or ULTIMATE. */
+/* note that gmime 2.6 is using the term "trust" here, which
+ * is WRONG.  It's actually user id "validity". */
+const char *name = g_mime_certificate_get_name (cert);
+if (name == NULL)
+   return name;
+GMimeCertificateTrust trust = g_mime_certificate_get_trust (cert);
+if (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)
+   return name;
+return NULL;
+}
+
 char *
 g_mime_message_get_address_string (GMimeMessage *message, GMimeRecipientType 
type)
 {
@@ -107,6 +122,19 @@ g_mime_utils_header_decode_date_unix (const char *date) {
 
 #else /* GMime >= 3.0 */
 
+const char *
+g_mime_certificate_get_valid_userid (GMimeCertificate *cert)
+{
+/* output user id only if validity is FULL or ULTIMATE. */
+const char *uid = g_mime_certificate_get_user_id (cert);
+if (uid == NULL)
+   return uid;
+GMimeValidity validity = g_mime_certificate_get_id_validity (cert);
+if (validity == GMIME_VALIDITY_FULL || validity == GMIME_VALIDITY_ULTIMATE)
+   return uid;
+return NULL;
+}
+
 const char*
 g_mime_certificate_get_fpr16 (GMimeCertificate *cert) {
 const char *fpr = g_mime_certificate_get_fingerprint (c

v3 of fixing signature verification

2017-12-08 Thread Daniel Kahn Gillmor

Bremner pointed out to me that v2 of this patch doesn't work with 2.6,
and that the name of the function should be
g_mime_certificate_get_valid_userid, instead of
g_mime_get_certificate_valid_userid.

Also, he asked that it be built against the "release" branch, since
it's important there, too.

So v3 is built against the release branch, though it is also trivial
to cherry-pick or merge into master.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] crypto: signature verification reports valid User IDs

2017-12-08 Thread Daniel Kahn Gillmor

When i'm trying to understand a message signature, i care that i know
who it came from (the "validity" of the identity associated with the
key), *not* whether i'm willing to accept the keyholder's other
identity assertions (the "trust" associated with the certificate).

We've been reporting User ID information based on the "trust"
associated with the certificate, because GMime didn't clearly expose
the validity of the User IDs.

This change relies on fixes made in GMime 3.0.3 and later which
include https://github.com/jstedfast/gmime/pull/18.
---
 configure  |  3 ++-
 notmuch-show.c | 10 +++---
 test/T355-smime.sh |  8 +++-
 util/gmime-extra.c | 26 ++
 util/gmime-extra.h | 17 ++---
 5 files changed, 44 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index fc70031b..b177b141 100755
--- a/configure
+++ b/configure
@@ -478,9 +478,10 @@ fi
 # we need to have a version >= 2.6.5 to avoid a crypto bug. We need
 # 2.6.7 for permissive "From " header handling.
 GMIME_MINVER=2.6.7
+GMIME3_MINVER=3.0.3
 
 printf "Checking for GMime development files... "
-if pkg-config --exists "gmime-3.0"; then
+if pkg-config --exists "gmime-3.0 > $GMIME3_MINVER"; then
 printf "Yes (3.0).\n"
 have_gmime=1
 gmime_cflags=$(pkg-config --cflags gmime-3.0)
diff --git a/notmuch-show.c b/notmuch-show.c
index 7afd3947..7d2c3813 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -446,15 +446,11 @@ format_part_sigstatus_sprinter (sprinter_t *sp, 
mime_node_t *node)
sp->map_key (sp, "expires");
sp->integer (sp, expires);
}
-   /* output user id only if validity is FULL or ULTIMATE. */
-   /* note that gmime is using the term "trust" here, which
-* is WRONG.  It's actually user id "validity". */
if (certificate) {
-   const char *name = g_mime_certificate_get_uid (certificate);
-   GMimeCertificateTrust trust = g_mime_certificate_get_trust 
(certificate);
-   if (name && (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)) {
+   const char *uid = g_mime_get_certificate_valid_userid 
(certificate);
+   if (uid) {
sp->map_key (sp, "userid");
-   sp->string (sp, name);
+   sp->string (sp, uid);
}
}
} else if (certificate) {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 1523f17b..be45e3b1 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -48,6 +48,12 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "signature verification (notmuch CLI)"
+if [ "${NOTMUCH_GMIME_MAJOR}" -lt 3 ]; then
+# gmime 2 can't report User IDs properly for S/MIME
+USERID=''
+else
+USERID='"userid": "CN=Notmuch Test Suite",'
+fi
 output=$(notmuch show --format=json --verify subject:"test signed message 001" 
\
 | notmuch_json_show_sanitize \
 | sed -e 's|"created": [-1234567890]*|"created": 946728000|' \
@@ -65,7 +71,7 @@ expected='[[[{"id": "X",
  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"fingerprint": "'$FINGERPRINT'",
- "status": "good",
+ "status": "good",'$USERID'
  "expires": 424242424,
  "created": 946728000}],
  "content-type": "multipart/signed",
diff --git a/util/gmime-extra.c b/util/gmime-extra.c
index 901d4d56..2d0f7443 100644
--- a/util/gmime-extra.c
+++ b/util/gmime-extra.c
@@ -196,4 +196,30 @@ g_mime_utils_header_decode_date_unix (const char *date) {
 return ret;
 }
 
+const char *
+g_mime_get_certificate_valid_userid (GMimeCertificate *cert)
+{
+/* output user id only if validity is FULL or ULTIMATE. */
+#if (GMIME_MAJOR_VERSION < 3)
+/* note that gmime 2.6 is using the term "trust" here, which
+ * is WRONG.  It's actually user id "validity". */
+const char *name = g_mime_certificate_get_name (cert);
+if (name == NULL)
+   return name;
+GMimeCertificateTrust trust = g_mime_certificate_get_trust (cert);
+if (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)
+   return name;
+return NULL;
+#else
+const char *uid = g_mime_certificate_get_user_id (cert);
+if (uid == NULL)
+   return uid;
+GMimeValidity validity = g_mime_certificate_get_id_validity (cert);
+if (validity == GMIME_VALIDITY_FULL || validity == GMIME_VALIDITY_ULTIMATE)
+   return uid;
+return NULL;
+#endif
+}
+
+
 #endif
diff --git a/util/gmime-extra.h b/util/gmime-extra.h
index 40bf1454..389cf38f 100644
--- a/util/gmime-extra.h
+++ b/util/gmime-extra.h
@@ -16,11 +16,9 @@ GMimeStream *g_mime_stream_stdout_new(void);
 #define g_mime_2_6_unref(obj) g_object_unref (obj)
 #define g_mime_3_unused(arg) arg
 #define g_mime_certificate_get_fpr16(cert) g_mime_certificate_get_key_id (cert)
-#define g_mime_certificate_get_uid(cert) g_mime_certificate_get_na

Re: [PATCH] crypto: signature verification reports valid User IDs

2017-12-08 Thread Daniel Kahn Gillmor

On Fri 2017-12-08 14:36:04 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> @@ -478,9 +478,10 @@ fi
>>  # we need to have a version >= 2.6.5 to avoid a crypto bug. We need
>>  # 2.6.7 for permissive "From " header handling.
>>  GMIME_MINVER=2.6.7
>> +GMIME3_MINVER=3.0.3
>
> Does this change mean notmuch won't build at all for people with gmime-3
> < 3.0.3? I understand that's not a problem in Debian, but what about in
> general?

iiuc, if you've only got gmime-3 version < 3.0.3, then you will not be
able to build notmuch.  however, if you have that *and* you have a
version of gmime-2.6, configure will just use gmime 2.6.

If you only have gmime-2.4, or a version of gmime 2.6 before 2.6.7, then
you are SOL.

I think that's an acceptable situation, though.  We're not taking away
too many options from users.

Also, when built against 3.0.0, 3.0.1, or 3.0.2, the identification
information shown during signed messages is strictly *wrong* -- it shows
the assigned ownertrust of a certificate where it should show the
validity.  This is not only wrong, but it's actively dangerous, because
it encourages users to mark keys with elevated ownertrust "just to turn
them green".  This is the equivalent of encouraging users to add
arbitrary root CAs to their list of trusted X.509 anchors.  Not
something a MUA should be doing.

>> +
>> +static const char*
>> +_get_certificate_valid_userid (GMimeCertificate *cert)
>> +{
>
> Since we already have util/gmime-extra.c to deal with differences
> between gmime-3.0 and 2.6, and since this code is purely gmime related
> (not using any notmuch data or types), it makes sense to me put this
> (split) there. I guess we might use a different prefix for stuff we add,
> although iirc we didn't bother for zlib-extra.c

The (very slight) disadvantage of putting this code in
util/gmime-extra.c is that it will no longer be static, but will instead
be exposed across compilation units.  That said, i don't think that's
actually a big deal, and i understand why you want it in
util/gmime-extra.c. So i'll send a new revision of this patch shortly.

--dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

2017-12-07 Thread Daniel Kahn Gillmor

On Thu 2017-12-07 00:20:12 -0800, Jameson Graef Rollins wrote:
> On Mon, Dec 04 2017, David Bremner  wrote:
>> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
>> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
>> at those.
>
> Yes, this new series is great and definitely addresses all my concerns.
> I'm stoked to see that the first part of it has been pushed, and looking
> forward to the full series!
>
> This is really great progress, Daniel.  Thanks for pushing on this.

Thanks for the review!  I've just pushed v3 of what remains of this
series, which is basically the same as the remaining patches here, with
a couple minor cleanups.

you can find it starting at
id:20171208062404.17269-1-...@fifthhorseman.net

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] crypto: signature verification reports valid User IDs

2017-12-07 Thread Daniel Kahn Gillmor

On Wed 2017-11-29 23:20:35 -0500, Daniel Kahn Gillmor wrote:
> When i'm trying to understand a message signature, i care that i know
> who it came from (the "validity" of the identity associated with the
> key), *not* whether i'm willing to accept the keyholder's other
> identity assertions (the "trust" associated with the certificate).
>
> We've been reporting User ID information based on the "trust"
> associated with the certificate, because GMime didn't clearly expose
> the validity of the User IDs.
>
> This change relies on fixes made in GMime 3.0.3 and later which
> include https://github.com/jstedfast/gmime/pull/18.

Without this patch, the notmuch test suite currently fails when built
against gmime 3 on debian testing, because gmime 3 on debian testing
now correctly returns key IDs for _get_key_id().

This patch corrects the misbehavior of previous versions of notmuch, and
restores the ability to display correct validity of the keys in
question.

I welcome review for it!

--dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 10/15] cli/new, insert, reindex: change index.decrypt to "auto" by default

2017-12-07 Thread Daniel Kahn Gillmor

The new "auto" decryption policy is not only good for "notmuch show"
and "notmuch reindex".  It's also useful for indexing messages --
there's no good reason to not try to go ahead and index the cleartext
of a message that we have a stashed session key for.

This change updates the defaults and tunes the test suite to make sure
that they have taken effect.
---
 doc/man1/notmuch-config.rst   | 2 +-
 lib/indexopts.c   | 8 +---
 test/T357-index-decryption.sh | 4 ++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 4835f897..1a9e08a3 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -154,7 +154,7 @@ The available configuration items are described below.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
-Default: ``false``.
+Default: ``auto``.
 
 **built_with.**
 
diff --git a/lib/indexopts.c b/lib/indexopts.c
index a04d1c1c..26a31e89 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,7 +26,7 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
-ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_AUTO;
 
 char * decrypt_policy;
 notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt_policy);
@@ -38,8 +38,10 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
(!(strcasecmp(decrypt_policy, "yes"))) ||
(!(strcasecmp(decrypt_policy, "1"
notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
-   else if (!strcasecmp(decrypt_policy, "auto"))
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+   else if ((!(strcasecmp(decrypt_policy, "false"))) ||
+(!(strcasecmp(decrypt_policy, "no"))) ||
+(!(strcasecmp(decrypt_policy, "0"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_FALSE);
 }
 
 free (decrypt_policy);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index c3730488..bd213415 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -142,7 +142,7 @@ test_expect_equal \
 
 # ensure no session keys are present:
 test_begin_subtest 'reindex using only session keys'
-test_expect_success 'notmuch reindex --decrypt=auto tag:encrypted and 
property:index.decryption=success'
+test_expect_success 'notmuch reindex tag:encrypted and 
property:index.decryption=success'
 test_begin_subtest "reindexed encrypted messages, decrypting only with session 
keys"
 output=$(notmuch search wumpus)
 expected=''
@@ -190,7 +190,7 @@ notmuch restore

[PATCH v3 12/15] crypto: actually stash session keys when decrypt=true

2017-12-07 Thread Daniel Kahn Gillmor

If you're going to store the cleartext index of an encrypted message,
in most situations you might just as well store the session key.
Doing this storage has efficiency and recoverability advantages.

Combined with a schedule of regular OpenPGP subkey rotation and
destruction, this can also offer security benefits, like "deletable
e-mail", which is the store-and-forward analog to "forward secrecy".

But wait, i hear you saying, i have a special need to store cleartext
indexes but it's really bad for me to store session keys!  Maybe
(let's imagine) i get lots of e-mails with incriminating photos
attached, and i want to be able to search for them by the text in the
e-mail, but i don't want someone with access to the index to be
actually able to see the photos themselves.

Fret not, the next patch in this series will support your wacky
uncommon use case.
---
 doc/man1/notmuch-config.rst | 10 ++
 doc/man1/notmuch-insert.rst | 13 +++--
 doc/man1/notmuch-new.rst| 21 +++--
 doc/man1/notmuch-reindex.rst| 10 +-
 doc/man7/notmuch-properties.rst |  4 
 lib/index.cc| 18 +-
 test/T357-index-decryption.sh   | 24 +++-
 util/crypto.c   | 16 +++-
 8 files changed, 88 insertions(+), 28 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 1a9e08a3..dabf269f 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -143,10 +143,12 @@ The available configuration items are described below.
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
-index the cleartext.  If ``auto``, it will try to index the
-cleartext if a stashed session key is already known for the message,
-but will not try to access your secret keys.  Use ``false`` to
-avoid decrypting even when a session key is already known.
+index the cleartext, stashing a copy of any discovered session
+keys for the message.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message
+(e.g. from a previous copy), but will not try to access your
+secret keys.  Use ``false`` to avoid decrypting even when a
+stashed session key is already present.
 
 Be aware that the notmuch index is likely sufficient to
 reconstruct the cleartext of the message itself, so please
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index b22be863..214f261b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -54,12 +54,13 @@ Supported options for **insert** include
 ``--decrypt=(true|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing.  If ``auto``, and notmuch already
-knows about a session key for the message, it will try
-decrypting using that session key but will not try to access
-the user's secret keys.  If decryption is successful, index
-the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).
+message while indexing, storing any session keys discovered.
+If ``auto``, and notmuch already knows about a session key for
+the message, it will try decrypting using that session key but
+will not try to access the user's secret keys.  If decryption
+is successful, index the cleartext itself.  Either way, the
+message is always stored to disk in its original form
+(ciphertext).
 
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 71df31d7..27676a19 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -46,16 +46,17 @@ Supported options for **new** include
 ``--decrypt=(true|auto|false)``
 
 If ``true``, when encountering an encrypted message, try to
-decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  If ``auto``, try to use any session key
-already known to belong to this message, but do not attempt to
-use the user's secret keys.
-
-Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected.  DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+decrypt it while indexing, and store any discovered session
+keys.  If ``auto``, try to use any session key already known
+to belong to this message, but do not attempt to use the

[PATCH v3 08/15] cli/new, insert, reindex: update documentation for --decrypt=auto

2017-12-07 Thread Daniel Kahn Gillmor

we also include --decrypt=auto in the tab completion.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   | 10 +++---
 doc/man1/notmuch-reindex.rst   | 23 ++-
 4 files changed, 34 insertions(+), 21 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index f94dbeed..272131e6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index eb9ff11b..b22be863 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,14 +51,18 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true and the message is encrypted, try to decrypt the
-message while indexing.  If decryption is successful, index
+If ``true`` and the message is encrypted, try to decrypt the
+message while indexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If decryption is successful, index
 the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).  Be aware
-that the index is likely sufficient to reconstruct the
-cleartext of the message itself, so please ensure that the
+stored to disk in its original form (ciphertext).
+
+Be aware that the index is likely sufficient to reconstruct
+the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
 ``--decrypt=true`` without considering the security of
 your index.
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 1df86f06..71df31d7 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -43,11 +43,15 @@ Supported options for **new** include
 ``--quiet``
 Do not print progress or results.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true, when encountering an encrypted message, try to
+If ``true``, when encountering an encrypted message, try to
 decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  Be aware that the index is likely
+the cleartext itself.  If ``auto``, try to use any session key
+already known to belong to this message, but do not attempt to
+use the user's secret keys.
+
+Be aware that the index is likely
 sufficient to reconstruct the cleartext of the message itself,
 so please ensure that the notmuch message index is adequately
 protected.  DO NOT USE ``--decrypt=true`` without
diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index 782b0d7b..d87e9d85 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -21,15 +21,20 @@ messages using the supplied options.
 
 Supported options for **reindex** include
 
-``--decrypt=(true|false)``
-
-If true, when encountering an encrypted message, try to
-decrypt it while reindexing.  If decryption is successful,
-index the cleartext itself.  Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected. DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+``--decrypt=(true|auto|false)``
+
+If ``true``, when encountering an encrypted message, try to
+decrypt it while reindexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If decrypti

[PATCH v3 14/15] docs: clean up documentation about decryption policies

2017-12-07 Thread Daniel Kahn Gillmor

Now that the range of sensible decryption policies has come into full
view, we take a bit of space to document the distinctions.

Most people will use either "auto" or "true" -- but we provide "false"
and "nostash" to handle use cases that might reasonably be requested.

Note also that these can be combined in sensible ways.  Like, if your
mail comes in regularly to a service that doesn't have access to your
secret keys, but does have access to your index, and you feel
comfortable adding selected encrypted messages to the index after
you've read them, you could stay in "auto" normally, and then when you
find yourself reading an indexable message (e.g. one you want to be
able to search for in the future, and that you don't mind exposing to
whatever entities have access to your inde), you can do:

notmuch reindex --decrypt=true id:whate...@example.biz

That leaves your default the same (still "auto") but you get the
cleartext index and stashed session key benefits for that particular
message.
---
 doc/man1/notmuch-config.rst | 33 -
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 773fd9da..3ba849b2 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,7 +142,9 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 
-One of ``false``, ``auto``, ``nostash``, or ``true``.
+Policy for decrypting encrypted messages during indexing.
+Must be one of: ``false``, ``auto``, ``nostash``, or
+``true``.
 
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
@@ -156,6 +158,34 @@ The available configuration items are described below.
 ``nostash`` is the same as ``true`` except that it will not
 stash newly-discovered session keys in the database.
 
+From the command line (i.e. during **notmuch-new(1)**,
+**notmuch-insert(1)**, or **notmuch-reindex(1)**), the user
+can override the database's stored decryption policy with the
+``--decrypt=`` option.
+
+Here is a table that summarizes the functionality of each of
+these policies:
+
+++---+--+-+--+
+|| false | auto | nostash | true |
+++===+==+=+==+
+| Index cleartext using  |   |  X   |X|  X   |
+| stashed session keys   |   |  | |  |
+++---+--+-+--+
+| Index cleartext|   |  |X|  X   |
+| using secret keys  |   |  | |  |
+++---+--+-+--+
+| Stash session keys |   |  | |  X   |
+++---+--+-+--+
+| Delete stashed session |   X   |  | |  |
+| keys on reindex|   |  | |  |
+++---+--+-+--+
+
+Stashed session keys are kept in the database as properties
+associated with the message.  See ``session-key`` in
+**notmuch-properties(7)** for more details about how they can
+be useful.
+
 Be aware that the notmuch index is likely sufficient (and a
 stashed session key is certainly sufficient) to reconstruct
 the cleartext of the message itself, so please ensure that the
@@ -201,5 +231,6 @@ SEE ALSO
 **notmuch-restore(1)**,
 **notmuch-search(1)**,
 **notmuch-search-terms(7)**,
+**notmuch-properties(7)**,
 **notmuch-show(1)**,
 **notmuch-tag(1)**
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 15/15] python: add decrypt_policy argument to Database.index_file()

2017-12-07 Thread Daniel Kahn Gillmor

We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.
---
 bindings/python/notmuch/database.py | 45 +++--
 bindings/python/notmuch/globals.py  |  5 +
 2 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 1279804a..2a07e346 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
 _str,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
+NotmuchIndexoptsP,
 NotmuchMessageP,
 NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
 MODE = Enum(['READ_ONLY', 'READ_WRITE'])
 """Constants: Mode in which to open the database"""
 
+DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+"""Constants: policies for decrypting messages during indexing"""
+
 """notmuch_database_get_directory"""
 _get_directory = nmlib.notmuch_database_get_directory
 _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
 # return the Directory, init it with the absolute path
 return Directory(abs_dirpath, dir_p, self)
 
+_get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+_get_default_indexopts.argtypes = [NotmuchDatabaseP]
+_get_default_indexopts.restype = NotmuchIndexoptsP
+
+_indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+_indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+_indexopts_set_decrypt_policy.restype = None
+
+_indexopts_destroy = nmlib.notmuch_indexopts_destroy
+_indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+_indexopts_destroy.restype = None
+
 _index_file = nmlib.notmuch_database_index_file
 _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
  c_void_p,
  POINTER(NotmuchMessageP)]
 _index_file.restype = c_uint
 
-def index_file(self, filename, sync_maildir_flags=False):
+def index_file(self, filename, sync_maildir_flags=False, 
decrypt_policy=None):
 """Adds a new message to the database
 
 :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
 API. You might want to look into the underlying method
 :meth:`Message.maildir_flags_to_tags`.
 
+:param decrypt_policy: If the message contains any encrypted
+parts, and decrypt_policy is set to
+:attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+decrypt the message and index the cleartext, stashing any
+discovered session keys.  If it is set to
+:attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+decrypt during indexing.  If it is set to
+:attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+any stashed session keys it knows about, but will not try
+to access the user's secret keys.
+:attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+:attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+are stashed in the database.  If decrypt_policy is set to
+None (the default), then the database itself will decide
+whether to decrypt, based on the `index.decrypt`
+configuration setting (see notmuch-config(1)).
+
 :returns: On success, we return
 
1) a :class:`Message` object that can be used for things
@@ -457,7 +490,15 @@ class Database(object):
 """
 self._assert_db_is_initialized()
 msg_p = NotmuchMessageP()
-status = self._index_file(self._db, _str(filename), c_void_p(None), 
byref(msg_p))
+indexopts = c_void_p(None)
+if decrypt_policy is not None:
+indexopts = self._get_default_indexopts(self._db)
+self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+status = self._index_file(self._db, _str(filename), indexopts, 
byref(msg_p))
+
+if indexopts:
+self._indexopts_destroy(indexopts)
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
 raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
 pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+pass
+NotmuchIndexoptsP = POINTER(NotmuchIndexoptsS)
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/l

[PATCH v3 06/15] cli/show: use decryption policy "auto" by default.

2017-12-07 Thread Daniel Kahn Gillmor

When showing a message, if the user doesn't specify --decrypt= at all,
but a stashed session key is known to notmuch, notmuch should just go
ahead and try to decrypt the message with the session key (without
bothering the user for access to their asymmetric secret key).

The user can disable this at the command line with --decrypt=false if
they really don't want to look at the e-mail that they've asked
notmuch to show them.

and of course, "notmuch show --decrypt" still works for accessing the
user's secret keys if necessary.
---
 completion/notmuch-completion.bash |  4 ++--
 doc/man1/notmuch-show.rst  |  4 
 notmuch-show.c | 17 +++--
 test/T357-index-decryption.sh  | 17 +
 4 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 1cd616b3..f94dbeed 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,7 +517,7 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body)
+   --exclude|--body|--decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -526,7 +526,7 @@ _notmuch_show()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt --include-html 
${_notmuch_shared_options}"
+   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt= --include-html 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 7ba091cf..64caa7a6 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -123,6 +123,10 @@ Supported options for **show** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
 needed credentials. Without one, the decryption will fail.
 
diff --git a/notmuch-show.c b/notmuch-show.c
index e840a470..591889a9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1105,13 +1105,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
.part = -1,
.omit_excluded = true,
.output_body = true,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = NOTMUCH_FORMAT_NOT_SPECIFIED;
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -1126,7 +1127,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
{ .opt_bool = ¶ms.entire_thread, .name = "entire-thread",
  .present = &entire_thread_set },
{ .opt_int = ¶ms.part, .name = "part" },
-   { .opt_bool = &decrypt, .name = "decrypt" },
+   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
{ .opt_bool = ¶ms.crypto.verify, .name = "verify" },
{ .opt_bool = ¶ms.output_body, .name = "body" },
{ .opt_bool = ¶ms.include_html, .name = "include-html" },
@@ -1140,10 +1141,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt) {
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
-   /* decryption implies verification */
-   params.crypto.verify = true;
+if (decrypt_set) {
+   if (decrypt) {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+   /* decryption implies verification */
+   params.crypto.verify = true;
+   } else {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+   }
 }
 
 /* specifying a part implies single message display */
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 31991e22..c3730488 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -210,6 +210,23 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch show should show cleartext if session key is 
present"
+output=$(notmuch show id:simple-encryp...@crypto.notmuchmail.org | awk 
'/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
+expected='This is a top sekrit message.'
+if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
+test_subtest_k

[PATCH v3 04/15] crypto: new decryption policy "auto"

2017-12-07 Thread Daniel Kahn Gillmor

This new automatic decryption policy should make it possible to
decrypt messages that we have stashed session keys for, without
incurring a call to the user's asymmetric keys.
---
 doc/man1/notmuch-config.rst   | 11 ---
 lib/index.cc  |  3 ++-
 lib/indexopts.c   | 13 -
 lib/notmuch.h |  1 +
 mime-node.c   |  7 ---
 notmuch-client.h  |  4 +++-
 notmuch.c |  3 ++-
 test/T357-index-decryption.sh | 12 +++-
 util/crypto.c |  9 -
 util/crypto.h |  3 ++-
 10 files changed, 49 insertions(+), 17 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index ea3d9754..4835f897 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,9 +142,14 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
-set to true, notmuch will try to decrypt the message and index
-the cleartext.  Be aware that the index is likely sufficient
-to reconstruct the cleartext of the message itself, so please
+set to ``true``, notmuch will try to decrypt the message and
+index the cleartext.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message,
+but will not try to access your secret keys.  Use ``false`` to
+avoid decrypting even when a session key is already known.
+
+Be aware that the notmuch index is likely sufficient to
+reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
diff --git a/lib/index.cc b/lib/index.cc
index 905366ae..af999bd3 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,7 +548,8 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (message, crypto_ctx, encrypted_data, 
NULL, &err);
+clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+message, crypto_ctx, encrypted_data, NULL, 
&err);
 if (err) {
_notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
   err->domain, err->code, err->message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 78f53391..a04d1c1c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -33,11 +33,14 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 if (err)
return ret;
 
-if (decrypt_policy &&
-   ((!(strcasecmp(decrypt_policy, "true"))) ||
-(!(strcasecmp(decrypt_policy, "yes"))) ||
-(!(strcasecmp(decrypt_policy, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+if (decrypt_policy) {
+   if ((!(strcasecmp(decrypt_policy, "true"))) ||
+   (!(strcasecmp(decrypt_policy, "yes"))) ||
+   (!(strcasecmp(decrypt_policy, "1"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+   else if (!strcasecmp(decrypt_policy, "auto"))
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+}
 
 free (decrypt_policy);
 return ret;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 47633496..ff860e06 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2241,6 +2241,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
 typedef enum {
 NOTMUCH_DECRYPT_FALSE,
 NOTMUCH_DECRYPT_TRUE,
+NOTMUCH_DECRYPT_AUTO,
 } notmuch_decryption_policy_t;
 
 /**
diff --git a/mime-node.c b/mime-node.c
index c4de708b..49d668fe 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -205,7 +205,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
break;
 
node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt (parent ? 
parent->envelope_file : NULL,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, &decrypt_result, &err);
 }
 if (! node->decrypted_child) {
@@ -270,7 +271,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt != 
NOTMUCH_DECRYPT_FALSE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_c

session keys followup, version 3

2017-12-07 Thread Daniel Kahn Gillmor

The first part of the session-keys series has already landed --
thanks to everyone who reviewed it and shepherded it on its way!

This is a respin of the remaining patches in the series, introducing
very minor changes from the previous series: typo corrections, and a
fix to the test suite to mark session-key-dependent subtests as broken
when notmuch is built against an older version of GMime.

Happy hacking,

  --dkg

history: 

v1 was introduced in id:20171025065203.24403-1-...@fifthhorseman.net

v2 was introduced in id:20171130085946.11332-1-...@fifthhorseman.net

the first 6 patches of v2 have already landed, so the numbers in this
series are 6 less than previously ("v3 01/15" corresponds to "v2
07/21", etc)


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 03/15] lib: convert notmuch decryption policy to an enum

2017-12-07 Thread Daniel Kahn Gillmor

Future patches in this series will introduce new policies; this merely
readies the way for them.

We also convert --try-decrypt to a keyword argument instead of a boolean.
---
 lib/index.cc  |  2 +-
 lib/indexopts.c   | 21 +++--
 lib/notmuch.h | 14 --
 mime-node.c   |  4 ++--
 notmuch-client.h  |  9 +
 notmuch-reply.c   |  6 +-
 notmuch-show.c| 10 +++---
 notmuch.c | 13 -
 test/T357-index-decryption.sh | 12 ++--
 util/crypto.h |  2 +-
 10 files changed, 58 insertions(+), 35 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index ff14e408..905366ae 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
+if (!indexopts || (notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 0f65b97c..78f53391 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,25 +26,26 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
 
-char * decrypt;
-notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt);
+char * decrypt_policy;
+notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt_policy);
 if (err)
return ret;
 
-if (decrypt &&
-   ((!(strcasecmp(decrypt, "true"))) ||
-(!(strcasecmp(decrypt, "yes"))) ||
-(!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, true);
+if (decrypt_policy &&
+   ((!(strcasecmp(decrypt_policy, "true"))) ||
+(!(strcasecmp(decrypt_policy, "yes"))) ||
+(!(strcasecmp(decrypt_policy, "1")
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
 
-free (decrypt);
+free (decrypt_policy);
 return ret;
 }
 
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy)
+ notmuch_decryption_policy_t 
decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
@@ -52,7 +53,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
 return NOTMUCH_STATUS_SUCCESS;
 }
 
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index ef463090..47633496 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2233,6 +2233,16 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 notmuch_indexopts_t *
 notmuch_database_get_default_indexopts (notmuch_database_t *db);
 
+/**
+ * Stating a policy about how to decrypt messages.
+ *
+ * See index.decrypt in notmuch-config(1) for more details.
+ */
+typedef enum {
+NOTMUCH_DECRYPT_FALSE,
+NOTMUCH_DECRYPT_TRUE,
+} notmuch_decryption_policy_t;
+
 /**
  * Specify whether to decrypt encrypted parts while indexing.
  *
@@ -2245,7 +2255,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  */
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy);
+ notmuch_decryption_policy_t 
decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
@@ -2253,7 +2263,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
diff --git a/mime-node.c b/mime-node.c
index daaae9f8..c4de708b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -270,7 +270,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && node->ctx->crypto->decrypt)
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_content_type (part);
const char *protocol = g_mime_content_type_get_parameter (content_type, 
"protocol");
@@ -286,7 +286,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 #endif
 
 /* Handle P

[PATCH v3 11/15] cli/reindex: destroy stashed session keys when --decrypt=false

2017-12-07 Thread Daniel Kahn Gillmor

There are some situations where the user wants to get rid of the
cleartext index of a message.  For example, if they're indexing
encrypted messages normally, but suddenly they run across a message
that they really don't want any trace of in their index.

In that case, the natural thing to do is:

   notmuch reindex --decrypt=false id:whate...@example.biz

But of course, clearing the cleartext index without clearing the
stashed session key is just silly.  So we do the expected thing and
also destroy any stashed session keys while we're destroying the index
of the cleartext.

Note that stashed session keys are stored in the xapian database, but
xapian does not currently allow safe deletion (see
https://trac.xapian.org/ticket/742).

As a workaround, after removing session keys and cleartext material
from the database, the user probably should do something like "notmuch
compact" to try to purge whatever recoverable data is left in the
xapian freelist.  This problem really needs to be addressed within
xapian, though, if we want it fixed right.
---
 doc/man1/notmuch-reindex.rst  |  3 +++
 lib/message.cc|  5 +
 test/T357-index-decryption.sh | 17 +
 3 files changed, 25 insertions(+)

diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index d87e9d85..e8174f39 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -30,6 +30,9 @@ Supported options for **reindex** include
 the user's secret keys.  If decryption is successful, index
 the cleartext itself.
 
+If ``false``, notmuch reindex will also delete any stashed
+session keys for all messages matching the search terms.
+
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
diff --git a/lib/message.cc b/lib/message.cc
index 12743460..d5db89b6 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -2002,6 +2002,11 @@ notmuch_message_reindex (notmuch_message_t *message,
 ret = notmuch_message_remove_all_properties_with_prefix (message, 
"index.");
 if (ret)
goto DONE; /* XXX TODO: distinguish from other error returns above? */
+if (indexopts && notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE) {
+   ret = notmuch_message_remove_all_properties (message, "session-key");
+   if (ret)
+   goto DONE;
+}
 
 /* re-add the filenames with the associated indexopts */
 for (; notmuch_filenames_valid (orig_filenames);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index bd213415..9f46a01b 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -227,6 +227,23 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "purging stashed session keys should lose access to the 
cleartext"
+notmuch reindex --decrypt=false id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest "and cleartext should be unrecoverable now that there are 
no stashed session keys"
+notmuch dump
+notmuch reindex --decrypt=true id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 02/15] indexopts: change _try_decrypt to _decrypt_policy

2017-12-07 Thread Daniel Kahn Gillmor

This terminology makes it clearer what's going on at the API layer,
and paves the way for future changesets that offer more nuanced
decryption policy.
---
 lib/index.cc |  2 +-
 lib/indexopts.c  | 10 +-
 lib/notmuch.h|  8 
 notmuch-client.h |  4 ++--
 notmuch.c| 12 ++--
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 6eb60f30..ff14e408 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_try_decrypt (indexopts))
+if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index ca6bf6c9..0f65b97c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -36,24 +36,24 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
((!(strcasecmp(decrypt, "true"))) ||
 (!(strcasecmp(decrypt, "yes"))) ||
 (!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_try_decrypt (ret, true);
+   notmuch_indexopts_set_decrypt_policy (ret, true);
 
 free (decrypt);
 return ret;
 }
 
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt)
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
-indexopts->crypto.decrypt = try_decrypt;
+indexopts->crypto.decrypt = decrypt_policy;
 return NOTMUCH_STATUS_SUCCESS;
 }
 
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts)
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
return false;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 2c5dcab5..ef463090 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2244,17 +2244,17 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt);
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
- * see notmuch_indexopts_set_try_decrypt.
+ * see notmuch_indexopts_set_decrypt_policy.
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts);
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
  * Destroy a notmuch_indexopts_t object.
diff --git a/notmuch-client.h b/notmuch-client.h
index f7524e59..fc981459 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -500,8 +500,8 @@ int notmuch_minimal_options (const char* subcommand_name,
 /* the state chosen by the user invoking one of the notmuch
  * subcommands that does indexing */
 struct _notmuch_client_indexing_cli_choices {
-bool try_decrypt;
-bool try_decrypt_set;
+bool decrypt_policy;
+bool decrypt_policy_set;
 notmuch_indexopts_t * opts;
 };
 extern struct _notmuch_client_indexing_cli_choices indexing_cli_choices;
diff --git a/notmuch.c b/notmuch.c
index 7ee3ad0b..89d5e77b 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -99,8 +99,8 @@ int notmuch_minimal_options (const char *subcommand_name,
 
 struct _notmuch_client_indexing_cli_choices indexing_cli_choices = { };
 const notmuch_opt_desc_t  notmuch_shared_indexing_options [] = {
-{ .opt_bool = &indexing_cli_choices.try_decrypt,
-  .present = &indexing_cli_choices.try_decrypt_set,
+{ .opt_bool = &indexing_cli_choices.decrypt_policy,
+  .present = &indexing_cli_choices.decrypt_policy_set,
   .name = "decrypt" },
 { }
 };
@@ -111,21 +111,21 @@ notmuch_process_shared_indexing_options 
(notmuch_database_t *notmuch, g_mime_3_u
 {
 if (indexing_cli_choices.opts == NULL)
indexing_cli_choices.opts = notmuch_database_get_default_indexopts 
(notmuch);
-if (indexing_cli_choices.try_decrypt_set) {
+if (indexing_cli_choices.decrypt_policy_set) {
notmuch_status_t status;
if (indexing_cli_choices.opts == NULL)
return NOTMUCH_STATUS_OUT_OF_MEMORY;
-   status = notmuch_indexopts_set_try_decrypt (indexing_cli_choices.opts, 
indexing_cli_choices.try_decrypt);
+   status = notmuch_indexopts_set_decrypt_policy 
(indexing_cli_choices.opts, indexing_cli_choices.decrypt_policy);
if (status != NOTMUCH_STATUS_SUCCESS) {
fprintf (stderr, "Error: Failed to set index decryption policy to 
%s.

[PATCH v3 09/15] crypto: record whether an actual decryption attempt happened

2017-12-07 Thread Daniel Kahn Gillmor

In our consolidation of _notmuch_crypto_decrypt, the callers lost
track a little bit of whether any actual decryption was attempted.

Now that we have the more-subtle "auto" policy, it's possible that
_notmuch_crypto_decrypt could be called without having any actual
decryption take place.

This change lets the callers be a little bit smarter about whether or
not any decryption was actually attempted.
---
 lib/index.cc  | 17 -
 mime-node.c   |  4 ++--
 util/crypto.c |  7 ++-
 util/crypto.h |  3 ++-
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index af999bd3..3914012a 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,12 +548,19 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+bool attempted = false;
+clear = _notmuch_crypto_decrypt (&attempted, 
notmuch_indexopts_get_decrypt_policy (indexopts),
 message, crypto_ctx, encrypted_data, NULL, 
&err);
-if (err) {
-   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
-  err->domain, err->code, err->message);
-   g_error_free(err);
+if (!attempted)
+   return;
+if (err || !clear) {
+   if (err) {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
+  err->domain, err->code, err->message);
+   g_error_free(err);
+   } else {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(unknown error)\n");
+   }
/* Indicate that we failed to decrypt during indexing */
status = notmuch_message_add_property (message, "index.decryption", 
"failure");
if (status)
diff --git a/mime-node.c b/mime-node.c
index 49d668fe..11df082b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -204,8 +204,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
if (parent->envelope_file)
break;
 
-   node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(&node->decrypt_attempted,
+
node->ctx->crypto->decrypt,
 parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, &decrypt_result, &err);
 }
diff --git a/util/crypto.c b/util/crypto.c
index bb587571..338f1d5d 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -140,7 +140,8 @@ void _notmuch_crypto_cleanup (unused(_notmuch_crypto_t 
*crypto))
 #endif
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 g_mime_3_unused(GMimeCryptoContext* crypto_ctx),
 GMimeMultipartEncrypted *part,
@@ -162,6 +163,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
g_error_free (*err);
*err = NULL;
}
+   if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
ret = g_mime_multipart_encrypted_decrypt_session (part,
  crypto_ctx,
@@ -191,6 +194,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
 if (decrypt == NOTMUCH_DECRYPT_AUTO)
return ret;
 
+if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
 ret = g_mime_multipart_encrypted_decrypt(part, crypto_ctx,
 decrypt_result, err);
diff --git a/util/crypto.h b/util/crypto.h
index dc95b4ca..c384601c 100644
--- a/util/crypto.h
+++ b/util/crypto.h
@@ -16,7 +16,8 @@ typedef struct _notmuch_crypto {
 } _notmuch_crypto_t;
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 GMimeCryptoContext* crypto_ctx,
 GMimeMultipartEncrypted *part,
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 13/15] crypto: add --decrypt=nostash to avoid stashing session keys

2017-12-07 Thread Daniel Kahn Gillmor

Here's the configuration choice for people who want a cleartext index,
but don't want stashed session keys.

Interestingly, this "nostash" decryption policy is actually the same
policy that should be used by "notmuch show" and "notmuch reply",
since they never modify the index or database when they are invoked
with --decrypt.

We take advantage of this parallel to tune the behavior of those
programs so that we're not requesting session keys from GnuPG during
"show" and "reply" that we would then otherwise just throw away.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-config.rst| 17 -
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   |  7 ---
 doc/man1/notmuch-reindex.rst   | 16 ++--
 lib/indexopts.c|  2 ++
 lib/notmuch.h  |  1 +
 notmuch-reply.c|  2 +-
 notmuch-show.c |  3 ++-
 notmuch.c  |  1 +
 test/T357-index-decryption.sh  | 23 +++
 util/crypto.c  |  4 ++--
 12 files changed, 71 insertions(+), 27 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 272131e6..948c153b 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index dabf269f..773fd9da 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -141,6 +141,9 @@ The available configuration items are described below.
 **index.decrypt**
 
 **[STORED IN DATABASE]**
+
+One of ``false``, ``auto``, ``nostash``, or ``true``.
+
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
 index the cleartext, stashing a copy of any discovered session
@@ -150,11 +153,15 @@ The available configuration items are described below.
 secret keys.  Use ``false`` to avoid decrypting even when a
 stashed session key is already present.
 
-Be aware that the notmuch index is likely sufficient to
-reconstruct the cleartext of the message itself, so please
-ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.decrypt=true`` without considering the
-security of your index.
+``nostash`` is the same as ``true`` except that it will not
+stash newly-discovered session keys in the database.
+
+Be aware that the notmuch index is likely sufficient (and a
+stashed session key is certainly sufficient) to reconstruct
+the cleartext of the message itself, so please ensure that the
+notmuch message index is adequately protected.  DO NOT USE
+``index.decrypt=true`` or ``index.decrypt=nostash`` without
+considering the security of your index.
 
 Default: ``auto``.
 
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index 214f261b..1a3dfe98 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,10 +51,10 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|auto|false)``
+``--decrypt=(true|nostash|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing, storing any session keys discovered.
+message while indexing, stashing any session keys discovered.
 If ``auto``, and notmuch already knows about a session key for
 the message, it will try decrypting using that session key but
 will not try to access the user's secret keys.  If decryption
@@ -62,11 +62,15 @@ Supported options for **insert** include
 message is always stored to disk in its original form
 (ciphertext).
 
-Be aware that the index is likely sufficient to reconstruct
-the cleartext of the message itself, so please ensure that the
+

[PATCH v3 05/15] cli/reply: use decryption policy "auto" by default.

2017-12-07 Thread Daniel Kahn Gillmor

If the user doesn't specify --decrypt= at all, but a stashed session
key is known to notmuch, when replying to an encrypted message,
notmuch should just go ahead and decrypt.

The user can disable this at the command line with --decrypt=false,
though it's not clear why they would ever want to do that.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-reply.rst |  6 +-
 notmuch-reply.c|  9 +
 test/T357-index-decryption.sh  | 10 ++
 4 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index e462a82a..1cd616b3 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -350,12 +350,16 @@ _notmuch_reply()
COMPREPLY=( $( compgen -W "all sender" -- "${cur}" ) )
return
;;
+   --decrypt)
+   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
 case "${cur}" in
-*)
-   local options="--format= --format-version= --reply-to= --decrypt 
${_notmuch_shared_options}"
+   local options="--format= --format-version= --reply-to= --decrypt= 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index b6aec3c8..ede77930 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -80,8 +80,12 @@ Supported options for **reply** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
+needed credentials. Without one, the decryption will likely fail.
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/notmuch-reply.c b/notmuch-reply.c
index eec34bed..fd990a9a 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -700,11 +700,12 @@ notmuch_reply_command (notmuch_config_t *config, int 
argc, char *argv[])
 int opt_index;
 notmuch_show_params_t params = {
.part = -1,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -718,7 +719,7 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = &decrypt, .name = "decrypt" },
+   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -728,8 +729,8 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt)
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+if (decrypt_set)
+   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_TRUE : 
NOTMUCH_DECRYPT_FALSE;
 
 notmuch_exit_if_unsupported_format ();
 
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 7996ec67..31991e22 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -200,6 +200,16 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch reply should show cleartext if session key is 
present"
+output=$(notmuch reply id:simple-encryp...@crypto.notmuchmail.org | grep '^>')
+expected='> This is a top sekrit message.'
+if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
+test_subtest_known_broken
+fi
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 07/15] cli/show, reply: document use of stashed session keys in notmuch-properties

2017-12-07 Thread Daniel Kahn Gillmor

The stashed session keys are stored internally as notmuch properties.
So a user or developer who is reading about those properties might
want to understand how they fit into the bigger picture.

Note here that decrypting with a stored session key no longer needs
-decrypt for "notmuch show" and "notmuch reply".
---
 doc/man7/notmuch-properties.rst | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst
index 05444f6c..1a3f690e 100644
--- a/doc/man7/notmuch-properties.rst
+++ b/doc/man7/notmuch-properties.rst
@@ -77,9 +77,12 @@ of its normal activity.
 **session-key**
 
 When **notmuch-show(1)** or **nomtuch-reply** encounters a message
-with an encrypted part and ``--decrypt`` is set, if notmuch finds a
-``session-key`` property associated with the message, it will try
-that stashed session key for decryption.
+with an encrypted part, if notmuch finds a ``session-key``
+property associated with the message, it will try that stashed
+session key for decryption.
+
+If you do not want to use any stashed session keys that might be
+present, you should pass those programs ``--decrypt=false``.
 
 Using a stashed session key with "notmuch show" will speed up
 rendering of long encrypted threads.  It also allows the user to
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 01/15] indexing: Change from try_decrypt to decrypt

2017-12-07 Thread Daniel Kahn Gillmor

the command-line interface for indexing (reindex, new, insert) used
--try-decrypt; and the configuration records used index.try_decrypt.
But by comparison with "show" and "reply", there doesn't seem to be
any reason for the "try" prefix.

This changeset adjusts the command-line interface and the
configuration interface.

For the moment, i've left indexopts_{set,get}_try_decrypt alone.  The
subsequent changeset will address those.
---
 NEWS   |  4 ++--
 completion/notmuch-completion.bash | 12 ++--
 doc/man1/notmuch-config.rst|  4 ++--
 doc/man1/notmuch-insert.rst|  6 +++---
 doc/man1/notmuch-new.rst   |  6 +++---
 doc/man1/notmuch-reindex.rst   |  6 +++---
 doc/man7/notmuch-properties.rst|  2 +-
 lib/indexopts.c| 14 +++---
 notmuch-config.c   |  2 +-
 notmuch.c  |  4 ++--
 test/T357-index-decryption.sh  | 18 +-
 test/test-lib.sh   |  2 +-
 12 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/NEWS b/NEWS
index 412c678d..0465b9e8 100644
--- a/NEWS
+++ b/NEWS
@@ -19,9 +19,9 @@ Indexing cleartext of encrypted e-mails
   It's now possible to include the cleartext of encrypted e-mails in
   the notmuch index.  This makes it possible to search your encrypted
   e-mails with the same ease as searching cleartext.  This can be done
-  on a per-message basis with the --try-decrypt argument to indexing
+  on a per-message basis with the --decrypt argument to indexing
   commands (new, insert, reindex), or by default by running "notmuch
-  config set index.try_decrypt true".
+  config set index.decrypt true".
 
   Note that the contents of the index are sufficient to roughly
   reconstruct the cleartext of the message itself, so please ensure
diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 7aae4297..e462a82a 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -287,7 +287,7 @@ _notmuch_insert()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
return
;;
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -296,7 +296,7 @@ _notmuch_insert()
 ! $split &&
 case "${cur}" in
--*)
-   local options="--create-folder --folder= --keep --no-hooks 
--try-decrypt= ${_notmuch_shared_options}"
+   local options="--create-folder --folder= --keep --no-hooks 
--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
return
@@ -319,7 +319,7 @@ _notmuch_new()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -328,7 +328,7 @@ _notmuch_new()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--no-hooks --try-decrypt= --quiet 
${_notmuch_shared_options}"
+   local options="--no-hooks --decrypt= --quiet 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "${options}" -- ${cur}) )
;;
@@ -437,7 +437,7 @@ _notmuch_reindex()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -446,7 +446,7 @@ _notmuch_reindex()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--try-decrypt= ${_notmuch_shared_options}"
+   local options="--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 6961737f..ea3d9754 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -138,7 +138,7 @@ The available configuration items are described below.
 
 Default: ``gpg``.
 
-**index.try_decrypt**
+**index.decrypt**
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
@@ -146,7 +146,7 @@ The available configuration items are described below.
 the cleartext.  Be aware that the index is likely sufficient
 to reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.try_decrypt=true`` without considering the
+DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
 Default: ``false``.
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index c500b251..eb9ff11b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rs

Re: [PATCH 5/5] debian/control: put elpa-notmuch in Section: lisp.

2017-12-07 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 04:40:43 -0500, Daniel Kahn Gillmor wrote:
> This makes lintian stop complaining about:
>
> W: elpa-notmuch: wrong-section-according-to-package-name elpa-notmuch => lisp

I'm withdrawing this patch for consideration.  I'm convinced by
bremner's argument in https://bugs.debian.org/883772 that
language-specific sections are a bad idea!

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

2017-12-06 Thread Daniel Kahn Gillmor

Hi Brian--

On Wed 2017-12-06 10:00:19 -0500, Brian Sniffen wrote:
> Okay, https://github.com/briansniffen/notmuch/tree/nmweb is now rebased
> onto the notmuchmail.org head as of this morning.  All of the changes
> are under contrib/notmuch-web.

thanks for doing this!  

traditionally, we've encouraged code contributions to come in in
separate patches (not one big lump), with each commit clearly justifying
their changes.  And yes, tests would make it much more likely that this
sweet contribution doesn't bitrot!

one thing i note is you've included a copy of jquery directly.  i think
we want to avoid having retrieved or generated code in the repository.
if there's an external dependency, the repo should point to it but it
shouldn't include it.  notmuch doesn't include gmime, but it states that
it has gmime as a dependency (with specific versions), and lets the
builder/maintainer retrieve the dependency themselves.  This makes
licensing, source vetting, and maintenance easier by ensuring that it
all stays loosely coupled.  Presumably you aren't signing up for
ensuring that any security vulnerabilities in jquery get pushed into the
notmuch repo too :)

Maybe you could do a similar explicitly-stated dependency for jquery
instead of including it directly?  debian ships jquery as libjs-jquery,
for example (though you could also use an explicit fetch from the web
for systems that aren't running an OS that provides a packaged form of
jquery).

other folks might have other preferences, i hope they'll chime in.

 --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] test: named query tests are broken when missing Xapian FieldProcessor

2017-12-05 Thread Daniel Kahn Gillmor

Named queries don't work without Xapian FieldProcessor.  Rather than
silently skipping them, we should explictly mark them as broken when
building against an older version of Xapian.
---
 test/T600-named-queries.sh | 33 -
 1 file changed, 20 insertions(+), 13 deletions(-)

diff --git a/test/T600-named-queries.sh b/test/T600-named-queries.sh
index 59496c3e..e38dc5bb 100755
--- a/test/T600-named-queries.sh
+++ b/test/T600-named-queries.sh
@@ -50,21 +50,28 @@ notmuch restore < BEFORE
 notmuch dump | grep '^#@' > OUTPUT
 test_expect_equal_file QUERIES.BEFORE OUTPUT
 
-if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 1 ]; then
-test_begin_subtest "search named query"
-notmuch search query:test > OUTPUT
-notmuch search $QUERYSTR > EXPECTED
-test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "search named query"
+notmuch search query:test > OUTPUT
+notmuch search $QUERYSTR > EXPECTED
+if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then
+test_subtest_known_broken
+fi
+test_expect_equal_file EXPECTED OUTPUT
 
-test_begin_subtest "search named query with other terms"
-notmuch search query:test and subject:Maildir > OUTPUT
-notmuch search $QUERYSTR and subject:Maildir > EXPECTED
-test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "search named query with other terms"
+notmuch search query:test and subject:Maildir > OUTPUT
+notmuch search $QUERYSTR and subject:Maildir > EXPECTED
+if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then
+test_subtest_known_broken
+fi
+test_expect_equal_file EXPECTED OUTPUT
 
-test_begin_subtest "search nested named query"
-notmuch search query:test2 > OUTPUT
-notmuch search $QUERYSTR2 > EXPECTED
-test_expect_equal_file EXPECTED OUTPUT
+test_begin_subtest "search nested named query"
+notmuch search query:test2 > OUTPUT
+notmuch search $QUERYSTR2 > EXPECTED
+if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then
+test_subtest_known_broken
 fi
+test_expect_equal_file EXPECTED OUTPUT
 
 test_done
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/3] test: retrying lock tests are broken when built against older Xapian

2017-12-05 Thread Daniel Kahn Gillmor

If we're building against a version of Xapian that doesn't offer
retrying the lock, we should be honest and describe the tests as
broken, rather than marking them as missing a test prerequisite.

missing test prerequisites should be for specific components of the
test harness that are missing, not for the backend library notmuch
uses.
---
 test/T620-lock.sh | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/test/T620-lock.sh b/test/T620-lock.sh
index 085ffe43..7aaaff2a 100755
--- a/test/T620-lock.sh
+++ b/test/T620-lock.sh
@@ -2,13 +2,12 @@
 test_description="locking"
 . $(dirname "$0")/test-lib.sh || exit 1
 
-if [ "${NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK}" = "0" ]; then
-test_subtest_missing_external_prereq_["lock retry support"]=t
-fi
-
 add_email_corpus
 
 test_begin_subtest "blocking open"
+if [ $NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK -ne 1 ]; then
+test_subtest_known_broken
+fi
 test_C ${MAIL_DIR} <<'EOF'
 #include 
 #include 
@@ -70,6 +69,9 @@ inbox
 parent
 unread
 EOF
+if [ $NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK -ne 1 ]; then
+test_subtest_known_broken
+fi
 test_expect_equal_file EXPECTED OUTPUT
 
 test_done
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

be more honest about test suite failures due to old libraries

2017-12-05 Thread Daniel Kahn Gillmor

We have several places where tests are skipped or marked as though
some test suite prereqs are missing, but in fact are due to building
against older versions of libraries that don't support certain
features.

This series tries to be more honest about some of those tests by
marking them as broken, rather than just skipping them.

The series still doesn't address T650-regexp-query.sh, which is
basically entirely skipped if Xapian doesn't have a field processor,
rather than marked as broken.  But it's still an improvement in
overall test suite honesty over the status quo.



___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/3] test: searching by absolute date is broken without Xapian FieldProcessor

2017-12-05 Thread Daniel Kahn Gillmor

Previously, the test suite had simply silently skipped the absolute
date test if we're using an archaic version of Xapian.  For
correctness, we should instead mark the test as broken.

This also changes from string to numeric comparison when checking
NOMTUCH_HAVE_XAPIAN_FIELD_PROCESSOR for consistency with other tests.
---
 test/T500-search-date.sh | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh
index 5c5b99a0..f84b0962 100755
--- a/test/T500-search-date.sh
+++ b/test/T500-search-date.sh
@@ -12,11 +12,12 @@ test_begin_subtest "Absolute date range with 'same' 
operator"
 output=$(notmuch search date:2010-12-16..! | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; 
Essai accentué (inbox unread)"
 
-if [ "${NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR}" = "1" ]; then
-test_begin_subtest "Absolute date field"
-output=$(notmuch search date:2010-12-16 | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; 
Essai accentué (inbox unread)"
+test_begin_subtest "Absolute date field"
+output=$(notmuch search date:2010-12-16 | notmuch_search_sanitize)
+if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then
+test_subtest_known_broken
 fi
+test_expect_equal "$output" "thread:XXX   2010-12-16 [1/1] Olivier Berger; 
Essai accentué (inbox unread)"
 
 test_begin_subtest "Absolute time range with TZ"
 notmuch search date:18-Nov-2009_02:19:26-0800..2009-11-18_04:49:52-06:00 | 
notmuch_search_sanitize > OUTPUT
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

2017-12-05 Thread Daniel Kahn Gillmor

On Mon 2017-12-04 21:59:18 -0400, David Bremner wrote:
> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
> at those.

to be fair, i thought Jamie's concerns were correct -- the normalized
interface is better.  i was only a bit leery about the change
introducing an extra delay.  But Jamie's concerns were also corroborated
by id:87r2szgvik@tethera.net -- maybe you could ask the author of
that message to review?  :P I think Jamie is currently slammed by
non-notmuch work, unfortunately.

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] debian/control: build-depend on python3-sphinx instead of python-sphinx

2017-12-05 Thread Daniel Kahn Gillmor

python2 is going to be deprecated, and python3-sphinx is available all
the way back to oldoldstable.  let's use the more modern version.

To make this work and still ship the manpages, tell ./configure to
prefer python3 over python, if it exists.
---
 configure  | 2 +-
 debian/control | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/configure b/configure
index f1974520..b177b141 100755
--- a/configure
+++ b/configure
@@ -557,7 +557,7 @@ fi
 printf "Checking for python... "
 have_python=0
 
-for name in ${PYTHON} python python2 python3; do
+for name in ${PYTHON} python3 python python2; do
 if command -v $name > /dev/null; then
have_python=1
python=$name
diff --git a/debian/control b/debian/control
index c53ba20a..51129886 100644
--- a/debian/control
+++ b/debian/control
@@ -18,7 +18,7 @@ Build-Depends:
  python3-all (>= 3.1.2-7~),
  dh-python,
  dh-elpa (>= 1.3),
- python-sphinx (>= 1.0),
+ python3-sphinx,
  ruby, ruby-dev (>>1:1.9.3~),
  emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) |
  emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~) |
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: session keys are known broken without session key support

2017-12-05 Thread Daniel Kahn Gillmor

On Tue 2017-12-05 13:40:27 -0500, Daniel Kahn Gillmor wrote:
> If the version of GMime we're building against doesn't support session
> key extraction or re-use, mark the tests that rely on session key
> capabilities as known-broken.
>
> This should resolve test suite failures on ubuntu trusty and debian
> jessie and earlier, which have GMime 2.6.20 -- session key support was
> introduced in GMime 2.6.21.

This should fix the current travis build failure.  If it's accepted,
i'll use the same technique to mark other tests in my session-keys
series as dependent upon session-key capability.

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: session keys are known broken without session key support

2017-12-05 Thread Daniel Kahn Gillmor

If the version of GMime we're building against doesn't support session
key extraction or re-use, mark the tests that rely on session key
capabilities as known-broken.

This should resolve test suite failures on ubuntu trusty and debian
jessie and earlier, which have GMime 2.6.20 -- session key support was
introduced in GMime 2.6.21.
---
 test/T357-index-decryption.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 11ea2074..3efaa61f 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -183,6 +183,9 @@ EOF
 notmuch reindex --try-decrypt id:simple-encryp...@crypto.notmuchmail.org
 output=$(notmuch search sekrit)
 expected='thread:0001   2016-12-22 [1/1] Daniel Kahn Gillmor; 
encrypted message (encrypted inbox unread)'
+if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
+test_subtest_known_broken
+fi
 test_expect_equal \
 "$output" \
 "$expected"
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 4/5] debian: bump Standards-Version to 4.1.1

2017-12-05 Thread Daniel Kahn Gillmor

On Mon 2017-12-04 21:30:00 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> No minor changes were necessary to become compliant with Debian policy
>> version 4.1.1, so this is basically a freebie.
>
> pushed 2,3,4. Waiting for you and Tomi to reach a fixed point on
> 1.

Tomi seemed to say "I'm, fine with original" in
id:m28teiw635@guru.guru-group.fi though i confess his punctuation is
a little confusing to me.

so i think that's an agreement that my proposal (preferring python3) is
acceptable.  Maybe Tomi will propose a concrete patch if he wants
something fancier.

> Skeptical of 5; I tend to think lintian is wrong here. Not sure if I
> care enough to override it.

I'm fine if you want to drop #5, as long as you file a bug report
against lintian explaining why you think it's wrong :)

thanks for the review!

--dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help: give a hint about notmuch-emacs-mua

2017-12-04 Thread Daniel Kahn Gillmor

On Mon 2017-12-04 10:44:40 -0500, Daniel Kahn Gillmor wrote:
> But maybe you'll find my revised version
> (id:20171204154333.27505-1-...@fifthhorseman.net) mitigates the error
> you're pointing out?

sorry, that should be id:20171204184310.17125-1-...@fifthhorseman.net --
the previous message-id was never sent.

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] cli/help: give a hint about notmuch-emacs-mua

2017-12-04 Thread Daniel Kahn Gillmor

"notmuch help" doesn't mention "notmuch-emacs-mua" even though we
support it through the try_external_command() mechanism.

In addition, "notmuch help emacs-mua" doesn't work, even though we
ship the appropriate manpage.

This changeset fixes both of these problems.
---
 configure | 1 +
 notmuch.c | 7 ++-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index bd034f91..f1974520 100755
--- a/configure
+++ b/configure
@@ -1185,6 +1185,7 @@ COMMON_CONFIGURE_CFLAGS = \\
\$(GMIME_CFLAGS) \$(TALLOC_CFLAGS) \$(ZLIB_CFLAGS)  \\
-DHAVE_VALGRIND=\$(HAVE_VALGRIND) \$(VALGRIND_CFLAGS)   \\
-DHAVE_GETLINE=\$(HAVE_GETLINE) \\
+   -DWITH_EMACS=\$(WITH_EMACS) \\
-DHAVE_CANONICALIZE_FILE_NAME=\$(HAVE_CANONICALIZE_FILE_NAME) \\
-DHAVE_STRCASESTR=\$(HAVE_STRCASESTR)   \\
-DHAVE_STRSEP=\$(HAVE_STRSEP)   \\
diff --git a/notmuch.c b/notmuch.c
index d5bf7902..2d193aa8 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -172,6 +172,10 @@ static command_t commands[] = {
   "Re-index all messages matching the search terms." },
 { "config", notmuch_config_command, NOTMUCH_CONFIG_OPEN,
   "Get or set settings in the notmuch configuration file." },
+#if WITH_EMACS
+{ "emacs-mua", NULL, 0,
+  "send mail with notmuch and emacs." },
+#endif
 { "help", notmuch_help_command, NOTMUCH_CONFIG_CREATE, /* create but don't 
save config */
   "This message, or more detailed help for the named command." }
 };
@@ -487,7 +491,8 @@ main (int argc, char *argv[])
 notmuch_process_shared_options (command_name);
 
 command = find_command (command_name);
-if (!command) {
+/* if command->function is NULL, try external command */
+if (!command || !command->function) {
/* This won't return if the external command is found. */
if (try_external_command(argv + opt_index))
fprintf (stderr, "Error: Unknown command '%s' (see \"notmuch 
help\")\n",
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help: give a hint about notmuch-emacs-mua

2017-12-04 Thread Daniel Kahn Gillmor

On Sun 2017-12-03 15:26:32 +0200, Tomi Ollila wrote:
> My thought was that even though user may compile notmuch without emacs
> support (and notmuch-emacs-mua not installed) we would be hinting 
> `notmuch emacs-mua` command to exist.

Currently, if the user *does* compile with emacs support, and installs
it, we are hinting that "notmuch emacs-mua" *doesn't* exist.

Given that "notmuch emacs-mua" does exist somewhere (even if it's not in
the local installation), i prefer the error in the direction of
inclusion.

But maybe you'll find my revised version
(id:20171204154333.27505-1-...@fifthhorseman.net) mitigates the error
you're pointing out?

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/5] debian/control: build-depend on python3-sphinx instead of python-sphinx

2017-12-04 Thread Daniel Kahn Gillmor

On Sun 2017-12-03 15:35:04 +0200, Tomi Ollila wrote:
> On Thu, Nov 30 2017, Daniel Kahn Gillmor wrote:
>
>> On Thu 2017-11-30 04:40:39 -0500, Daniel Kahn Gillmor wrote:
>>> python2 is going to be deprecated, and python3-sphinx is available all
>>> the way back to oldoldstable.  let's use the more modern version.
>>> ---
>>>  debian/control | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/debian/control b/debian/control
>>> index 20b8a2db..3a624fdc 100644
>>> --- a/debian/control
>>> +++ b/debian/control
>>> @@ -18,7 +18,7 @@ Build-Depends:
>>>   python3-all (>= 3.1.2-7~),
>>>   dh-python,
>>>   dh-elpa (>= 1.3),
>>> - python-sphinx (>= 1.0),
>>> + python3-sphinx,
>>>   ruby, ruby-dev (>>1:1.9.3~),
>>>   emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) |
>>>   emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~) |
>>
>> hm, on further reflection, this one is specifically problematic because
>> of the use of:
>>
>> ${python} -m sphinx.writers.manpage
>>
>> in ./configure, used to test for the presence of the sphinx module.
>>
>> This can be fixed by preferring python3 here:
>>
>> --- a/configure
>> +++ b/configure
>> @@ -557,7 +557,7 @@ fi
>>  printf "Checking for python... "
>>  have_python=0
>>  
>> -for name in ${PYTHON} python python2 python3; do
>> +for name in ${PYTHON} python3 python python2; do
>
>>  if command -v $name > /dev/null; then
>> have_python=1
>> python=$name
>>
>> any thoughts or preferences on this?
>
> I'd like this change. I was going to suggest "${PYTHON}" there
> but perhaps if one sets e.g.
>
> PYTHON='/usr/local/bin/python3 -B -E -u'
>
> The above would still work (I'd say this is potentially more useful
> than supporting whitespace in paths).

does it work?  i think the -B -E -u will all be tried as possible python
implementations separately, no?  and $python itself will just get set to
/usr/local/bin/python3 with the arguments dropped :(

0 dkg@alice:~$ x="a b c"
0 dkg@alice:~$ for y in ${x} d e; do printf ':%s\n' "$y"; done
:a
:b
:c
:d
:e
0 dkg@alice:~$ 

While i think this doesn't support Tomi's extended use case, i still
think it's the right direction to move in.

Does anyone have any objection to the preference order change above?  Is
there any reason we should be preferring python 2 to python3 as we move
forward?

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 15/18] crypto: actually stash session keys when try-decrypt=true

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-16 08:53:14 -0400, David Bremner wrote:
> I'd be happier if we didn't further entrench the text format in the test
> suite. How hard would it be to use json output (+maybe python?) here? 

json output seems clunkier to me, and i don't think it's necessary for
the purposes of these tests.  Using python here isn't possible without
updating the python bindings to accomodate decryption policy, which
doesn't come until later in the series.

so i'd prefer to leave it as-is, but i wouldn't object if someone wanted
to propose a good patch to these tests that uses json.

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 18/18] python: add try_decrypt argument to Database.index_file()

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-16 09:06:09 -0400, David Bremner wrote:
> I think this new bindings functionality needs a test.

agreed, the python bindings do need to be added to the test suite (this
is also true in the newer version of the series).

I'm happy to add those tests as a condition of getting the python
bindings merged, but i hope they won't block the review and merge of the
rest of the series :)

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 21/21] python: add decrypt_policy argument to Database.index_file()

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 03:59:46 -0500, Daniel Kahn Gillmor wrote:
> @@ -454,10 +487,19 @@ class Database(object):
>:attr:`STATUS`.READ_ONLY_DATABASE
>Database was opened in read-only mode so no message can
>be added.
> +
>  """
>  self._assert_db_is_initialized()

ugh, somehow i missed this gratuitous whitespace addition the second
time around.  This is fixed in my gitlab session-keys branch [0], but it
doesn't seem worth re-posting the entire 21-message series here to fix
it.

--dkg

[0] https://gitlab.com/dkg/notmuch

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 04/21] crypto: use stashed session-key properties for decryption, if available

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 03:59:29 -0500, Daniel Kahn Gillmor wrote:
> +hexidecimal representation of the algorithm-specific key.  For

ugh, this should be hexadecimal, not hexidecimal.

This is fixed in my gitlab session-keys branch [0], but doesn't seem
worth re-posting the entire series for. :)

  --dkg

[0] https://gitlab.com/dkg/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 09:44:19 -0400, David Bremner wrote:
> Floris Bruynooghe  writes:
>
>> This also has the binary question problem, is this returned as bytes or
>> as str?  Current Python bindings seem to go for .decode('utf-8',
>> errors='ignore') afaik which is somewhat lossy.
>>
>
> IMHO there's no reason to support non-utf8 properties. We should
> document this restriction now while adoption is relatively light.

Agreed, my reading of the source is that we expect strings for both
property names and property values.  If they're both strings, we should
not support anything other than UTF-8 in either case.

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help: give a hint about notmuch-emacs-mua

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 11:08:05 +0200, Tomi Ollila wrote:
> On Thu, Nov 30 2017, Daniel Kahn Gillmor wrote:
>
>> On Thu 2017-10-26 18:27:51 -0400, Daniel Kahn Gillmor wrote:
>>> "notmuch help" doesn't mention "notmuch-emacs-mua" even though we
>>> support it through the try_external_command() mechanism.
>>>
>>> In addition, "notmuch help emacs-mua" doesn't work, even though we
>>> ship the appropriate manpage.
>>>
>>> This changeset fixes both of these problems.
>>
>> Ping!  i don't think this patch is complex or objectionable.  Can it be
>> merged?
>
> it may (*) look like we don't have #if WITH_EMACS (**) cpp macros defined ???
>
> Tomi
>
> (*) did not check...
>
> (**) this is emacs-specific stull, less generic that other hints (if that
> matters) ...
>
> Sorry to cause confusion...

confusion is right!  i don't understand what you're saying, sorry.  are
you suggesting a different change somehow?  Want to propose a different patch?

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/5] debian/control: build-depend on python3-sphinx instead of python-sphinx

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 04:40:39 -0500, Daniel Kahn Gillmor wrote:
> python2 is going to be deprecated, and python3-sphinx is available all
> the way back to oldoldstable.  let's use the more modern version.
> ---
>  debian/control | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/debian/control b/debian/control
> index 20b8a2db..3a624fdc 100644
> --- a/debian/control
> +++ b/debian/control
> @@ -18,7 +18,7 @@ Build-Depends:
>   python3-all (>= 3.1.2-7~),
>   dh-python,
>   dh-elpa (>= 1.3),
> - python-sphinx (>= 1.0),
> + python3-sphinx,
>   ruby, ruby-dev (>>1:1.9.3~),
>   emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) |
>   emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~) |

hm, on further reflection, this one is specifically problematic because
of the use of:

${python} -m sphinx.writers.manpage

in ./configure, used to test for the presence of the sphinx module.

This can be fixed by preferring python3 here:

--- a/configure
+++ b/configure
@@ -557,7 +557,7 @@ fi
 printf "Checking for python... "
 have_python=0
 
-for name in ${PYTHON} python python2 python3; do
+for name in ${PYTHON} python3 python python2; do
 if command -v $name > /dev/null; then
have_python=1
python=$name

any thoughts or preferences on this?

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: debian packaging cleanup

2017-11-30 Thread Daniel Kahn Gillmor

On Thu 2017-11-30 08:01:18 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> The following patch series should be fairly unobjectionable cleanup of
>> the debian packaging for notmuch.
>>
>> Let me know if you think there are problems with any of it.
>
> Not problems per se, but I would expect several of these things to be
> documented in debian/changelog, including the standards version change.

I was presuming to do one rollup patch to build out the debian/changelog
entry as release time approaches, rather than including changelog
messages in the patches themselves.

That rollup patch should be fairly easy to do with, e.g.:

gbp dch --new-version=0.26-1 debian/

I prefer this approach because:

 (a) it doesn't confuse changelog messages with patches, and

 (b) it means that we don't end up with heavily-ordered patches, where
 it's more difficult merge some patches from a series than others,
 because each patch to d/changelog depends on the immediately prior
 changeset.

I'd much rather you (or anyone else reviewing) can immediately and
trivially pull the changes you like from the series while leaving the
ones you're less certain about unmerged, for further discussion.

Case in point here, the change to python3-sphinx might be more
problematic than i'd originally expected.  I wouldn't want anyone to
have to rewrite the subsequent trivial patches just to leave that one
behind.

Is there a reason to prefer patching d/changelog in each individual
changeset?

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/5] debian/control: build-depend on python3-sphinx instead of python-sphinx

2017-11-30 Thread Daniel Kahn Gillmor

python2 is going to be deprecated, and python3-sphinx is available all
the way back to oldoldstable.  let's use the more modern version.
---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 20b8a2db..3a624fdc 100644
--- a/debian/control
+++ b/debian/control
@@ -18,7 +18,7 @@ Build-Depends:
  python3-all (>= 3.1.2-7~),
  dh-python,
  dh-elpa (>= 1.3),
- python-sphinx (>= 1.0),
+ python3-sphinx,
  ruby, ruby-dev (>>1:1.9.3~),
  emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) |
  emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~) |
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/5] debian: bump Standards-Version to 4.1.1

2017-11-30 Thread Daniel Kahn Gillmor

No minor changes were necessary to become compliant with Debian policy
version 4.1.1, so this is basically a freebie.
---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index bb88b0f7..51129886 100644
--- a/debian/control
+++ b/debian/control
@@ -28,7 +28,7 @@ Build-Depends:
  gpgsm ,
  gnupg ,
  bash-completion (>=1.9.0~)
-Standards-Version: 4.0.0
+Standards-Version: 4.1.1
 Homepage: https://notmuchmail.org/
 Vcs-Git: git://notmuchmail.org/git/notmuch
 Vcs-Browser: https://git.notmuchmail.org/git/notmuch
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] debian/control: put elpa-notmuch in Section: lisp.

2017-11-30 Thread Daniel Kahn Gillmor

This makes lintian stop complaining about:

W: elpa-notmuch: wrong-section-according-to-package-name elpa-notmuch => lisp
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index 51129886..cff610cf 100644
--- a/debian/control
+++ b/debian/control
@@ -121,6 +121,7 @@ Description: thread-based email index, search and tagging 
(transitional package)
 Package: elpa-notmuch
 Architecture: all
 Depends: ${misc:Depends}, ${elpa:Depends}
+Section: lisp
 Description: thread-based email index, search and tagging (emacs interface)
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/5] debian/changelog: remove trailing whitespace

2017-11-30 Thread Daniel Kahn Gillmor

---
 debian/changelog | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 67282a07..decef1e9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -836,7 +836,7 @@ notmuch (0.9~rc2-1) experimental; urgency=low
 - New SONAME for libnotmuch
 - bindings changes for ruby and python
   * Less non-text parts reported in replies.
-  * emacs: provide button action to fetch unknown gpg keys 
+  * emacs: provide button action to fetch unknown gpg keys
 
  -- David Bremner   Fri, 07 Oct 2011 18:53:04 -0300
 
@@ -845,7 +845,7 @@ notmuch (0.9~rc1-1) experimental; urgency=low
   * Upstream release candidate
 - Atomicity improvements, thanks to Austin Clements
 - Add missing call to g_type_init, thanks to Aaron Ecay
-  * notmuch-emacs: add versioned dependency on notmuch.  
+  * notmuch-emacs: add versioned dependency on notmuch.
 (Closes: #642240).
 
  -- David Bremner   Sun, 25 Sep 2011 11:26:01 -0300
@@ -854,7 +854,7 @@ notmuch (0.8-1) unstable; urgency=low
 
   * New upstream version.
 - Improved handling of message/rfc822 parts
-- Improved Build system portability 
+- Improved Build system portability
 - Documentation update for Ruby bindings
 - Unicode, iterator, PEP8 changes for python bindings
 
@@ -936,7 +936,7 @@ notmuch (0.6~254) experimental; urgency=low
 
   [ Jameson Rollins ]
   * Bump standards version to 3.9.2 (No changes).
-  
+
  -- David Bremner   Thu, 23 Jun 2011 07:50:05 -0300
 
 notmuch (0.6~237) experimental; urgency=low
@@ -981,8 +981,8 @@ notmuch (0.6~171) experimental; urgency=low
   * notmuch reply: Avoid segmentation fault when printing multiple parts
   * notmuch show: New part output handling.
   * emacs: Show cleaner `From:' addresses in the summary line.
-  * emacs: Add custom `notmuch-show-elide-same-subject', 
-`notmuch-show-always-show-subject' 
+  * emacs: Add custom `notmuch-show-elide-same-subject',
+`notmuch-show-always-show-subject'
   * emacs: Render text/x-vcalendar parts.
   * emacs: Add `notmuch-show-multipart/alternative-discouraged'.
   * vim: parse 'from' address, use sendmail directly, implement archive in
@@ -1114,7 +1114,7 @@ notmuch (0.2) unstable; urgency=low
   * Fix headers to be properly decoded in "notmuch reply"
   * emacs: Show the last few lines of citations as well as the first few lines.
   * emacs: The '+' and '-' commands can now add and remove tags by region.
-  * emacs: More meaningful buffer names for thread-view buffers. 
+  * emacs: More meaningful buffer names for thread-view buffers.
   * emacs: Customized colors of threads in search view based on tags.
 
  -- Carl Worth   Fri, 16 Apr 2010 10:20:23 -0700
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

debian packaging cleanup

2017-11-30 Thread Daniel Kahn Gillmor

The following patch series should be fairly unobjectionable cleanup of
the debian packaging for notmuch.

Let me know if you think there are problems with any of it.

--dkg

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/5] debian/control: Priority: extra is deprecated

2017-11-30 Thread Daniel Kahn Gillmor

Move transitional package to to "oldlibs/optional"

This resolves two lintian warnings:

W: notmuch-emacs: transitional-package-should-be-oldlibs-optional oldlibs/extra
W: notmuch-emacs: priority-extra-is-replaced-by-priority-optional
---
 debian/control | 1 -
 1 file changed, 1 deletion(-)

diff --git a/debian/control b/debian/control
index 3a624fdc..bb88b0f7 100644
--- a/debian/control
+++ b/debian/control
@@ -113,7 +113,6 @@ Description: Ruby interface to the notmuch mail search and 
index library
 
 Package: notmuch-emacs
 Section: oldlibs
-Priority: extra
 Architecture: all
 Depends: elpa-notmuch, ${misc:Depends}
 Description: thread-based email index, search and tagging (transitional 
package)
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 01/21] mime-node: handle decrypt_result more safely

2017-11-30 Thread Daniel Kahn Gillmor

If (for whatever reason) we don't get a decrypt_result back, or it's
not structured the way we expect it to be, we shouldn't choke on it.
---
 mime-node.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index e1aca969..7c8b2602 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -214,13 +214,15 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
 node->decrypt_success = true;
 node->verify_attempted = true;
 
-/* This may be NULL if the part is not signed. */
-node->sig_list = g_mime_decrypt_result_get_signatures (decrypt_result);
-if (node->sig_list) {
-   g_object_ref (node->sig_list);
-   set_signature_list_destructor (node);
+if (decrypt_result) {
+   /* This may be NULL if the part is not signed. */
+   node->sig_list = g_mime_decrypt_result_get_signatures (decrypt_result);
+   if (node->sig_list) {
+   g_object_ref (node->sig_list);
+   set_signature_list_destructor (node);
+   }
+   g_object_unref (decrypt_result);
 }
-g_object_unref (decrypt_result);
 
  DONE:
 if (err)
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 02/21] crypto: add _notmuch_crypto_decrypt wrapper function

2017-11-30 Thread Daniel Kahn Gillmor

We will use this centralized function to consolidate the awkward
behavior around different gmime versions.

It's only invoked from two places: mime-node.c's
node_decrypt_and_verify() and lib/index.cc's
_index_encrypted_mime_part().

However, those two places have some markedly distinct logic, so the
interface for this _notmuch_crypto_decrypt function is going to get a
little bit clunky.  It's worthwhile, though, for the sake of keeping
these #if directives reasonably well-contained.
---
 lib/index.cc  |  8 ++--
 mime-node.c   |  8 ++--
 util/crypto.c | 18 ++
 util/crypto.h |  7 +--
 4 files changed, 27 insertions(+), 14 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 6e684f5f..19d03456 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -530,9 +530,9 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 
 notmuch = _notmuch_message_database (message);
 
+GMimeCryptoContext* crypto_ctx = NULL;
 #if (GMIME_MAJOR_VERSION < 3)
 {
-   GMimeCryptoContext* crypto_ctx = NULL;
const char *protocol = NULL;
protocol = g_mime_content_type_get_parameter (content_type, "protocol");
status = _notmuch_crypto_get_gmime_ctx_for_protocol 
(&(indexopts->crypto),
@@ -546,13 +546,9 @@ _index_encrypted_mime_part (notmuch_message_t *message,
  "property (%d)\n", status);
return;
}
-   clear = g_mime_multipart_encrypted_decrypt(encrypted_data, crypto_ctx,
-  NULL, &err);
 }
-#else
-clear = g_mime_multipart_encrypted_decrypt(encrypted_data, 
GMIME_DECRYPT_NONE, NULL,
-  NULL, &err);
 #endif
+clear = _notmuch_crypto_decrypt (crypto_ctx, encrypted_data, NULL, &err);
 if (err) {
_notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
   err->domain, err->code, err->message);
diff --git a/mime-node.c b/mime-node.c
index 7c8b2602..ae3ebef6 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -199,12 +199,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
 GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 
 node->decrypt_attempted = true;
-node->decrypted_child = g_mime_multipart_encrypted_decrypt
-#if (GMIME_MAJOR_VERSION < 3)
-   (encrypteddata, cryptoctx, &decrypt_result, &err);
-#else
-(encrypteddata, GMIME_DECRYPT_NONE, NULL, &decrypt_result, &err);
-#endif
+if (! node->decrypted_child)
+   node->decrypted_child = _notmuch_crypto_decrypt (cryptoctx, 
encrypteddata, &decrypt_result, &err);
 if (! node->decrypted_child) {
fprintf (stderr, "Failed to decrypt part: %s\n",
 err ? err->message : "no error explanation given");
diff --git a/util/crypto.c b/util/crypto.c
index 5c84282e..087536ec 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -138,3 +138,21 @@ void _notmuch_crypto_cleanup (unused(_notmuch_crypto_t 
*crypto))
 {
 }
 #endif
+
+GMimeObject *
+_notmuch_crypto_decrypt (g_mime_3_unused(GMimeCryptoContext* crypto_ctx),
+GMimeMultipartEncrypted *part,
+GMimeDecryptResult **decrypt_result,
+GError **err)
+{
+GMimeObject *ret = NULL;
+
+#if (GMIME_MAJOR_VERSION < 3)
+ret = g_mime_multipart_encrypted_decrypt(part, crypto_ctx,
+decrypt_result, err);
+#else
+ret = g_mime_multipart_encrypted_decrypt(part, GMIME_DECRYPT_NONE, NULL,
+decrypt_result, err);
+#endif
+return ret;
+}
diff --git a/util/crypto.h b/util/crypto.h
index 1ff0297d..d68634f3 100644
--- a/util/crypto.h
+++ b/util/crypto.h
@@ -2,10 +2,8 @@
 #define _CRYPTO_H
 
 #include 
-#if (GMIME_MAJOR_VERSION < 3)
 #include "gmime-extra.h"
 #include "notmuch.h"
-#endif
 
 typedef struct _notmuch_crypto {
 bool verify;
@@ -17,6 +15,11 @@ typedef struct _notmuch_crypto {
 #endif
 } _notmuch_crypto_t;
 
+GMimeObject *
+_notmuch_crypto_decrypt (GMimeCryptoContext* crypto_ctx,
+GMimeMultipartEncrypted *part,
+GMimeDecryptResult **decrypt_result,
+GError **err);
 
 #if (GMIME_MAJOR_VERSION < 3)
 notmuch_status_t
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 03/21] configure: session key handling in gmime maps to built_with("session_key")

2017-11-30 Thread Daniel Kahn Gillmor

This flag should make it easier to write the code for session-key
handling.

Note that this only works for GMime 2.6.21 and later (the session key
interface wasn't available before then).  It should be fine to build
the rest of notmuch if this functionality isn't available.

Note that this also adds the "session_key" built_with() aspect to
libnotmuch.
---
 configure| 14 ++
 lib/built-with.c |  2 ++
 2 files changed, 16 insertions(+)

diff --git a/configure b/configure
index cfbf827c..bd034f91 100755
--- a/configure
+++ b/configure
@@ -487,14 +487,21 @@ if pkg-config --exists "gmime-3.0 > $GMIME3_MINVER"; then
 gmime_cflags=$(pkg-config --cflags gmime-3.0)
 gmime_ldflags=$(pkg-config --libs gmime-3.0)
 gmime_major=3
+have_gmime_session_keys=1
 elif pkg-config --exists "gmime-2.6 >= $GMIME_MINVER"; then
 printf "Yes (2.6).\n"
 have_gmime=1
 gmime_cflags=$(pkg-config --cflags gmime-2.6)
 gmime_ldflags=$(pkg-config --libs gmime-2.6)
 gmime_major=2
+if pkg-config --exists "gmime-2.6 >= 2.6.21"; then
+have_gmime_session_keys=1
+else
+have_gmime_session_keys=0
+fi
 else
 have_gmime=0
+have_gmime_session_keys=0
 printf "No.\n"
 errors=$((errors + 1))
 fi
@@ -1097,6 +1104,9 @@ HAVE_TIMEGM = ${have_timegm}
 # Whether struct dirent has d_type (if not, then notmuch will use stat)
 HAVE_D_TYPE = ${have_d_type}
 
+# Whether the GMime version can handle extraction and reuse of session keys
+HAVE_GMIME_SESSION_KEYS = ${have_gmime_session_keys}
+
 # Whether the Xapian version in use supports compaction
 HAVE_XAPIAN_COMPACT = ${have_xapian_compact}
 
@@ -1182,6 +1192,7 @@ COMMON_CONFIGURE_CFLAGS = \\
-DHAVE_D_TYPE=\$(HAVE_D_TYPE)   \\
-DSTD_GETPWUID=\$(STD_GETPWUID) \\
-DSTD_ASCTIME=\$(STD_ASCTIME)   \\
+   -DHAVE_GMIME_SESSION_KEYS=\$(HAVE_GMIME_SESSION_KEYS)   \\
-DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT)   \\
-DSILENCE_XAPIAN_DEPRECATION_WARNINGS   \\
-DHAVE_XAPIAN_FIELD_PROCESSOR=\$(HAVE_XAPIAN_FIELD_PROCESSOR) \\
@@ -1210,6 +1221,9 @@ 
NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR=${have_xapian_field_processor}
 # Whether the Xapian version in use supports lock retry
 NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK=${have_xapian_db_retry_lock}
 
+# Whether the GMime version can handle extraction and reuse of session keys
+NOTMUCH_HAVE_GMIME_SESSION_KEYS=${have_gmime_session_keys}
+
 # Which backend will Xapian use by default?
 NOTMUCH_DEFAULT_XAPIAN_BACKEND=${default_xapian_backend}
 
diff --git a/lib/built-with.c b/lib/built-with.c
index 27384bd0..9cffd9f9 100644
--- a/lib/built-with.c
+++ b/lib/built-with.c
@@ -30,6 +30,8 @@ notmuch_built_with (const char *name)
return HAVE_XAPIAN_FIELD_PROCESSOR;
 } else if (STRNCMP_LITERAL (name, "retry_lock") == 0) {
return HAVE_XAPIAN_DB_RETRY_LOCK;
+} else if (STRNCMP_LITERAL (name, "session_key") == 0) {
+   return HAVE_GMIME_SESSION_KEYS;
 } else {
return false;
 }
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 16/21] cli/new, insert, reindex: change index.decrypt to "auto" by default

2017-11-30 Thread Daniel Kahn Gillmor

The new "auto" decryption policy is not only good for "notmuch show"
and "notmuch reindex".  It's also useful for indexing messages --
there's no good reason to not try to go ahead and index the cleartext
of a message that we have a stashed session key for.

This change updates the defaults and tunes the test suite to make sure
that they have taken effect.
---
 doc/man1/notmuch-config.rst   | 2 +-
 lib/indexopts.c   | 8 +---
 test/T357-index-decryption.sh | 4 ++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 4835f897..1a9e08a3 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -154,7 +154,7 @@ The available configuration items are described below.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
-Default: ``false``.
+Default: ``auto``.
 
 **built_with.**
 
diff --git a/lib/indexopts.c b/lib/indexopts.c
index a04d1c1c..26a31e89 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,7 +26,7 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
-ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_AUTO;
 
 char * decrypt_policy;
 notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt_policy);
@@ -38,8 +38,10 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
(!(strcasecmp(decrypt_policy, "yes"))) ||
(!(strcasecmp(decrypt_policy, "1"
notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
-   else if (!strcasecmp(decrypt_policy, "auto"))
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+   else if ((!(strcasecmp(decrypt_policy, "false"))) ||
+(!(strcasecmp(decrypt_policy, "no"))) ||
+(!(strcasecmp(decrypt_policy, "0"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_FALSE);
 }
 
 free (decrypt_policy);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 3157fd8c..b2717a7a 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -142,7 +142,7 @@ test_expect_equal \
 
 # ensure no session keys are present:
 test_begin_subtest 'reindex using only session keys'
-test_expect_success 'notmuch reindex --decrypt=auto tag:encrypted and 
property:index.decryption=success'
+test_expect_success 'notmuch reindex tag:encrypted and 
property:index.decryption=success'
 test_begin_subtest "reindexed encrypted messages, decrypting only with session 
keys"
 output=$(notmuch search wumpus)
 expected=''
@@ -190,7 +190,7 @@ notmuch restore

[PATCH v2 08/21] indexopts: change _try_decrypt to _decrypt_policy

2017-11-30 Thread Daniel Kahn Gillmor

This terminology makes it clearer what's going on at the API layer,
and paves the way for future changesets that offer more nuanced
decryption policy.
---
 lib/index.cc |  2 +-
 lib/indexopts.c  | 10 +-
 lib/notmuch.h|  8 
 notmuch-client.h |  4 ++--
 notmuch.c| 12 ++--
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 6eb60f30..ff14e408 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_try_decrypt (indexopts))
+if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index ca6bf6c9..0f65b97c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -36,24 +36,24 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
((!(strcasecmp(decrypt, "true"))) ||
 (!(strcasecmp(decrypt, "yes"))) ||
 (!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_try_decrypt (ret, true);
+   notmuch_indexopts_set_decrypt_policy (ret, true);
 
 free (decrypt);
 return ret;
 }
 
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt)
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
-indexopts->crypto.decrypt = try_decrypt;
+indexopts->crypto.decrypt = decrypt_policy;
 return NOTMUCH_STATUS_SUCCESS;
 }
 
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts)
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
return false;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 2c5dcab5..ef463090 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2244,17 +2244,17 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt);
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
- * see notmuch_indexopts_set_try_decrypt.
+ * see notmuch_indexopts_set_decrypt_policy.
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts);
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
  * Destroy a notmuch_indexopts_t object.
diff --git a/notmuch-client.h b/notmuch-client.h
index f7524e59..fc981459 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -500,8 +500,8 @@ int notmuch_minimal_options (const char* subcommand_name,
 /* the state chosen by the user invoking one of the notmuch
  * subcommands that does indexing */
 struct _notmuch_client_indexing_cli_choices {
-bool try_decrypt;
-bool try_decrypt_set;
+bool decrypt_policy;
+bool decrypt_policy_set;
 notmuch_indexopts_t * opts;
 };
 extern struct _notmuch_client_indexing_cli_choices indexing_cli_choices;
diff --git a/notmuch.c b/notmuch.c
index 5f5ac38a..e13f4b73 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -99,8 +99,8 @@ int notmuch_minimal_options (const char *subcommand_name,
 
 struct _notmuch_client_indexing_cli_choices indexing_cli_choices = { };
 const notmuch_opt_desc_t  notmuch_shared_indexing_options [] = {
-{ .opt_bool = &indexing_cli_choices.try_decrypt,
-  .present = &indexing_cli_choices.try_decrypt_set,
+{ .opt_bool = &indexing_cli_choices.decrypt_policy,
+  .present = &indexing_cli_choices.decrypt_policy_set,
   .name = "decrypt" },
 { }
 };
@@ -111,21 +111,21 @@ notmuch_process_shared_indexing_options 
(notmuch_database_t *notmuch, g_mime_3_u
 {
 if (indexing_cli_choices.opts == NULL)
indexing_cli_choices.opts = notmuch_database_get_default_indexopts 
(notmuch);
-if (indexing_cli_choices.try_decrypt_set) {
+if (indexing_cli_choices.decrypt_policy_set) {
notmuch_status_t status;
if (indexing_cli_choices.opts == NULL)
return NOTMUCH_STATUS_OUT_OF_MEMORY;
-   status = notmuch_indexopts_set_try_decrypt (indexing_cli_choices.opts, 
indexing_cli_choices.try_decrypt);
+   status = notmuch_indexopts_set_decrypt_policy 
(indexing_cli_choices.opts, indexing_cli_choices.decrypt_policy);
if (status != NOTMUCH_STATUS_SUCCESS) {
fprintf (stderr, "Error: Failed to set index decryption policy to 
%s.

[PATCH v2 20/21] docs: clean up documentation about decryption policies

2017-11-30 Thread Daniel Kahn Gillmor

Now that the range of sensible decryption policies has come into full
view, we take a bit of space to document the distinctions.

Most people will use either "auto" or "true" -- but we provide "false"
and "nostash" to handle use cases that might reasonably be requested.

Note also that these can be combined in sensible ways.  Like, if your
mail comes in regularly to a service that doesn't have access to your
secret keys, but does have access to your index, and you feel
comfortable adding selected encrypted messages to the index after
you've read them, you could stay in "auto" normally, and then when you
find yourself reading an indexable message (e.g. one you want to be
able to search for in the future, and that you don't mind exposing to
whatever entities have access to your inde), you can do:

notmuch reindex --decrypt=true id:whate...@example.biz

That leaves your default the same (still "auto") but you get the
cleartext index and stashed session key benefits for that particular
message.
---
 doc/man1/notmuch-config.rst | 33 -
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 773fd9da..3ba849b2 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,7 +142,9 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 
-One of ``false``, ``auto``, ``nostash``, or ``true``.
+Policy for decrypting encrypted messages during indexing.
+Must be one of: ``false``, ``auto``, ``nostash``, or
+``true``.
 
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
@@ -156,6 +158,34 @@ The available configuration items are described below.
 ``nostash`` is the same as ``true`` except that it will not
 stash newly-discovered session keys in the database.
 
+From the command line (i.e. during **notmuch-new(1)**,
+**notmuch-insert(1)**, or **notmuch-reindex(1)**), the user
+can override the database's stored decryption policy with the
+``--decrypt=`` option.
+
+Here is a table that summarizes the functionality of each of
+these policies:
+
+++---+--+-+--+
+|| false | auto | nostash | true |
+++===+==+=+==+
+| Index cleartext using  |   |  X   |X|  X   |
+| stashed session keys   |   |  | |  |
+++---+--+-+--+
+| Index cleartext|   |  |X|  X   |
+| using secret keys  |   |  | |  |
+++---+--+-+--+
+| Stash session keys |   |  | |  X   |
+++---+--+-+--+
+| Delete stashed session |   X   |  | |  |
+| keys on reindex|   |  | |  |
+++---+--+-+--+
+
+Stashed session keys are kept in the database as properties
+associated with the message.  See ``session-key`` in
+**notmuch-properties(7)** for more details about how they can
+be useful.
+
 Be aware that the notmuch index is likely sufficient (and a
 stashed session key is certainly sufficient) to reconstruct
 the cleartext of the message itself, so please ensure that the
@@ -201,5 +231,6 @@ SEE ALSO
 **notmuch-restore(1)**,
 **notmuch-search(1)**,
 **notmuch-search-terms(7)**,
+**notmuch-properties(7)**,
 **notmuch-show(1)**,
 **notmuch-tag(1)**
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 11/21] cli/reply: use decryption policy "auto" by default.

2017-11-30 Thread Daniel Kahn Gillmor

If the user doesn't specify --decrypt= at all, but a stashed session
key is known to notmuch, when replying to an encrypted message,
notmuch should just go ahead and decrypt.

The user can disable this at the command line with --decrypt=false,
though it's not clear why they would ever want to do that.
---
 completion/notmuch-completion.bash | 6 +-
 doc/man1/notmuch-reply.rst | 6 +-
 notmuch-reply.c| 9 +
 test/T357-index-decryption.sh  | 7 +++
 4 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index e462a82a..1cd616b3 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -350,12 +350,16 @@ _notmuch_reply()
COMPREPLY=( $( compgen -W "all sender" -- "${cur}" ) )
return
;;
+   --decrypt)
+   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
 case "${cur}" in
-*)
-   local options="--format= --format-version= --reply-to= --decrypt 
${_notmuch_shared_options}"
+   local options="--format= --format-version= --reply-to= --decrypt= 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index b6aec3c8..ede77930 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -80,8 +80,12 @@ Supported options for **reply** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
+needed credentials. Without one, the decryption will likely fail.
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/notmuch-reply.c b/notmuch-reply.c
index eec34bed..fd990a9a 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -700,11 +700,12 @@ notmuch_reply_command (notmuch_config_t *config, int 
argc, char *argv[])
 int opt_index;
 notmuch_show_params_t params = {
.part = -1,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -718,7 +719,7 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = &decrypt, .name = "decrypt" },
+   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -728,8 +729,8 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt)
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+if (decrypt_set)
+   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_TRUE : 
NOTMUCH_DECRYPT_FALSE;
 
 notmuch_exit_if_unsupported_format ();
 
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 2901c390..06008e81 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -197,6 +197,13 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch reply should show cleartext if session key is 
present"
+output=$(notmuch reply id:simple-encryp...@crypto.notmuchmail.org | grep '^>')
+expected='> This is a top sekrit message.'
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 12/21] cli/show: use decryption policy "auto" by default.

2017-11-30 Thread Daniel Kahn Gillmor

When showing a message, if the user doesn't specify --decrypt= at all,
but a stashed session key is known to notmuch, notmuch should just go
ahead and try to decrypt the message with the session key (without
bothering the user for access to their asymmetric secret key).

The user can disable this at the command line with --decrypt=false if
they really don't want to look at the e-mail that they've asked
notmuch to show them.

and of course, "notmuch show --decrypt" still works for accessing the
user's secret keys if necessary.
---
 completion/notmuch-completion.bash |  4 ++--
 doc/man1/notmuch-show.rst  |  4 
 notmuch-show.c | 17 +++--
 test/T357-index-decryption.sh  | 14 ++
 4 files changed, 31 insertions(+), 8 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 1cd616b3..f94dbeed 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,7 +517,7 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body)
+   --exclude|--body|--decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -526,7 +526,7 @@ _notmuch_show()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt --include-html 
${_notmuch_shared_options}"
+   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt= --include-html 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 7ba091cf..64caa7a6 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -123,6 +123,10 @@ Supported options for **show** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
 needed credentials. Without one, the decryption will fail.
 
diff --git a/notmuch-show.c b/notmuch-show.c
index e840a470..591889a9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1105,13 +1105,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
.part = -1,
.omit_excluded = true,
.output_body = true,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = NOTMUCH_FORMAT_NOT_SPECIFIED;
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = &format, .name = "format", .keywords =
@@ -1126,7 +1127,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
{ .opt_bool = ¶ms.entire_thread, .name = "entire-thread",
  .present = &entire_thread_set },
{ .opt_int = ¶ms.part, .name = "part" },
-   { .opt_bool = &decrypt, .name = "decrypt" },
+   { .opt_bool = &decrypt, .name = "decrypt", .present = &decrypt_set },
{ .opt_bool = ¶ms.crypto.verify, .name = "verify" },
{ .opt_bool = ¶ms.output_body, .name = "body" },
{ .opt_bool = ¶ms.include_html, .name = "include-html" },
@@ -1140,10 +1141,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt) {
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
-   /* decryption implies verification */
-   params.crypto.verify = true;
+if (decrypt_set) {
+   if (decrypt) {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+   /* decryption implies verification */
+   params.crypto.verify = true;
+   } else {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+   }
 }
 
 /* specifying a part implies single message display */
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 06008e81..3157fd8c 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -204,6 +204,20 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch show should show cleartext if session key is 
present"
+output=$(notmuch show id:simple-encryp...@crypto.notmuchmail.org | awk 
'/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
+expected='This is a top sekrit message.'
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtes

[PATCH v2 10/21] crypto: new decryption policy "auto"

2017-11-30 Thread Daniel Kahn Gillmor

This new automatic decryption policy should make it possible to
decrypt messages that we have stashed session keys for, without
incurring a call to the user's asymmetric keys.
---
 doc/man1/notmuch-config.rst   | 11 ---
 lib/index.cc  |  3 ++-
 lib/indexopts.c   | 13 -
 lib/notmuch.h |  1 +
 mime-node.c   |  7 ---
 notmuch-client.h  |  4 +++-
 notmuch.c |  3 ++-
 test/T357-index-decryption.sh | 12 +++-
 util/crypto.c |  9 -
 util/crypto.h |  3 ++-
 10 files changed, 49 insertions(+), 17 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index ea3d9754..4835f897 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,9 +142,14 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
-set to true, notmuch will try to decrypt the message and index
-the cleartext.  Be aware that the index is likely sufficient
-to reconstruct the cleartext of the message itself, so please
+set to ``true``, notmuch will try to decrypt the message and
+index the cleartext.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message,
+but will not try to access your secret keys.  Use ``false`` to
+avoid decrypting even when a session key is already known.
+
+Be aware that the notmuch index is likely sufficient to
+reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
diff --git a/lib/index.cc b/lib/index.cc
index 905366ae..af999bd3 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,7 +548,8 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (message, crypto_ctx, encrypted_data, 
NULL, &err);
+clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+message, crypto_ctx, encrypted_data, NULL, 
&err);
 if (err) {
_notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
   err->domain, err->code, err->message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 78f53391..a04d1c1c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -33,11 +33,14 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 if (err)
return ret;
 
-if (decrypt_policy &&
-   ((!(strcasecmp(decrypt_policy, "true"))) ||
-(!(strcasecmp(decrypt_policy, "yes"))) ||
-(!(strcasecmp(decrypt_policy, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+if (decrypt_policy) {
+   if ((!(strcasecmp(decrypt_policy, "true"))) ||
+   (!(strcasecmp(decrypt_policy, "yes"))) ||
+   (!(strcasecmp(decrypt_policy, "1"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+   else if (!strcasecmp(decrypt_policy, "auto"))
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+}
 
 free (decrypt_policy);
 return ret;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 47633496..ff860e06 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2241,6 +2241,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
 typedef enum {
 NOTMUCH_DECRYPT_FALSE,
 NOTMUCH_DECRYPT_TRUE,
+NOTMUCH_DECRYPT_AUTO,
 } notmuch_decryption_policy_t;
 
 /**
diff --git a/mime-node.c b/mime-node.c
index c4de708b..49d668fe 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -205,7 +205,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
break;
 
node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt (parent ? 
parent->envelope_file : NULL,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, &decrypt_result, &err);
 }
 if (! node->decrypted_child) {
@@ -270,7 +271,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt != 
NOTMUCH_DECRYPT_FALSE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_c

[PATCH v2 09/21] lib: convert notmuch decryption policy to an enum

2017-11-30 Thread Daniel Kahn Gillmor

Future patches in this series will introduce new policies; this merely
readies the way for them.

We also convert --try-decrypt to a keyword argument instead of a boolean.
---
 lib/index.cc  |  2 +-
 lib/indexopts.c   | 21 +++--
 lib/notmuch.h | 14 --
 mime-node.c   |  4 ++--
 notmuch-client.h  |  9 +
 notmuch-reply.c   |  6 +-
 notmuch-show.c| 10 +++---
 notmuch.c | 13 -
 test/T357-index-decryption.sh | 12 ++--
 util/crypto.h |  2 +-
 10 files changed, 58 insertions(+), 35 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index ff14e408..905366ae 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
+if (!indexopts || (notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 0f65b97c..78f53391 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,25 +26,26 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
 
-char * decrypt;
-notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt);
+char * decrypt_policy;
+notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
&decrypt_policy);
 if (err)
return ret;
 
-if (decrypt &&
-   ((!(strcasecmp(decrypt, "true"))) ||
-(!(strcasecmp(decrypt, "yes"))) ||
-(!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, true);
+if (decrypt_policy &&
+   ((!(strcasecmp(decrypt_policy, "true"))) ||
+(!(strcasecmp(decrypt_policy, "yes"))) ||
+(!(strcasecmp(decrypt_policy, "1")
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
 
-free (decrypt);
+free (decrypt_policy);
 return ret;
 }
 
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy)
+ notmuch_decryption_policy_t 
decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
@@ -52,7 +53,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
 return NOTMUCH_STATUS_SUCCESS;
 }
 
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index ef463090..47633496 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2233,6 +2233,16 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 notmuch_indexopts_t *
 notmuch_database_get_default_indexopts (notmuch_database_t *db);
 
+/**
+ * Stating a policy about how to decrypt messages.
+ *
+ * See index.decrypt in notmuch-config(1) for more details.
+ */
+typedef enum {
+NOTMUCH_DECRYPT_FALSE,
+NOTMUCH_DECRYPT_TRUE,
+} notmuch_decryption_policy_t;
+
 /**
  * Specify whether to decrypt encrypted parts while indexing.
  *
@@ -2245,7 +2255,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  */
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy);
+ notmuch_decryption_policy_t 
decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
@@ -2253,7 +2263,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
diff --git a/mime-node.c b/mime-node.c
index daaae9f8..c4de708b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -270,7 +270,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && node->ctx->crypto->decrypt)
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_content_type (part);
const char *protocol = g_mime_content_type_get_parameter (content_type, 
"protocol");
@@ -286,7 +286,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 #endif
 
 /* Handle P

[PATCH v2 14/21] cli/new, insert, reindex: update documentation for --decrypt=auto

2017-11-30 Thread Daniel Kahn Gillmor

we also include --decrypt=auto in the tab completion.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   | 10 +++---
 doc/man1/notmuch-reindex.rst   | 23 ++-
 4 files changed, 34 insertions(+), 21 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index f94dbeed..272131e6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index eb9ff11b..b22be863 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,14 +51,18 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true and the message is encrypted, try to decrypt the
-message while indexing.  If decryption is successful, index
+If ``true`` and the message is encrypted, try to decrypt the
+message while indexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If decryption is successful, index
 the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).  Be aware
-that the index is likely sufficient to reconstruct the
-cleartext of the message itself, so please ensure that the
+stored to disk in its original form (ciphertext).
+
+Be aware that the index is likely sufficient to reconstruct
+the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
 ``--decrypt=true`` without considering the security of
 your index.
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 1df86f06..71df31d7 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -43,11 +43,15 @@ Supported options for **new** include
 ``--quiet``
 Do not print progress or results.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true, when encountering an encrypted message, try to
+If ``true``, when encountering an encrypted message, try to
 decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  Be aware that the index is likely
+the cleartext itself.  If ``auto``, try to use any session key
+already known to belong to this message, but do not attempt to
+use the user's secret keys.
+
+Be aware that the index is likely
 sufficient to reconstruct the cleartext of the message itself,
 so please ensure that the notmuch message index is adequately
 protected.  DO NOT USE ``--decrypt=true`` without
diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index 782b0d7b..d87e9d85 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -21,15 +21,20 @@ messages using the supplied options.
 
 Supported options for **reindex** include
 
-``--decrypt=(true|false)``
-
-If true, when encountering an encrypted message, try to
-decrypt it while reindexing.  If decryption is successful,
-index the cleartext itself.  Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected. DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+``--decrypt=(true|auto|false)``
+
+If ``true``, when encountering an encrypted message, try to
+decrypt it while reindexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If decrypti

[PATCH v2 15/21] crypto: record whether an actual decryption attempt happened

2017-11-30 Thread Daniel Kahn Gillmor

In our consolidation of _notmuch_crypto_decrypt, the callers lost
track a little bit of whether any actual decryption was attempted.

Now that we have the more-subtle "auto" policy, it's possible that
_notmuch_crypto_decrypt could be called without having any actual
decryption take place.

This change lets the callers be a little bit smarter about whether or
not any decryption was actually attempted.
---
 lib/index.cc  | 17 -
 mime-node.c   |  4 ++--
 util/crypto.c |  7 ++-
 util/crypto.h |  3 ++-
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index af999bd3..3914012a 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,12 +548,19 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+bool attempted = false;
+clear = _notmuch_crypto_decrypt (&attempted, 
notmuch_indexopts_get_decrypt_policy (indexopts),
 message, crypto_ctx, encrypted_data, NULL, 
&err);
-if (err) {
-   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
-  err->domain, err->code, err->message);
-   g_error_free(err);
+if (!attempted)
+   return;
+if (err || !clear) {
+   if (err) {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
+  err->domain, err->code, err->message);
+   g_error_free(err);
+   } else {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(unknown error)\n");
+   }
/* Indicate that we failed to decrypt during indexing */
status = notmuch_message_add_property (message, "index.decryption", 
"failure");
if (status)
diff --git a/mime-node.c b/mime-node.c
index 49d668fe..11df082b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -204,8 +204,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
if (parent->envelope_file)
break;
 
-   node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(&node->decrypt_attempted,
+
node->ctx->crypto->decrypt,
 parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, &decrypt_result, &err);
 }
diff --git a/util/crypto.c b/util/crypto.c
index bb587571..338f1d5d 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -140,7 +140,8 @@ void _notmuch_crypto_cleanup (unused(_notmuch_crypto_t 
*crypto))
 #endif
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 g_mime_3_unused(GMimeCryptoContext* crypto_ctx),
 GMimeMultipartEncrypted *part,
@@ -162,6 +163,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
g_error_free (*err);
*err = NULL;
}
+   if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
ret = g_mime_multipart_encrypted_decrypt_session (part,
  crypto_ctx,
@@ -191,6 +194,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
 if (decrypt == NOTMUCH_DECRYPT_AUTO)
return ret;
 
+if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
 ret = g_mime_multipart_encrypted_decrypt(part, crypto_ctx,
 decrypt_result, err);
diff --git a/util/crypto.h b/util/crypto.h
index dc95b4ca..c384601c 100644
--- a/util/crypto.h
+++ b/util/crypto.h
@@ -16,7 +16,8 @@ typedef struct _notmuch_crypto {
 } _notmuch_crypto_t;
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 GMimeCryptoContext* crypto_ctx,
 GMimeMultipartEncrypted *part,
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

session keys, version 2

2017-11-30 Thread Daniel Kahn Gillmor

This is the second revision of the session keys series.  the earlier
version of this series can be found following
id:20171025065203.24403-1-...@fifthhorseman.net.

This version addresses the ideas and critiques raised on list about
the first version.

In particular:

 * ./configure now detects and sets a HAVE_GMIME_SESSION_KEYS #define
   to consolidate the version number checking in one place.

 * the library contains a built_with("session_key") marker.

 * the command line syntax for new, insert, and reindex is now
   --decrypt=, instead of --try-decrypt=

 * the configuration option is now index.decrypt, instead of
   index.try_decrypt.

 * in alignment with the shift to a "decryption policy" vocabulary,
   i've changed notmuch_indexopts_{set,get}_try_decrypt to
   notmuch_indexopts_{set,get}_decrypt_policy.  I think this is OK
   because the API has not been released yet.

The decryption policies remain the same:

   ++---+--+-+--+
   || false | auto | nostash | true |
   ++===+==+=+==+
   | Index cleartext using  |   |  X   |X|  X   |
   | stashed session keys   |   |  | |  |
   ++---+--+-+--+
   | Index cleartext|   |  |X|  X   |
   | using secret keys  |   |  | |  |
   ++---+--+-+--+
   | Stash session keys |   |  | |  X   |
   ++---+--+-+--+
   | Delete stashed session |   X   |  | |  |
   | keys on reindex|   |  | |  |
   ++---+--+-+--+

I believe this addresses all the major concerns raised about the
earlier draft of this series, and i know that there are at least a few
people other than myself currently using this series.

Please let me know if you have any other feedback about this new
revision!  I'd love to try to land this in 0.26.

 --dkg

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 13/21] cli/show, reply: document use of stashed session keys in notmuch-properties

2017-11-30 Thread Daniel Kahn Gillmor

The stashed session keys are stored internally as notmuch properties.
So a user or developer who is reading about those properties might
want to understand how they fit into the bigger picture.

Note here that decrypting with a stored session key no longer needs
-decrypt for "notmuch show" and "notmuch reply".
---
 doc/man7/notmuch-properties.rst | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst
index f27dd607..1bb7b1ca 100644
--- a/doc/man7/notmuch-properties.rst
+++ b/doc/man7/notmuch-properties.rst
@@ -77,9 +77,12 @@ of its normal activity.
 **session-key**
 
 When **notmuch-show(1)** or **nomtuch-reply** encounters a message
-with an encrypted part and ``--decrypt`` is set, if notmuch finds a
-``session-key`` property associated with the message, it will try
-that stashed session key for decryption.
+with an encrypted part, if notmuch finds a ``session-key``
+property associated with the message, it will try that stashed
+session key for decryption.
+
+If you do not want to use any stashed session keys that might be
+present, you should pass those programs ``--decrypt=false``.
 
 Using a stashed session key with "notmuch show" will speed up
 rendering of long encrypted threads.  It also allows the user to
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 19/21] crypto: add --decrypt=nostash to avoid stashing session keys

2017-11-30 Thread Daniel Kahn Gillmor

Here's the configuration choice for people who want a cleartext index,
but don't want stashed session keys.

Interestingly, this "nostash" decryption policy is actually the same
policy that should be used by "notmuch show" and "notmuch reply",
since they never modify the index or database when they are invoked
with --decrypt.

We take advantage of this parallel to tune the behavior of those
programs so that we're not requesting session keys from GnuPG during
"show" and "reply" that we would then otherwise just throw away.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-config.rst| 17 -
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   |  7 ---
 doc/man1/notmuch-reindex.rst   | 16 ++--
 lib/indexopts.c|  2 ++
 lib/notmuch.h  |  1 +
 notmuch-reply.c|  2 +-
 notmuch-show.c |  3 ++-
 notmuch.c  |  1 +
 test/T357-index-decryption.sh  | 23 +++
 util/crypto.c  |  4 ++--
 12 files changed, 71 insertions(+), 27 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 272131e6..948c153b 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index dabf269f..773fd9da 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -141,6 +141,9 @@ The available configuration items are described below.
 **index.decrypt**
 
 **[STORED IN DATABASE]**
+
+One of ``false``, ``auto``, ``nostash``, or ``true``.
+
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
 index the cleartext, stashing a copy of any discovered session
@@ -150,11 +153,15 @@ The available configuration items are described below.
 secret keys.  Use ``false`` to avoid decrypting even when a
 stashed session key is already present.
 
-Be aware that the notmuch index is likely sufficient to
-reconstruct the cleartext of the message itself, so please
-ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.decrypt=true`` without considering the
-security of your index.
+``nostash`` is the same as ``true`` except that it will not
+stash newly-discovered session keys in the database.
+
+Be aware that the notmuch index is likely sufficient (and a
+stashed session key is certainly sufficient) to reconstruct
+the cleartext of the message itself, so please ensure that the
+notmuch message index is adequately protected.  DO NOT USE
+``index.decrypt=true`` or ``index.decrypt=nostash`` without
+considering the security of your index.
 
 Default: ``auto``.
 
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index 214f261b..1a3dfe98 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,10 +51,10 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|auto|false)``
+``--decrypt=(true|nostash|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing, storing any session keys discovered.
+message while indexing, stashing any session keys discovered.
 If ``auto``, and notmuch already knows about a session key for
 the message, it will try decrypting using that session key but
 will not try to access the user's secret keys.  If decryption
@@ -62,11 +62,15 @@ Supported options for **insert** include
 message is always stored to disk in its original form
 (ciphertext).
 
-Be aware that the index is likely sufficient to reconstruct
-the cleartext of the message itself, so please ensure that the
+

[PATCH v2 18/21] crypto: actually stash session keys when decrypt=true

2017-11-30 Thread Daniel Kahn Gillmor

If you're going to store the cleartext index of an encrypted message,
in most situations you might just as well store the session key.
Doing this storage has efficiency and recoverability advantages.

Combined with a schedule of regular OpenPGP subkey rotation and
destruction, this can also offer security benefits, like "deletable
e-mail", which is the store-and-forward analog to "forward secrecy".

But wait, i hear you saying, i have a special need to store cleartext
indexes but it's really bad for me to store session keys!  Maybe
(let's imagine) i get lots of e-mails with incriminating photos
attached, and i want to be able to search for them by the text in the
e-mail, but i don't want someone with access to the index to be
actually able to see the photos themselves.

Fret not, the next patch in this series will support your wacky
uncommon use case.
---
 doc/man1/notmuch-config.rst | 10 ++
 doc/man1/notmuch-insert.rst | 13 +++--
 doc/man1/notmuch-new.rst| 21 +++--
 doc/man1/notmuch-reindex.rst| 10 +-
 doc/man7/notmuch-properties.rst |  4 
 lib/index.cc| 18 +-
 test/T357-index-decryption.sh   | 18 +-
 util/crypto.c   | 16 +++-
 8 files changed, 82 insertions(+), 28 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 1a9e08a3..dabf269f 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -143,10 +143,12 @@ The available configuration items are described below.
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
-index the cleartext.  If ``auto``, it will try to index the
-cleartext if a stashed session key is already known for the message,
-but will not try to access your secret keys.  Use ``false`` to
-avoid decrypting even when a session key is already known.
+index the cleartext, stashing a copy of any discovered session
+keys for the message.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message
+(e.g. from a previous copy), but will not try to access your
+secret keys.  Use ``false`` to avoid decrypting even when a
+stashed session key is already present.
 
 Be aware that the notmuch index is likely sufficient to
 reconstruct the cleartext of the message itself, so please
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index b22be863..214f261b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -54,12 +54,13 @@ Supported options for **insert** include
 ``--decrypt=(true|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing.  If ``auto``, and notmuch already
-knows about a session key for the message, it will try
-decrypting using that session key but will not try to access
-the user's secret keys.  If decryption is successful, index
-the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).
+message while indexing, storing any session keys discovered.
+If ``auto``, and notmuch already knows about a session key for
+the message, it will try decrypting using that session key but
+will not try to access the user's secret keys.  If decryption
+is successful, index the cleartext itself.  Either way, the
+message is always stored to disk in its original form
+(ciphertext).
 
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 71df31d7..27676a19 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -46,16 +46,17 @@ Supported options for **new** include
 ``--decrypt=(true|auto|false)``
 
 If ``true``, when encountering an encrypted message, try to
-decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  If ``auto``, try to use any session key
-already known to belong to this message, but do not attempt to
-use the user's secret keys.
-
-Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected.  DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+decrypt it while indexing, and store any discovered session
+keys.  If ``auto``, try to use any session key already known
+to belong to this message, but do not attempt to use the
+

[PATCH v2 07/21] indexing: Change from try_decrypt to decrypt

2017-11-30 Thread Daniel Kahn Gillmor

the command-line interface for indexing (reindex, new, insert) used
--try-decrypt; and the configuration records used index.try_decrypt.
But by comparison with "show" and "reply", there doesn't seem to be
any reason for the "try" prefix.

This changeset adjusts the command-line interface and the
configuration interface.

For the moment, i've left indexopts_{set,get}_try_decrypt alone.  The
subsequent changeset will address those.
---
 NEWS   |  4 ++--
 completion/notmuch-completion.bash | 12 ++--
 doc/man1/notmuch-config.rst|  4 ++--
 doc/man1/notmuch-insert.rst|  6 +++---
 doc/man1/notmuch-new.rst   |  6 +++---
 doc/man1/notmuch-reindex.rst   |  6 +++---
 doc/man7/notmuch-properties.rst|  2 +-
 lib/indexopts.c| 14 +++---
 notmuch-config.c   |  2 +-
 notmuch.c  |  4 ++--
 test/T357-index-decryption.sh  | 18 +-
 test/test-lib.sh   |  2 +-
 12 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/NEWS b/NEWS
index 412c678d..0465b9e8 100644
--- a/NEWS
+++ b/NEWS
@@ -19,9 +19,9 @@ Indexing cleartext of encrypted e-mails
   It's now possible to include the cleartext of encrypted e-mails in
   the notmuch index.  This makes it possible to search your encrypted
   e-mails with the same ease as searching cleartext.  This can be done
-  on a per-message basis with the --try-decrypt argument to indexing
+  on a per-message basis with the --decrypt argument to indexing
   commands (new, insert, reindex), or by default by running "notmuch
-  config set index.try_decrypt true".
+  config set index.decrypt true".
 
   Note that the contents of the index are sufficient to roughly
   reconstruct the cleartext of the message itself, so please ensure
diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 7aae4297..e462a82a 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -287,7 +287,7 @@ _notmuch_insert()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
return
;;
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -296,7 +296,7 @@ _notmuch_insert()
 ! $split &&
 case "${cur}" in
--*)
-   local options="--create-folder --folder= --keep --no-hooks 
--try-decrypt= ${_notmuch_shared_options}"
+   local options="--create-folder --folder= --keep --no-hooks 
--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
return
@@ -319,7 +319,7 @@ _notmuch_new()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -328,7 +328,7 @@ _notmuch_new()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--no-hooks --try-decrypt= --quiet 
${_notmuch_shared_options}"
+   local options="--no-hooks --decrypt= --quiet 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "${options}" -- ${cur}) )
;;
@@ -437,7 +437,7 @@ _notmuch_reindex()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -446,7 +446,7 @@ _notmuch_reindex()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--try-decrypt= ${_notmuch_shared_options}"
+   local options="--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 6961737f..ea3d9754 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -138,7 +138,7 @@ The available configuration items are described below.
 
 Default: ``gpg``.
 
-**index.try_decrypt**
+**index.decrypt**
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
@@ -146,7 +146,7 @@ The available configuration items are described below.
 the cleartext.  Be aware that the index is likely sufficient
 to reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.try_decrypt=true`` without considering the
+DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
 Default: ``false``.
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index c500b251..eb9ff11b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rs

[PATCH v2 04/21] crypto: use stashed session-key properties for decryption, if available

2017-11-30 Thread Daniel Kahn Gillmor

When doing any decryption, if the notmuch database knows of any
session keys associated with the message in question, try them before
defaulting to using default symmetric crypto.

This changeset does the primary work in _notmuch_crypto_decrypt, which
grows some new parameters to handle it.

The primary advantage this patch offers is a significant speedup when
rendering large encrypted threads ("notmuch show") if session keys
happen to be cached.

Additionally, it permits message composition without access to
asymmetric secret keys ("notmuch reply"); and it permits recovering a
cleartext index when reindexing after a "notmuch restore" for those
messages that already have a session key stored.

Note that we may try multiple decryptions here (e.g. if there are
multiple session keys in the database), but we will ignore and throw
away all the GMime errors except for those that come from last
decryption attempt.  Since we don't necessarily know at the time of
the decryption that this *is* the last decryption attempt, we'll ask
for the errors each time anyway.

This does nothing if no session keys are stashed in the database,
which is fine.  Actually stashing session keys in the database will
come as a subsequent patch.
---
 doc/man7/notmuch-properties.rst | 31 +++
 lib/index.cc|  2 +-
 mime-node.c | 13 ++---
 util/crypto.c   | 39 ++-
 util/crypto.h   |  3 ++-
 5 files changed, 82 insertions(+), 6 deletions(-)

diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst
index 68121359..49602b73 100644
--- a/doc/man7/notmuch-properties.rst
+++ b/doc/man7/notmuch-properties.rst
@@ -74,6 +74,35 @@ of its normal activity.
 **notmuch-config(1)**), then this property will not be set on that
 message.
 
+**session-key**
+
+When **notmuch-show(1)** or **nomtuch-reply** encounters a message
+with an encrypted part and ``--decrypt`` is set, if notmuch finds a
+``session-key`` property associated with the message, it will try
+that stashed session key for decryption.
+
+Using a stashed session key with "notmuch show" will speed up
+rendering of long encrypted threads.  It also allows the user to
+destroy the secret part of any expired encryption-capable subkey
+while still being able to read any retained messages for which
+they have stashed the session key.  This enables truly deletable
+e-mail, since (once the session key and asymmetric subkey are both
+destroyed) there are no keys left that can be used to decrypt any
+copy of the original message previously stored by an adversary.
+
+However, access to the stashed session key for an encrypted message
+permits full byte-for-byte reconstruction of the cleartext
+message.  This includes attachments, cryptographic signatures, and
+other material that cannot be reconstructed from the index alone.
+
+The session key should be in the ASCII text form produced by
+GnuPG.  For OpenPGP, that consists of a decimal representation of
+the hash algorithm used (identified by number from RFC 4880,
+e.g. 9 means AES-256) followed by a colon, followed by a
+hexidecimal representation of the algorithm-specific key.  For
+example, an AES-128 key might be stashed in a notmuch property as:
+``session-key=7:14B16AF65536C28AF209828DFE34C9E0``.
+
 SEE ALSO
 
 
@@ -83,5 +112,7 @@ SEE ALSO
 **notmuch-insert(1)**,
 **notmuch-new(1)**,
 **notmuch-reindex(1)**,
+**notmuch-reply(1)**,
 **notmuch-restore(1)**,
+**notmuch-show(1)**,
 ***notmuch-search-terms(7)**
diff --git a/lib/index.cc b/lib/index.cc
index 19d03456..6eb60f30 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,7 +548,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (crypto_ctx, encrypted_data, NULL, &err);
+clear = _notmuch_crypto_decrypt (message, crypto_ctx, encrypted_data, 
NULL, &err);
 if (err) {
_notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
   err->domain, err->code, err->message);
diff --git a/mime-node.c b/mime-node.c
index ae3ebef6..daaae9f8 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -198,9 +198,16 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
 GMimeDecryptResult *decrypt_result = NULL;
 GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 
-node->decrypt_attempted = true;
-if (! node->decrypted_child)
-   node->decrypted_child = _notmuch_crypto_decrypt (cryptoctx, 
encrypteddata, &decrypt_result, &err);
+if (! node->decrypted_child) {
+   mime_node_t *parent;
+   for (parent = node; parent; parent = parent->parent)
+   if (parent->envelope_file)
+   break;
+
+   node->decrypt_attempted = true;
+   n

[PATCH v2 05/21] test/corpora: add an encrypted message for index decryption tests

2017-11-30 Thread Daniel Kahn Gillmor

---
 test/corpora/README  |  5 +
 test/corpora/crypto/simple-encrypted | 36 
 2 files changed, 41 insertions(+)
 create mode 100644 test/corpora/crypto/simple-encrypted

diff --git a/test/corpora/README b/test/corpora/README
index c9a35fed..05ea386d 100644
--- a/test/corpora/README
+++ b/test/corpora/README
@@ -12,3 +12,8 @@ broken
 
 html
   The html corpus contains html parts
+
+crypto
+  The crypto corpus contains encrypted messages for testing.
+  It should probably also contain signed messages in the future.
+  Please add them!
diff --git a/test/corpora/crypto/simple-encrypted 
b/test/corpora/crypto/simple-encrypted
new file mode 100644
index ..6869972d
--- /dev/null
+++ b/test/corpora/crypto/simple-encrypted
@@ -0,0 +1,36 @@
+From: Daniel Kahn Gillmor 
+To: d...@fifthhorseman.net
+Subject: encrypted message
+Date: Mon, 22 Dec 2016 08:34:56 -0400
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/encrypted; boundary="=-=-=";
+   protocol="application/pgp-encrypted"
+
+--=-=-=
+Content-Type: application/pgp-encrypted
+
+Version: 1
+
+--=-=-=
+Content-Type: application/octet-stream
+
+-BEGIN PGP MESSAGE-
+
+hQIMAzt6p/AU5ptaAQ/+IDRx5dZWKjUz9qITP76w8OvtmTV9p871UoGWil1DSd3J
+dHgf56rXDWS73dzJ5EigevxLVMD3Xv8QEJBgMWb6yC+uR8ZdJ8h7hlE2lYyEg3Ch
+smqzcaYp2nKWw9JZQqubsMaeIgVu0exb4YE8g/qlUOL2mD64dXhnkJ68GGMmiEPJ
++d0H7fTMwstbxvKPKDmFJUztH43V2NSfxpeQUTi4iWmQtUGw6THYjjAWNFy7jVEE
+ozloT9W3ER5cRaXyKE4GWMBlUAOB0YwwsVnBU2JGUtTBzNHxQAbeoKrG6myrzmnC
+LhUNag0gMuNEbGR78XfWKpCbccEC1VLf9uUXze6yeuRXDuhfsvilKOH9MpaR9n+G
+JuYVAAobDc5wGOt5VGMka50ToaALrNt3FrWqni/0jqwqshEVKM3Kqzq6cSjh1TPf
+pfxE9aUDdvf+Nn16ZBGgyLox2NV4GkSQYq2ySzAk3XwLU80F0nCmtOV3EH+OMrv6
+sZI/Svwzaqzrs/w17cvpf0czXjE/N9V1MHdNtIkfb707WkO0l9/wtYvlrg/KyrU2
+FH5aecEO9VpMumgzBqP1MrrnzVlSM3kgRLIu06oshQYD+jjFn2YzvkwZ+pWoAxsQ
+ninQxoF8Ck2D7s8uGzx+HSQQpRVBM5AGfVdEkmzW/sZrlz63ZGUFh0FYqLl30rfS
+cQGqSoZz0ugTSxnTlg0nuzFmG7ylC1cx3dlSrnfv+l1azwLAr58ptoYZ0mO+D+Fy
+bePpCGoFAPKi0cZ/4eFlLKL7uYPmGeEo5Ku2wXU/SXtPfl4vRxzvPXTJTvD06vIx
+Dh1P0ChAJtxUjGGY6xkCHMY0
+=frmz
+-END PGP MESSAGE-
+--=-=-=--
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 21/21] python: add decrypt_policy argument to Database.index_file()

2017-11-30 Thread Daniel Kahn Gillmor

We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.
---
 bindings/python/notmuch/database.py | 46 +++--
 bindings/python/notmuch/globals.py  |  5 
 2 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 1279804a..ef370839 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
 _str,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
+NotmuchIndexoptsP,
 NotmuchMessageP,
 NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
 MODE = Enum(['READ_ONLY', 'READ_WRITE'])
 """Constants: Mode in which to open the database"""
 
+DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+"""Constants: policies for decrypting messages during indexing"""
+
 """notmuch_database_get_directory"""
 _get_directory = nmlib.notmuch_database_get_directory
 _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
 # return the Directory, init it with the absolute path
 return Directory(abs_dirpath, dir_p, self)
 
+_get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+_get_default_indexopts.argtypes = [NotmuchDatabaseP]
+_get_default_indexopts.restype = NotmuchIndexoptsP
+
+_indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+_indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+_indexopts_set_decrypt_policy.restype = None
+
+_indexopts_destroy = nmlib.notmuch_indexopts_destroy
+_indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+_indexopts_destroy.restype = None
+
 _index_file = nmlib.notmuch_database_index_file
 _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
  c_void_p,
  POINTER(NotmuchMessageP)]
 _index_file.restype = c_uint
 
-def index_file(self, filename, sync_maildir_flags=False):
+def index_file(self, filename, sync_maildir_flags=False, 
decrypt_policy=None):
 """Adds a new message to the database
 
 :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
 API. You might want to look into the underlying method
 :meth:`Message.maildir_flags_to_tags`.
 
+:param decrypt_policy: If the message contains any encrypted
+parts, and decrypt_policy is set to
+:attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+decrypt the message and index the cleartext, stashing any
+discovered session keys.  If it is set to
+:attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+decrypt during indexing.  If it is set to
+:attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+any stashed session keys it knows about, but will not try
+to access the user's secret keys.
+:attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+:attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+are stashed in the database.  If decrypt_policy is set to
+None (the default), then the database itself will decide
+whether to decrypt, based on the `index.decrypt`
+configuration setting (see notmuch-config(1)).
+
 :returns: On success, we return
 
1) a :class:`Message` object that can be used for things
@@ -454,10 +487,19 @@ class Database(object):
   :attr:`STATUS`.READ_ONLY_DATABASE
   Database was opened in read-only mode so no message can
   be added.
+
 """
 self._assert_db_is_initialized()
 msg_p = NotmuchMessageP()
-status = self._index_file(self._db, _str(filename), c_void_p(None), 
byref(msg_p))
+indexopts = c_void_p(None)
+if decrypt_policy is not None:
+indexopts = self._get_default_indexopts(self._db)
+self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+status = self._index_file(self._db, _str(filename), indexopts, 
byref(msg_p))
+
+if indexopts:
+self._indexopts_destroy(indexopts)
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
 raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
 pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+pass
+NotmuchIndexoptsP = PO

[PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

2017-11-30 Thread Daniel Kahn Gillmor

If you've got a notmuch dump that includes stashed session keys for
every decrypted message, and you've got your message archive, you
should be able to get back to the same index that you had before.

Here we add a simple test that give some flavor of how that works.
---
 test/T357-index-decryption.sh | 31 +++
 1 file changed, 31 insertions(+)

diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 22e716c6..11ea2074 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -156,6 +156,37 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+add_email_corpus crypto
+
+test_begin_subtest "indexing message fails when secret key not available"
+notmuch reindex --try-decrypt id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch dump )
+expected='#notmuch-dump batch-tag:3 config,properties,tags
++encrypted +inbox +unread -- id:simple-encryp...@crypto.notmuchmail.org
+#= simple-encryp...@crypto.notmuchmail.org index.decryption=failure'
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest "cannot find cleartext index"
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest "cleartext index recovery on reindexing with stashed 
session keys"
+notmuch restore

[PATCH v2 17/21] cli/reindex: destroy stashed session keys when --decrypt=false

2017-11-30 Thread Daniel Kahn Gillmor

There are some situations where the user wants to get rid of the
cleartext index of a message.  For example, if they're indexing
encrypted messages normally, but suddenly they run across a message
that they really don't want any trace of in their index.

In that case, the natural thing to do is:

   notmuch reindex --decrypt=false id:whate...@example.biz

But of course, clearing the cleartext index without clearing the
stashed session key is just silly.  So we do the expected thing and
also destroy any stashed session keys while we're destroying the index
of the cleartext.

Note that stashed session keys are stored in the xapian database, but
xapian does not currently allow safe deletion (see
https://trac.xapian.org/ticket/742).

As a workaround, after removing session keys and cleartext material
from the database, the user probably should do something like "notmuch
compact" to try to purge whatever recoverable data is left in the
xapian freelist.  This problem really needs to be addressed within
xapian, though, if we want it fixed right.
---
 doc/man1/notmuch-reindex.rst  |  3 +++
 lib/message.cc|  5 +
 test/T357-index-decryption.sh | 17 +
 3 files changed, 25 insertions(+)

diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index d87e9d85..e8174f39 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -30,6 +30,9 @@ Supported options for **reindex** include
 the user's secret keys.  If decryption is successful, index
 the cleartext itself.
 
+If ``false``, notmuch reindex will also delete any stashed
+session keys for all messages matching the search terms.
+
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
diff --git a/lib/message.cc b/lib/message.cc
index 12743460..d5db89b6 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -2002,6 +2002,11 @@ notmuch_message_reindex (notmuch_message_t *message,
 ret = notmuch_message_remove_all_properties_with_prefix (message, 
"index.");
 if (ret)
goto DONE; /* XXX TODO: distinguish from other error returns above? */
+if (indexopts && notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE) {
+   ret = notmuch_message_remove_all_properties (message, "session-key");
+   if (ret)
+   goto DONE;
+}
 
 /* re-add the filenames with the associated indexopts */
 for (; notmuch_filenames_valid (orig_filenames);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index b2717a7a..64317c64 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -218,6 +218,23 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "purging stashed session keys should lose access to the 
cleartext"
+notmuch reindex --decrypt=false id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest "and cleartext should be unrecoverable now that there are 
no stashed session keys"
+notmuch dump
+notmuch reindex --decrypt=true id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 13/18] cli/new, insert, reindex: change index.try_decrypt to "auto" by default

2017-11-29 Thread Daniel Kahn Gillmor

On Thu 2017-11-16 08:40:41 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> The new "auto" decryption policy is not only good for "notmuch show"
>> and "notmuch reindex".  It's also useful for indexing messages --
>> there's no good reason to not try to go ahead and index the cleartext
>> of a message that we have a stashed session key for.
>
> I'm confused here. You talk about indexing other than reindex, but the
> only tests that change are reindex? Is this meant to change "notmuch
> new" behaviour?

the "auto" policy won't change the behavior of notmuch upon seeing a new
message (new, insert) from "false" -- all it does differently from
"false" is try to use session keys when they are available (and there's
no way for them to be available on a never-before-seen message).

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help: give a hint about notmuch-emacs-mua

2017-11-29 Thread Daniel Kahn Gillmor

On Thu 2017-10-26 18:27:51 -0400, Daniel Kahn Gillmor wrote:
> "notmuch help" doesn't mention "notmuch-emacs-mua" even though we
> support it through the try_external_command() mechanism.
>
> In addition, "notmuch help emacs-mua" doesn't work, even though we
> ship the appropriate manpage.
>
> This changeset fixes both of these problems.

Ping!  i don't think this patch is complex or objectionable.  Can it be
merged?

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help, completion: added pointers to notmuch-properties(7)

2017-11-29 Thread Daniel Kahn Gillmor

On Thu 2017-10-26 18:28:12 -0400, Daniel Kahn Gillmor wrote:
> ---
>  completion/notmuch-completion.bash | 2 +-
>  notmuch.c  | 2 ++
>  2 files changed, 3 insertions(+), 1 deletion(-)

Ping on this patch as well.  I think this should be safe and simple to
merge.

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] crypto: signature verification reports valid User IDs

2017-11-29 Thread Daniel Kahn Gillmor

When i'm trying to understand a message signature, i care that i know
who it came from (the "validity" of the identity associated with the
key), *not* whether i'm willing to accept the keyholder's other
identity assertions (the "trust" associated with the certificate).

We've been reporting User ID information based on the "trust"
associated with the certificate, because GMime didn't clearly expose
the validity of the User IDs.

This change relies on fixes made in GMime 3.0.3 and later which
include https://github.com/jstedfast/gmime/pull/18.
---
 configure  |  3 ++-
 notmuch-show.c | 36 +---
 test/T355-smime.sh |  8 +++-
 util/gmime-extra.h | 11 ---
 4 files changed, 38 insertions(+), 20 deletions(-)

diff --git a/configure b/configure
index d3e30b53..cfbf827c 100755
--- a/configure
+++ b/configure
@@ -478,9 +478,10 @@ fi
 # we need to have a version >= 2.6.5 to avoid a crypto bug. We need
 # 2.6.7 for permissive "From " header handling.
 GMIME_MINVER=2.6.7
+GMIME3_MINVER=3.0.3
 
 printf "Checking for GMime development files... "
-if pkg-config --exists "gmime-3.0"; then
+if pkg-config --exists "gmime-3.0 > $GMIME3_MINVER"; then
 printf "Yes (3.0).\n"
 have_gmime=1
 gmime_cflags=$(pkg-config --cflags gmime-3.0)
diff --git a/notmuch-show.c b/notmuch-show.c
index 7afd3947..4b89be0a 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -401,6 +401,32 @@ format_signature_errors (sprinter_t *sp, GMimeSignature 
*signature)
 }
 #endif
 
+
+static const char*
+_get_certificate_valid_userid (GMimeCertificate *cert)
+{
+/* output user id only if validity is FULL or ULTIMATE. */
+#if (GMIME_MAJOR_VERSION < 3)
+/* note that gmime 2.6 is using the term "trust" here, which
+ * is WRONG.  It's actually user id "validity". */
+const char *name = g_mime_certificate_get_name (cert);
+if (name == NULL)
+   return name;
+GMimeCertificateTrust trust = g_mime_certificate_get_trust (cert);
+if (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)
+   return name;
+return NULL;
+#else
+const char *uid = g_mime_certificate_get_user_id (cert);
+if (uid == NULL)
+   return uid;
+GMimeValidity validity = g_mime_certificate_get_id_validity (cert);
+if (validity == GMIME_VALIDITY_FULL || validity == GMIME_VALIDITY_ULTIMATE)
+   return uid;
+return NULL;
+#endif
+}
+
 /* Signature status sprinter (GMime 2.6) */
 static void
 format_part_sigstatus_sprinter (sprinter_t *sp, mime_node_t *node)
@@ -446,15 +472,11 @@ format_part_sigstatus_sprinter (sprinter_t *sp, 
mime_node_t *node)
sp->map_key (sp, "expires");
sp->integer (sp, expires);
}
-   /* output user id only if validity is FULL or ULTIMATE. */
-   /* note that gmime is using the term "trust" here, which
-* is WRONG.  It's actually user id "validity". */
if (certificate) {
-   const char *name = g_mime_certificate_get_uid (certificate);
-   GMimeCertificateTrust trust = g_mime_certificate_get_trust 
(certificate);
-   if (name && (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)) {
+   const char *uid = _get_certificate_valid_userid (certificate);
+   if (uid) {
sp->map_key (sp, "userid");
-   sp->string (sp, name);
+   sp->string (sp, uid);
}
}
} else if (certificate) {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 1523f17b..be45e3b1 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -48,6 +48,12 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "signature verification (notmuch CLI)"
+if [ "${NOTMUCH_GMIME_MAJOR}" -lt 3 ]; then
+# gmime 2 can't report User IDs properly for S/MIME
+USERID=''
+else
+USERID='"userid": "CN=Notmuch Test Suite",'
+fi
 output=$(notmuch show --format=json --verify subject:"test signed message 001" 
\
 | notmuch_json_show_sanitize \
 | sed -e 's|"created": [-1234567890]*|"created": 946728000|' \
@@ -65,7 +71,7 @@ expected='[[[{"id": "X",
  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"fingerprint": "'$FINGERPRINT'",
- "status": "good",
+ "status": "good",'$USERID'
  "expires": 424242424,
  "created": 946728000}],
  "content-type": "multipart/signed",
diff --git a/util/gmime-extra.h b/util/gmime-extra.h
index 40bf1454..ec4ca186 100644
--- a/util/gmime-extra.h
+++ b/util/gmime-extra.h
@@ -16,11 +16,9 @@ GMimeStream *g_mime_stream_stdout_new(void);
 #define g_mime_2_6_unref(obj) g_object_unref (obj)
 #define g_mime_3_unused(arg) arg
 #define g_mime_certificate_get_fpr16(cert) g_mime_certificate_get_key_id (cert)
-#define g_mime_certificate_get_uid(cert) g_mime_certificate_get_name (cert);
 #else /* GMime >= 3.0 */
 
 #define GMIME_

Re: [PATCH] python: add bindings for notmuch_message_get_propert(y/ies)

2017-11-28 Thread Daniel Kahn Gillmor

On Tue 2017-11-28 23:46:11 +0100, Ruben Pollan wrote:
> Message.get_property (prop) returns a string with the value of the property 
> and
> Message.get_properties (prop, exact=False) returns a list [(key, value)]

This looks like a sensible approach to me.  I'd be curious to hear what
others think of this.

In considering the API design space here, it occurs to me that it might
be more pythonic for get_properties to return a dict like:

   { key: [ value, … ], key: [ value, … ] }

Any reason you chose one over the other?  My python-fu is shallow, so
please don't take my aesthetic guesswork as authoritative; but i'm
imagining a user wanting to grab a bunch of properties and then easily
access them by key, and the dict seems like the simple way to do that.

Also, does get_properties() work with prop=None to fetch all properties?
if so, maybe that should be the default?

To be clear, I'd be fine with a response that disagrees with these
suggestions (especially if it explains why) and then adopting this
patch; i just want to make sure they've been considered before we lock
in the API.

Thanks for working on this, Meskio!

   --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] python: add bindings for notmuch_message_get_property

2017-11-19 Thread Daniel Kahn Gillmor

On Wed 2017-11-15 23:29:54 +0100, Ruben Pollan wrote:
> Message.get_property (prop) returns a string with the value of the property.

Upon review, this is actually insufficient for making robust use of the
session-key series :(

In particular, it only returns the first value for the session key
returned.

There are (at least) two situations where a message may have more than
one session key:

 * if two copies of the message are received by different channels, and
   each channel somehow obtains a different session key.

   For example: i send the message to an encrypted mailing list like
   schleuder that unwraps and then rewraps the encrypted message -- in
   this case, the version saved to sent-mail during sending has session
   key A, and the version received back from schleuder has session key B

 * if one encrypted message contains another encrypted message.  then
   the outer message has session key A, and the inner attachment has
   session key B.

of course there are more ways this can happen, as well as combinations
of these ways :/

it mostly won't happen!  so things will look like they're looking fine,
but then you'll get a message (or two copies of a single message) and at
some point you'll try to render one part or one version, but you'll only
have the other session key available.

In the session-key series, i work around this by simply trying each
session key against an encrypted part until i find one that works. It
would be "cleaner" (more principled) to somehow associate each session
key with the part(s) of the message file(s) that it is capable of
decrypting, but that's a lot of bookkeeping -- i think it's actually
"cleaner" (less code, less computation in the standard case) to just
take the current approach.

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stashed session keys

2017-11-16 Thread Daniel Kahn Gillmor

Hi meskio--

On Wed 2017-11-15 23:41:28 +0100, meskio wrote:
> Nice feature. I'm using it and it works fine. I notice some speed up, 
> improving 
> the painfulness of reading long encrypted threads in alot. And I like to 
> don't 
> be able to have around my old private keys.

cool, i'm glad it's working for you!

> I implemented some support for it in alot (using the patch I just sent adding 
> notmuch_message_get_property to the python binding):
> https://github.com/meskio/alot/tree/session-key

very nice :)

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 05/18] crypto: Test restore of cleartext index from stashed session keys

2017-11-14 Thread Daniel Kahn Gillmor

On Tue 2017-11-14 09:13:52 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> If you've got a notmuch dump that includes stashed session keys for
>> every decrypted message, and you've got your message archive, you
>> should be able to get back to the same index that you had before.
>
> Out of curiousity, have you given any thought to what happens when
> someone sends a message with the same message-id but a different
> session-key? it seems like the user can potentially lose access to the
> encrypted message.

yep!  I even have that case in my own mailbox due to messages i've sent
to schleuder encrypted mailing lists to which i'm also subscribed.

It works fine.  notmuch stashes both session keys against the message-id
(you can have multiple properties with the same name as long as they
have different values).  And upon decryption, it tries each session-key
in succession.  This is a little bit sloppy (maybe it would be less
sloppy to associate each message key with each version of the message
somehow?), but it's significantly simpler and basically unnoticeable
compared to the speedup gains provided by the rest of the series.

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 03/18] crypto: use stashed session-key properties for decryption, if available

2017-11-14 Thread Daniel Kahn Gillmor

Hi Bremner--

Thanks for the review!

On Tue 2017-11-14 09:02:08 -0400, David Bremner wrote:

> Since you wrote this, I've deprecated GMime 2.6. I'm not sure that
> changes anything here, but seems worth mentioning

well, i'm happy to hear that -- i've got no problem with deprecating
GMime 2.6, and would be fine with maintaining GMime 3.0 in
stretch-backports if that would make you feel more comfortable about the
decision.

Still, I'll be kind of bummed to have to rewrite this series to strip
out the 2.6 support: i originally wrote it only with 3.0 support, and
then went back in and added 2.6 support because at the time, you didn't
want to deprecate 2.6 :( our coding cadence isn't very well synced :/

> Its a nitpick, but I don't really understand/like including = with the
> property name.  That will break, e.g. for anyone attempting to use it
> from the API.

I don't mind changing the documentation to use ``session-key`` instead
of ``session-key=``.  *shrug*

> The way the diff works out, I was pretty confused by seeing several
> "wrong" calls to _notmuch_crypto_decrypt before the actual change. It
> would be nice to telegraph that somehow, perhaps in the commit message.

sure, i can add to the commit message that _notmuch_crypto_decrypt is
growing a new parameter.

> Personally I would be fine with (and probably happier) only supporting
> new features when using gmime-3.0.  Debugging crypto related stuff is
> always hard (see recent discussion about _mime_node_create, where we
> still don't know what's wrong, and are just papering over the problem),
> and it seems worth striving for simplicity as much as possible.  I also
> don't know how motivated gmime upstream is to fix bugs in 2.6; I could
> certainly understand if the answer was "not very".

I believe the answer is "not very" -- but if there are serious bugs (i
don't think we've talked about any of this stuff as bugs in gmime) then
we should probably try to raise them with him.

> There is, by the way, a function notmuch_built_with that can be used to
> introspect the library as to what optional features it is built
> with. It's used in notmuch_config to report back to the user about the
> presence of optional features.

Is there any naming convention for these features?  do you want me to
add a "session-key" label with a future revision of this branch?  or are
you asking for something else?

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stashed session keys

2017-11-12 Thread Daniel Kahn Gillmor

On Sun 2017-11-12 11:51:11 +0800, Daniel Kahn Gillmor wrote:
> On Sat 2017-11-11 15:31:36 -0800, Jameson Graef Rollins wrote:
>> I haven't decided what's the best way to do that yet, but something
>> like the following happening automatically at inbox view might do the
>> trick:
>>
>>   notmuch reindex --try-decrypt=true (tag:inbox AND tag:encrypted)
>
> This seems like a reasonable way to ensure that your long-term, personal
> secret keys only get accessed when you are interactively working with
> your mail user agent.
>
> You might be able to target the reindex even more narrowly by adding
> something like "AND not property:index-decryption=success"

Sorry, this should be "AND not property:index.decryption=success"

   --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 07/18] crypto: new decryption policy "auto"

2017-11-11 Thread Daniel Kahn Gillmor

On Sat 2017-11-11 15:14:03 -0800, Jameson Graef Rollins wrote:
> On Wed, Oct 25 2017, Daniel Kahn Gillmor  wrote:
>> diff --git a/util/crypto.h b/util/crypto.h
>> index b23ca747..dc95b4ca 100644
>> --- a/util/crypto.h
>> +++ b/util/crypto.h
>> @@ -16,7 +16,8 @@ typedef struct _notmuch_crypto {
>>  } _notmuch_crypto_t;
>>  
>>  GMimeObject *
>> -_notmuch_crypto_decrypt (notmuch_message_t *message,
>> +_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
>> + notmuch_message_t *message,
>>   GMimeCryptoContext* crypto_ctx,
>>   GMimeMultipartEncrypted *part,
>>   GMimeDecryptResult **decrypt_result,
>
> Why does _notmuch_crypt_decrypt need to have
> "notmuch_decryption_policy_t decrypt" as an input argument?  Isn't
> notmuch_decryption_policy_t already an attribute of the crypto_ctx?  Is
> there some situation where the policy would differ from what's specified
> in the crypto_ctx?

crypto_ctx here is just a GMimeCryptoContext, which doesn't know
anything about notmuch_decryption_policy_t.  Maybe i'm misunderstanding
your question?

I'd be happy to streamline the interface to this internal function, but
given that it's not an exported API, i'm not as concerned about things
like future cleanliness -- the notmuch source contains all invocations
of the function anywhere, so if we find a nicer way to streamline it in
the future, we can do that cleanup across the codebase in a single
commit.

Thanks for the review!

--dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stashed session keys

2017-11-11 Thread Daniel Kahn Gillmor

On Sat 2017-11-11 15:31:36 -0800, Jameson Graef Rollins wrote:
> I have reviewed and tested this series, and it seems solidly
> implemented and very well motivated.  I have been using it regularly
> for a couple weeks now and have found no issues with it's usage, and
> have noticed the considerable speed up when viewing encrypted threads
> (as much as x8 for show on a thread of just 8 encrypted messages).  I
> fully support it's integration unconditionally.

thanks for the review, the testing, and the reportback.  I'm glad to
hear that it's giving you the same sort of speedups that it gives me.

> Daniel likes to think of this in terms of being able to "delete"
> encrypted messages in the wild (via deletion of the original
> encryption key) whereas I like to think of it in terms of preserving
> access to received encrypted messages after key rotation.  Both
> benefits hold, though, obviously.

yes, these are different ways of looking at the same key management
strategy.

> I think these policies cover all potential use cases that I can see.
> However, there will need to be further work on the UX to make things
> flow more smoothly.

Agreed.  The goal of this series is to provide the framework that can be
used to build smoother UX, but it doesn't get all the way to providing
the smoothest possible UX.  Such is the nature of toolkit development.

> I haven't decided what's the best way to do that yet, but something
> like the following happening automatically at inbox view might do the
> trick:
>
>   notmuch reindex --try-decrypt=true (tag:inbox AND tag:encrypted)

This seems like a reasonable way to ensure that your long-term, personal
secret keys only get accessed when you are interactively working with
your mail user agent.

You might be able to target the reindex even more narrowly by adding
something like "AND not property:index-decryption=success"

> Finally, I think it would be worthwhile to resolve the disparity between
> the usage of "decrypt" and "try-decrypt" in the CLI and config options.
> I'm not sure why we're using different terms in different contexts, even
> though the meanings are essentially the same.  A follow-up patch series
> changing "try-decrypt" -> "decrypt" would probably be in order.

If this series lands, i'd be happy to supply such an term-normalizing
series for subsequent consideration.

If people feel that this term normalization is the main blocker to
landing this series, i could try to rebase it with a different UI terms,
but rebasing the series for this change feels like busy-work to me (and
would be more effort than a simple normalization patch on top).  i'd
rather spend my limited notmuch hacking+reviewing time providing useful
features.

  --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Stashed session keys

2017-11-10 Thread Daniel Kahn Gillmor

On Wed 2017-10-25 02:51:45 -0400, Daniel Kahn Gillmor wrote:
> Now that cleartext indexing is merged, let's add the ability to stash
> session keys!

I'd really appreciate feedback on this series.  It works for me, and i'm
using it.  I don't want this to take another two years to land, if
possible.  And it blocks additional improvements that i would like to
eventually land as well.

So any review of any part of this series would be appreciated.  I want
to make notmuch a mail client that provides actually usable encrypted
mail.

Regards,

  --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] NEWS: cleartext indexing

2017-10-31 Thread Daniel Kahn Gillmor

On Mon 2017-10-30 12:16:25 -0400, Antoine Beaupré wrote:
> I think that assumption should be made clear in the documentation,
> because "security of your index" means nothing to me. Explicitly mention
> FDE as an example may be a good start.

again, i'm not convinced that "full disk" encryption is what's
warranted, although filesystem, directory, or per-file encryption might
be part of the solution.

I don't want the documentation produced here to prescribe a particular
solution, because i *want* people to experiment and investigate.

I also don't think that the notmuch documentation is the right place to
put a primer on filesystem encryption, or a treatise on the comparison
of data at rest to data in flight.  I wouldn't object to pointers to
more discussion here though, for people who want to read more.

> Frankly, I don't have a good solution for this. I was thinking that
> there may be a way to just encrypt the whole notmuch database with gpg
> and decrypt it on the fly as needed, but that's probably a ludicrous
> idea.

Given the size of the database on the corpora i'm used to running
notmuch over, i don't see an efficient way to do that workflow.  If the
database was smaller, that'd be no problem.

But even if it was doable efficiently, you're still left with a question
of when you plan to unlock or re-lock the database.  And while the
database is unlocked, how do you limit access to it to notmuch itself?

And if you do limit it to notmuch itself, what about the other tools
that might want to interface with notmuch?  are they allowed access to
the cleartext?

These are much more interesting questions to my mind than whether we use
gpg specifically or something else.

They would also apply to filesystem-level encryption.

> So I hear all those arguments and mostly agree with them. That's the
> "rationale about the decision" part, what I'm missing is the "mitigation
> strategies". What I'm hearing is simply "use FDE", but I already do that
> and I don't feel it brings much added security.

It's interesting that you heard "FDE" when i've explicitly said "not
FDE". Filesystem encryption ≠ "full disk" encryption ≠ per-directory
encryption.

For example, you could have a separate local filesystem that contains
only your message store and your notmuch index, mounted atop a distinct
crypto-mapped block device.

Then, when you're willing to allow access to your index (and the rest of
your "at-rest" mail) you simply map the block device and mount the
filesystem where the user account that uses notmuch can read it.

when you're done with mail, you umount and unmap.

This proposal also adds "encryption at rest" to the mail that wasn't
even encrypted in transit, as a bonus :)

You could even put the whole underlying block device on removable
storage, making it truly inaccessible when it is not plugged in.

Or you could try to use ext4's new-ish encryption features:

https://lwn.net/Articles/639427/
https://wiki.archlinux.org/index.php/Ext4#Using_file-based_encryption
http://kernsec.org/files/lss2014/Halcrow_EXT4_Encryption.pdf

I'd be happy to experiment with that with you and report back to the
list if you like.  Maybe we could prototype a "notmuch lock" command?

Again, i don't think that notmuch documentation is a great place to
document these ideas, unless we're actually implementing them in a
simple and straightforward way so that the user can trigger the actions
easily.

And, sadly, note that my proposal above usually requires root access
(mount/umount/device-map) on most GNU/Linux systems (i haven't looked
into other systems in enough detail).  This is certainly an obstacle to
deployment.

> Having my emails encrypted adds another layer of security to that
> content. FDE is good for data "at rest", e.g. when i travel with my
> laptop. But when my laptop is opened and running, I like to think that a
> part of it isn't accessible without the security layers behind PGP and
> actual human confirmation.

It sounds to me like you think that all invocations of PGP are going to
be mapped behind human confirmation.  This is not the general GnuPG use
case these days (due to caching in gpg-agent), except for people with an
external crypto token that itself physically requires presence to
decrypt.

In the event of rendering a 10-message encrypted thread, the "human
confirmation" approach requires 10 touches of the cryptotoken.  IIRC,
even if the human is alert and ready to touch the token when needed,
we're talking about probably at least half a minute of delay between
"i'd like to read this thread" and "ah, there it is".  and the user
can't even do something else with that time because they're all tied up
watching for and performing the crypto-token-touch.

I don't think that's acceptable for an e-mail client that you expect
users to actually use, unless you're running some sort of skinnerian
behaviorist experiement.

So in practice, one authorization of your PGP key is likely to enable
arbitrary program

Re: [PATCH] NEWS: cleartext indexing

2017-10-30 Thread Daniel Kahn Gillmor

On Mon 2017-10-30 08:46:12 -0400, Antoine Beaupré wrote:
> On 2017-10-22 11:36:34, Daniel Kahn Gillmor wrote:
>> +  Note that the contents of the index are sufficient to roughly
>> +  reconstruct the cleartext of the message itself, so please ensure
>> +  that the notmuch index itself is adequately protected.  DO NOT USE
>> +  this feature without considering the security of your index.
>
> Could we expand on what those security options could be? Full disk
> encryption? Or is there some way to PGP-encrypt the index and have it
> decrypted on the fly?

This is deliberately out of scope of the series -- i don't want this
series to introduce notmuch-specific index encryption. Though if someone
wanted to propose that i'd be happy to review it; fwiw, i have looked at
a few deterministic mail index encryption schemes, and i'm not convinced
that they meet any realistic threat model.

But yes, my basic assumption is that people who care will keep their
index on an encrypted filesystem, or at least on a filesystem that sits
atop an encrypted block device layer.  I don't know whether it's
relevant if the encryption layer is "full-disk" or not.

> Security, in this context, seems a little broad... I do have a antsy
> feeling at decrypting all my private emails in a cleartext database
> without additional measures. I'd sure love to see this notion expanded
> here somehow.

expand away!  I'd be interested in reading your detailed thoughts on
this.

My basic rationale is:

 * i find myself not encrypting some e-mails because encrypted e-mails
   are too hard to use.  in particular, they're hard to search through,
   and they are expensive to render. ("where was the address of that
   restaurant again?  if only i had sent it in cleartext...")

 * the result is that i either send those mails in the clear (leaking
   the cleartext irrevocably), or i move my communications off of e-mail
   (typically to platforms on which i have even less control than
   e-mail).

 * This is actually a messaging security *anti-pattern*, and i've
   observed it in someone who takes communications confidentiality
   seriously (myself).

 * furthermore, it harms the ability for a conscientious sender to
   produce encrypted e-mail by default, because they're likely to be
   concerned about the same usability obstacles on the receiver's side.

 * making the use of encrypted e-mails more fluid and accessible should
   actually increase the total amount of encrypted mail, providing more
   protection in general.

 * the steps you need to take to secure your cleartext index are similar
   (though not congruent) to the steps you need to take to secure your
   MUA itself (endpoint-level security hardening).  If you aren't
   already taking basic steps to protect your endpoint, it's not clear
   to me that keeping encrypted messages out of your index protects them
   significantly against a motivated attacker.

I'd love to talk more at some point about dynamic or scoped access to a
subsets of an encrypted filesystem (i.e. limiting access to your
cleartext index so that it's *only* available to your MUA, and not to
other programs running as your user account).

Hopefully this series will trigger that as an additional discussion,
though, rather than block on it.  let's make encrypted e-mail as usable
as cleartext e-mail (or as close to it as possible) to protect more
communications overall.

> By the way, I have similar concerns about the autocrypt approach, which
> goes even further and says private key material should not be protected
> by a password at all:
>
> http://autocrypt.readthedocs.io/en/latest/level1.html#secret-key-protection-at-rest
>
> It would be interesting to explain the rationale around those decisions
> (which autocrypt does) and possible safeguards that mitigate those
> issues (which autocrypt doesn't).

Yes, we should have this discussion, but i don't think the notmuch
mailing list is the right place for it.  Feel free to come talk it over
on the autocrypt list!

   --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

2017-10-27 Thread Daniel Kahn Gillmor

On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote:
> With bleach integrated (all of five lines), I think this is safe enough
> to let random notmuch users run it.

hm, bleach might be a little too aggressive.

jrollins just pointed toward:

https://nmweb.evenmere.org/show/87innmvvam.fsf%40ligo.caltech.edu

which i'm pretty sure had actual content initially
(id:87innmvvam@ligo.caltech.edu) but it starts with stdin
redirection (using a left angle bracket) and then the rest of the
message is gone :/

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

2017-10-26 Thread Daniel Kahn Gillmor

On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote:
> Thanks!  The part I'm happiest about is the speed:

amen, it feels very lightweight.

> Very careful examination would have shown that the em-dashes between
> author and subject were red for matches.  Now matches are in italics.

cool.  perhaps assigning a class to those elements and stashing some CSS
would make that easier for folks to experiment with (and probably reduce
the bytecount transfered)?

or would that hurt the rendering time for some reason i'm unaware of?  i
haven't thought about these mechanics as much as you have.

> Yup.  The thread object isn't accessible by then: it existed in the
> scope of the search query, and is gone by the time we show the message.
> get_replies isn't available.  So what's the alternative?
> get_thread_id(), search for that thread id, identify this message *in*
> that thread id, and then link to the next message with a "next" link?
> While doing it, why not show the thread structure at the bottom of the
> message, I guess.

yep, i think that's right.

> With bleach integrated (all of five lines), I think this is safe enough
> to let random notmuch users run it.  The worst they'll do is expose
> their mailstore on tcp/8080.  Any interest in taking this into the
> upstream contrib directory?

Yes, i think this should move into contrib/ upstream.  And we should
think about what might be the appropriate way to package it for debian,
too.

--dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] cli/help: give a hint about notmuch-emacs-mua

2017-10-26 Thread Daniel Kahn Gillmor

"notmuch help" doesn't mention "notmuch-emacs-mua" even though we
support it through the try_external_command() mechanism.

In addition, "notmuch help emacs-mua" doesn't work, even though we
ship the appropriate manpage.

This changeset fixes both of these problems.
---
 notmuch.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/notmuch.c b/notmuch.c
index efbe32ff..8eab561b 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -172,6 +172,8 @@ static command_t commands[] = {
   "Re-index all messages matching the search terms." },
 { "config", notmuch_config_command, NOTMUCH_CONFIG_OPEN,
   "Get or set settings in the notmuch configuration file." },
+{ "emacs-mua", NULL, 0,
+  "send mail with notmuch and emacs." },
 { "help", notmuch_help_command, NOTMUCH_CONFIG_CREATE, /* create but don't 
save config */
   "This message, or more detailed help for the named command." }
 };
@@ -487,7 +489,8 @@ main (int argc, char *argv[])
 notmuch_process_shared_options (command_name);
 
 command = find_command (command_name);
-if (!command) {
+/* if command->function is NULL, try external command */
+if (!command || !command->function) {
/* This won't return if the external command is found. */
if (try_external_command(argv + opt_index))
fprintf (stderr, "Error: Unknown command '%s' (see \"notmuch 
help\")\n",
-- 
2.14.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] cli/help,completion: added pointers to notmuch-properties(7)

2017-10-26 Thread Daniel Kahn Gillmor

---
 completion/notmuch-completion.bash | 2 +-
 notmuch.c  | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index a0c3ac17..895fc554 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -611,7 +611,7 @@ _notmuch()
esac
 elif [ "${arg}" = "help" ]; then
# handle help command specially due to _notmuch_commands usage
-   local help_topics="$_notmuch_commands hooks search-terms"
+   local help_topics="$_notmuch_commands hooks search-terms properties"
COMPREPLY=( $(compgen -W "${help_topics}" -- ${cur}) )
 else
# complete using _notmuch_subcommand if one exist
diff --git a/notmuch.c b/notmuch.c
index 2f43ad28..efbe32ff 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -186,6 +186,8 @@ static help_topic_t help_topics[] = {
   "Common search term syntax." },
 { "hooks",
   "Hooks that will be run before or after certain commands." },
+{ "properties",
+  "Message property conventions and documentation." },
 };
 
 static command_t *
-- 
2.14.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

2017-10-26 Thread Daniel Kahn Gillmor

On Wed 2017-10-25 18:03:01 -0400, Brian Sniffen wrote:
> That's inspiring!  Now there's a demo of nmweb at
>
> https://nmweb.evenmere.org/

this is very nice, Brian.

Your URL highlighter seems a bit trigger-happy though:

   https://nmweb.evenmere.org/show/8760s7zr47.fsf%40zancas.localnet

I don't think bremner was trying to link to http://index.cc !

> It's possible to get it to dump the whole mbox by clicking through the
> obvious links; please consider exploring at
> https://nmweb.evenmere.org/search/monkey instead.

this is interesting because it shows me threads where some messages have
monkey in them, but i can't tell which messages actually have the
relevant search term.  Maybe it could highlight the found messages?

Also, once i'm looking at one message, i don't see an easy way to go
"next" in the thread.

see: you show off a cool trick, you get requests for more cool tricks :)

> There are not many monkeys in the inbox.

speak for your own inbox, please.

  --dkg

signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: fix test database backup/restore location

2017-10-26 Thread Daniel Kahn Gillmor

On Tue 2017-10-24 21:36:11 +0300, Jani Nikula wrote:
> backup_database() and restore_database() used to store the backups in
> the test specific temporary directory, through the current working
> directory being there. Commit 8e7fb88237ae ("test: use source and
> build paths in test-lib-common.sh") started using a test specific
> backup directories under the build tree test directory. This was in
> error. Switch back to the old location, but using paths to the
> location instead of relying on current working directory.
>
> Reported by Daniel Kahn Gillmor .
> ---
>  test/test-lib-common.sh | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
> index 4300eb65418f..6c3571d4c560 100644
> --- a/test/test-lib-common.sh
> +++ b/test/test-lib-common.sh
> @@ -31,14 +31,14 @@ fi
>  
>  backup_database () {
>  test_name=$(basename $0 .sh)
> -rm -rf $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."$test_name"
> -cp -pR ${MAIL_DIR}/.notmuch 
> $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}"
> +rm -rf $TMP_DIRECTORY/notmuch-dir-backup."$test_name"
> +cp -pR ${MAIL_DIR}/.notmuch 
> $TMP_DIRECTORY/notmuch-dir-backup."${test_name}"
>  }
>  
>  restore_database () {
>  test_name=$(basename $0 .sh)
>  rm -rf ${MAIL_DIR}/.notmuch
> -cp -pR $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}" 
> ${MAIL_DIR}/.notmuch
> +cp -pR $TMP_DIRECTORY/notmuch-dir-backup."${test_name}" 
> ${MAIL_DIR}/.notmuch
>  }
>  
>  # Test the binaries we have just built.  The tests are kept in

Thanks for this, Jani.

i can confirm that with this patch applied, i no longer see the extra
directories lingering after a run of the test suite.  This solved my
problem.  I recommend merging :)

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

< 4 5 6 7 8 9 10 11 12 13 >

801 - 900 of 1575 matches

Mail list logo