Re: [PATCH v8 0/3] notmuch-search: Structured Output Formatters

[David Bremner]

cra...@gmx.net writes:

 This patch series amends the situation by introducing structured
 formatters, which allow different implementations of structures like
 lists, maps, strings and numbers.


pushed, 

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v4 0/8] emacs: JSON-based search cleanups

[David Bremner]

Austin Clements amdra...@mit.edu writes:

 This version fixes several bugs found in the previous version.  

pushed, 

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: Fix notmuch-message-mark-replied.

[David Bremner]

Ingo Lohmar i.loh...@gmail.com writes:

 notmuch-message-mark-replied used apply to change message tags
 according to notmuch-message-replied-tags after sending a reply.  This
 works if the latter is a single-element list.  But with the recently
 changed format of tag changes, it breaks for multiple-element lists.
 Use funcall to properly pass the list of tag changes as a single
 argument.

pushed,

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch-emacs and bbdb

[Daniel Reusche]

On 22.07.2012 02:24, Daniel Bergey wrote:

Glad to hear I have another user!

Your welcome, nice snippet!


My function bbdb/notmuch-snarf-to doesn't work yet on more than one
recipient.

Probably not before late August, though.


Well, if I find some time, I will at least try it too.

best,
dan
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 0/4] Add a --body option to notmuch show

[David Bremner]

Mark Walters markwalters1...@gmail.com writes:

 Version 2 of this patch set (version 1 at
 id:1343060241-18283-1-git-send-email-markwalters1...@gmail.com). This
 corrects the errors Austin spotted in v1, and makes it explicit that
 --body=false and --part0 are incompatible.

2/4 doesn't apply anymore to current master, and I was a bit leery of
resolving conflicts in the tests; can you rebase and resend? no need for
review of the rebased series.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 (rebased) 0/4] Add a --body option to notmuch show

[Mark Walters]

This is a rebased version of the series
id:1343066259-22523-1-git-send-email-markwalters1...@gmail.com

It is otherwise unchanged. (The rebase is a change in the context part
of the test patch).

Best wishes

Mark


Mark Walters (4):
  cli: add --body=true|false option to notmuch-show.c
  test: add tests for the new --body=true|false option
  man: update man page for the new --body=true|false option
  schemata: update for --body=true|false option

 devel/schemata  |2 +-
 man/man1/notmuch-show.1 |   16 
 notmuch-client.h|3 ++-
 notmuch-reply.c |2 +-
 notmuch-show.c  |   30 ++
 test/json   |9 +
 6 files changed, 51 insertions(+), 11 deletions(-)

-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 (rebased) 1/4] cli: add --body=true|false option to notmuch-show.c

[Mark Walters]

This option allows the caller to suppress the output of the bodies of
the messages. Currently this is only implemented for format=json.

This is used by notmuch-pick.el (although not needed) because it gives
a speed-up of at least a factor of a two (and in some cases a speed up
of more than a factor of 8); moreover it reduces the memory usage in
emacs hugely.
---
 notmuch-client.h |3 ++-
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   30 ++
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 0c17b79..f930798 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -87,6 +87,7 @@ typedef struct notmuch_crypto {
 typedef struct notmuch_show_params {
 notmuch_bool_t entire_thread;
 notmuch_bool_t omit_excluded;
+notmuch_bool_t output_body;
 notmuch_bool_t raw;
 int part;
 notmuch_crypto_t crypto;
@@ -176,7 +177,7 @@ notmuch_status_t
 show_one_part (const char *filename, int part);
 
 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first);
+format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body);
 
 void
 format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 3a038ed..de21f3b 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -620,7 +620,7 @@ notmuch_reply_format_json(void *ctx,
 /* Start the original */
 printf (, \original\: );
 
-format_part_json (ctx, node, TRUE);
+format_part_json (ctx, node, TRUE, TRUE);
 
 /* End */
 printf (}\n);
diff --git a/notmuch-show.c b/notmuch-show.c
index 8f3c60e..d3419e4 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -559,7 +559,7 @@ format_part_text (const void *ctx, mime_node_t *node,
 }
 
 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first)
+format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body)
 {
 /* Any changes to the JSON format should be reflected in the file
  * devel/schemata. */
@@ -571,10 +571,12 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first)
printf (\headers\: );
format_headers_json (ctx, GMIME_MESSAGE (node-part), FALSE);
 
-   printf (, \body\: [);
-   format_part_json (ctx, mime_node_child (node, 0), first);
-
-   printf (]});
+   if (output_body) {
+   printf (, \body\: [);
+   format_part_json (ctx, mime_node_child (node, 0), first, TRUE);
+   printf (]);
+   }
+   printf (});
return;
 }
 
@@ -652,16 +654,16 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first)
 talloc_free (local);
 
 for (i = 0; i  node-nchildren; i++)
-   format_part_json (ctx, mime_node_child (node, i), i == 0);
+   format_part_json (ctx, mime_node_child (node, i), i == 0, TRUE);
 
 printf (%s}, terminator);
 }
 
 static notmuch_status_t
 format_part_json_entry (const void *ctx, mime_node_t *node, unused (int 
indent),
-   unused (const notmuch_show_params_t *params))
+   const notmuch_show_params_t *params)
 {
-format_part_json (ctx, node, TRUE);
+format_part_json (ctx, node, TRUE, params-output_body);
 
 return NOTMUCH_STATUS_SUCCESS;
 }
@@ -1004,6 +1006,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
 notmuch_show_params_t params = {
.part = -1,
.omit_excluded = TRUE,
+   .output_body = TRUE,
.crypto = {
.verify = FALSE,
.decrypt = FALSE
@@ -1032,6 +1035,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
{ NOTMUCH_OPT_INT, params.part, part, 'p', 0 },
{ NOTMUCH_OPT_BOOLEAN, params.crypto.decrypt, decrypt, 'd', 0 },
{ NOTMUCH_OPT_BOOLEAN, params.crypto.verify, verify, 'v', 0 },
+   { NOTMUCH_OPT_BOOLEAN, params.output_body, body, 'b', 0 },
{ 0, 0, 0, 0, 0 }
 };
 
@@ -1086,6 +1090,16 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
entire_thread = ENTIRE_THREAD_FALSE;
 }
 
+if (!params.output_body) {
+   if (params.part  0) {
+   fprintf (stderr, Warning: --body=false is incompatible with --part 
 0. Disabling.\n);
+   params.output_body = TRUE;
+   } else {
+   if (format != format_json)
+   fprintf (stderr, Warning: --body=false only implemented for 
format=json\n);
+   }
+}
+
 if (entire_thread == ENTIRE_THREAD_TRUE)
params.entire_thread = TRUE;
 else
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 (rebased) 3/4] man: update man page for the new --body=true|false option

[Mark Walters]

---
 man/man1/notmuch-show.1 |   16 
 1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
index b51a54c..506583a 100644
--- a/man/man1/notmuch-show.1
+++ b/man/man1/notmuch-show.1
@@ -152,6 +152,22 @@ The default is
 
 .RE
 
+.RS 4
+.TP 4
+.B \-\-body=(true|false)
+
+If true (the default)
+.B notmuch show
+includes the bodies of the messages in the output; if false,
+bodies are omitted.
+.B --body=false
+is only implemented for the json format and it is incompatible with
+.B --part  0.
+
+This is useful if the caller only needs the headers as body-less
+output is much faster and substantially smaller.
+.RE
+
 A common use of
 .B notmuch show
 is to display a single thread of email messages. For this, use a
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 (rebased) 4/4] schemata: update for --body=true|false option

[Mark Walters]

Previously body: was a compulsory field in a message. The new
--body=false option causes notmuch show to omit this field so update
schemata to reflect this.
---
 devel/schemata |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/devel/schemata b/devel/schemata
index 6677a1c..9cb25f5 100644
--- a/devel/schemata
+++ b/devel/schemata
@@ -47,7 +47,7 @@ message = {
 tags:   [string*],
 
 headers:headers,
-body:   [part]
+body?:  [part]# omitted if --body=false
 }
 
 # A MIME part (format_part_json)
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 (rebased) 2/4] test: add tests for the new --body=true|false option

[Mark Walters]

---
 test/json |9 +
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/test/json b/test/json
index 337b3f5..831e105 100755
--- a/test/json
+++ b/test/json
@@ -7,6 +7,15 @@ add_message [subject]=\json-show-subject\ [date]=\Sat, 
01 Jan 2000 12:00:0
 output=$(notmuch show --format=json json-show-message)
 test_expect_equal $output [[[{\id\: \${gen_msg_id}\, \match\: true, 
\excluded\: false, \filename\: \${gen_msg_filename}\, \timestamp\: 
946728000, \date_relative\: \2000-01-01\, \tags\: [\inbox\,\unread\], 
\headers\: {\Subject\: \json-show-subject\, \From\: \Notmuch Test 
Suite test_su...@notmuchmail.org\, \To\: \Notmuch Test Suite 
test_su...@notmuchmail.org\, \Date\: \Sat, 01 Jan 2000 12:00:00 +\}, 
\body\: [{\id\: 1, \content-type\: \text/plain\, \content\: 
\json-show-message\n\}]}, [
 
+# This should be the same output as above.
+test_begin_subtest Show message: json --body=true
+output=$(notmuch show --format=json --body=true json-show-message)
+test_expect_equal $output [[[{\id\: \${gen_msg_id}\, \match\: true, 
\excluded\: false, \filename\: \${gen_msg_filename}\, \timestamp\: 
946728000, \date_relative\: \2000-01-01\, \tags\: [\inbox\,\unread\], 
\headers\: {\Subject\: \json-show-subject\, \From\: \Notmuch Test 
Suite test_su...@notmuchmail.org\, \To\: \Notmuch Test Suite 
test_su...@notmuchmail.org\, \Date\: \Sat, 01 Jan 2000 12:00:00 +\}, 
\body\: [{\id\: 1, \content-type\: \text/plain\, \content\: 
\json-show-message\n\}]}, [
+
+test_begin_subtest Show message: json --body=false
+output=$(notmuch show --format=json --body=false json-show-message)
+test_expect_equal $output [[[{\id\: \${gen_msg_id}\, \match\: true, 
\excluded\: false, \filename\: \${gen_msg_filename}\, \timestamp\: 
946728000, \date_relative\: \2000-01-01\, \tags\: [\inbox\,\unread\], 
\headers\: {\Subject\: \json-show-subject\, \From\: \Notmuch Test 
Suite test_su...@notmuchmail.org\, \To\: \Notmuch Test Suite 
test_su...@notmuchmail.org\, \Date\: \Sat, 01 Jan 2000 12:00:00 
+\}}, [
+
 test_begin_subtest Search message: json
 add_message [subject]=\json-search-subject\ [date]=\Sat, 01 Jan 2000 
12:00:00 -\ [body]=\json-search-message\
 output=$(notmuch search --format=json json-search-message | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 (rebased) 0/4] Add a --body option to notmuch show

[David Bremner]

Mark Walters markwalters1...@gmail.com writes:

 This is a rebased version of the series
 id:1343066259-22523-1-git-send-email-markwalters1...@gmail.com

 It is otherwise unchanged. (The rebase is a change in the context part
 of the test patch).

pushed, thanks.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] man: show: update man page for entire-thread and json.

[Mark Walters]

Previously in notmuch show --format=json implied --entire-thread. This
is still the default but it is now possible to disable this. Update
the manpage to reflect this.
---

Update the manpage since --format=json and --entire-thread=false is
now supported. Reworded as Austin suggested.

Best wishes

Mark 

 man/man1/notmuch-show.1 |   21 +
 1 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
index 506583a..765b22c 100644
--- a/man/man1/notmuch-show.1
+++ b/man/man1/notmuch-show.1
@@ -24,11 +24,14 @@ Supported options for
 include
 .RS 4
 .TP 4
-.B \-\-entire\-thread
+.B \-\-entire\-thread=(true|false)
 
-By default only those messages that match the search terms will be
-displayed. With this option, all messages in the same thread as any
-matched message will be displayed.
+If true,
+.B notmuch show
+outputs all messages in the thread of any message matching the search
+terms; if false, it outputs only the matching messages. For
+.B --format=json
+this defaults to true.  For other formats, this defaults to false.
 .RE
 
 .RS 4
@@ -55,11 +58,13 @@ be nested.
 The output is formatted with Javascript Object Notation (JSON). This
 format is more robust than the text format for automated
 processing. The nested structure of multipart MIME messages is
-reflected in nested JSON output. JSON output always includes all
-messages in a matching thread; in effect
+reflected in nested JSON output. By default JSON output includes all
+messages in a matching thread; that is, by default,
 .B \-\-format=json
-implies
-.B \-\-entire\-thread
+sets
+.B \-\-entire\-thread
+The caller can disable this behaviour by setting
+.B \-\-entire\-thread=false
 
 .RE
 .RS 4
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 0/5] Notmuch Pick (WIP or contrib)

[Mark Walters]

Hello

Notmuch pick is an emacs view which displays a threaded view of
messages: each message has its own line and the thread structure is
shown with UTF-8 graphics characters (so it looks vaguely similar to
mutt's threaded view): see http://kanelephant.com/screen.png for a screenshot.

Pick was originally written by David Edmondon and posted on irc and I
have been developing it on and off since.  Now that almost all the
backend changes it uses are in I would like to see about getting it
into mainline.

The code is not of the same standard as mainline code: in particular a
lot of the code is written by me and is working but unidiomatic
lisp. It has also not had widespread testing so I would expect it to
have several bugs.

We could try getting it into mainline using the normal review type
approach, but the patch is necessarily large (it implements a new view
similar in size to show or search) with the main pick.el file being
nearly 900 lines.

An alternative approach would be to accept it into contrib and then
reviewers/users could submit fixes for the problems directly.

This patch series implements the latter approach, but I am definitely
happy to try for the former or some other approach.

In its current form the user needs to copy (or link) the
notmuch-pick.el from contrib into the emacs directory and then build
notmuch as usual. There are two very small patches to mainline code:
one to compile and load the pick file if present and one small tweak
to notmuch-show.el. Then in contrib/notmuch-pick there are three
files: the notmuch-pick.el file itself, a README describing
documenting install and use, and a TODO which contains the main things
I think need doing (and I will try to update this in light of comments
received).

Any comments on the code or the correct approach, patches etc
gratefully received!

Best wishes

Mark

PS If it is in contrib then I am happy to maintain it (with the aim of
an eventual move to mainline).


Mark Walters (5):
  emacs: compile and load notmuch-pick.el if present.
  emacs: make notmuch-show return its buffer
  contrib: add notmuch-pick.el file itself
  contrib: Added pick README.
  contrib: add pick TODO file

 contrib/notmuch-pick/README  |   36 ++
 contrib/notmuch-pick/TODO|   25 +
 contrib/notmuch-pick/notmuch-pick.el |  876 ++
 emacs/Makefile.local |3 +-
 emacs/notmuch-show.el|3 +-
 emacs/notmuch.el |5 +
 6 files changed, 946 insertions(+), 2 deletions(-)
 create mode 100644 contrib/notmuch-pick/README
 create mode 100644 contrib/notmuch-pick/TODO
 create mode 100644 contrib/notmuch-pick/notmuch-pick.el

-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/5] emacs: make notmuch-show return its buffer

[Mark Walters]

notmuch-pick uses the returned buffer to try and make sure it does not
close the wrong buffer.
---
 emacs/notmuch-show.el |3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6335d45..1cabd17 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1031,7 +1031,8 @@ function is used.
  notmuch-show-parent-buffer parent-buffer
  notmuch-show-query-context query-context)
 (notmuch-show-build-buffer)
-(notmuch-show-goto-first-wanted-message)))
+(notmuch-show-goto-first-wanted-message)
+(current-buffer)))
 
 (defun notmuch-show-build-buffer ()
   (let ((inhibit-read-only t))
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/5] emacs: compile and load notmuch-pick.el if present.

[Mark Walters]

Compile and load notmuch-pick.el if present.

All the actual setup of pick is done in the function notmuch-pick-init
so we call that in the notmuch init function if it is bound. This
function will setup all extra keybinding etc.
---
 emacs/Makefile.local |3 ++-
 emacs/notmuch.el |5 +
 2 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/emacs/Makefile.local b/emacs/Makefile.local
index fb82247..9f4dba6 100644
--- a/emacs/Makefile.local
+++ b/emacs/Makefile.local
@@ -15,7 +15,8 @@ emacs_sources := \
$(dir)/notmuch-crypto.el \
$(dir)/notmuch-tag.el \
$(dir)/coolj.el \
-   $(dir)/notmuch-print.el
+   $(dir)/notmuch-print.el \
+   $(wildcard $(dir)/notmuch-pick.el)
 
 emacs_images := \
$(srcdir)/$(dir)/notmuch-logo.png
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index fd1836f..4f3da4f 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -59,6 +59,9 @@
 (require 'notmuch-maildir-fcc)
 (require 'notmuch-message)
 
+;; Load notmuch-pick if available (but do not error if not present).
+(load notmuch-pick t)
+
 (defcustom notmuch-search-result-format
   `((date . %12s )
 (count . %-7s )
@@ -1088,6 +1091,8 @@ current search results AND that are tagged with the given 
tag.
 (defun notmuch ()
   Run notmuch and display saved searches, known tags, etc.
   (interactive)
+  (when (fboundp 'notmuch-pick-init)
+(notmuch-pick-init))
   (notmuch-hello))
 
 (defun notmuch-interesting-buffer (b)
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/5] contrib: Added pick README.

[Mark Walters]

---
 contrib/notmuch-pick/README |   36 
 1 files changed, 36 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/README

diff --git a/contrib/notmuch-pick/README b/contrib/notmuch-pick/README
new file mode 100644
index 000..1b2399f
--- /dev/null
+++ b/contrib/notmuch-pick/README
@@ -0,0 +1,36 @@
+Notmuch Pick
+
+Notmuch pick is an experimental threaded message view for the emacs
+interface. Each message is one line in the results and the thread
+structure is shown using UFT-8 box drawing characters (similar to
+Mutt's threaded view). It comes between search and show in terms of
+amount of output and can be useful for viewing both single theads and
+multiple threads.
+
+Install
+
+To use notmuch pick copy or link the notmuch-pick.el file into the
+emacs directory of the main source and then compile and install
+notmuch as usual.
+
+Using Pick
+
+The main key entries to notmuch pick are
+
+'z' enter a query to view using notmuch pick (works in hello, search,
+show and pick itself).
+'Z' view the current query in pick (works from search and show)
+'M-RET' view the selected thread in pick (works in search mode)
+
+Once in pick mode keybinding are mostly inline with the rest of
+notmuch and are all viewable with '?' as usual.
+
+Customising Pick
+
+Pick has several customisation variables. The most significant is the
+first notmuch-pick-show-out which determines the behaviour when
+selecting a message (with RET) in the pick view. By default pick uses
+a split window showing the single message in the bottom pane. However,
+if this option is set then it views the whole thread in the complete
+window jumping to the selected message in the thread. In either case
+M-RET selects the other option.
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] contrib: add pick TODO file

[Mark Walters]

---
 contrib/notmuch-pick/TODO |   25 +
 1 files changed, 25 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/TODO

diff --git a/contrib/notmuch-pick/TODO b/contrib/notmuch-pick/TODO
new file mode 100644
index 000..2d0796e
--- /dev/null
+++ b/contrib/notmuch-pick/TODO
@@ -0,0 +1,25 @@
+TODO for notmuch-pick
+
+(These are the things I can think of: to be added to as problems get
+reported or found!)
+
+Things that need fixing before acceptance to mainline
+
+- Review lisp to make idiomatic.
+- Unify functions with search or show where appropriate.
+- Work out a fall-back if the font does not contain box graphic characters.
+- Add extra functionality?
+
+- Decide whether the asynchronous parser is satisfactory.
+- Remove debugging/timing information.
+
+- Add tests (I have some but I am not sure how to add them if pick is
+  in contrib).
+
+Bugs:
+
+- The display flickers while pick is running. I have no idea why.
+
+Other todo items
+
+?
\ No newline at end of file
-- 
1.7.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/5] contrib: add notmuch-pick.el file itself

[Mark Walters]

This adds the main notmuch-pick.el file.
---
 contrib/notmuch-pick/notmuch-pick.el |  876 ++
 1 files changed, 876 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/notmuch-pick.el

diff --git a/contrib/notmuch-pick/notmuch-pick.el 
b/contrib/notmuch-pick/notmuch-pick.el
new file mode 100644
index 000..ded0c42
--- /dev/null
+++ b/contrib/notmuch-pick/notmuch-pick.el
@@ -0,0 +1,876 @@
+;; notmuch-pick.el --- displaying notmuch forests.
+;;
+;; Copyright © Carl Worth
+;; Copyright © David Edmondson
+;; Copyright © Mark Walters
+;;
+;; This file is part of Notmuch.
+;;
+;; Notmuch is free software: you can redistribute it and/or modify it
+;; under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+;;
+;; Notmuch is distributed in the hope that it will be useful, but
+;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;; General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with Notmuch.  If not, see http://www.gnu.org/licenses/.
+;;
+;; Authors: David Edmondson d...@dme.org
+;;  Mark Walters markwalters1...@gmail.com
+
+(require 'mail-parse)
+
+(require 'notmuch-lib)
+(require 'notmuch-query)
+(require 'notmuch-show)
+(eval-when-compile (require 'cl))
+
+(declare-function notmuch-call-notmuch-process notmuch (rest args))
+(declare-function notmuch-show notmuch-show (rest args))
+(declare-function notmuch-tag notmuch (query rest tags))
+(declare-function notmuch-show-strip-re notmuch-show (subject))
+(declare-function notmuch-show-clean-address notmuch-show (parsed-address))
+(declare-function notmuch-show-spaces-n notmuch-show (n))
+(declare-function notmuch-read-query notmuch (prompt))
+(declare-function notmuch-read-tag-changes notmuch (optional initial-input 
rest search-terms))
+(declare-function notmuch-update-tags notmuch (current-tags tag-changes))
+(declare-function notmuch-hello-trim notmuch-hello (search))
+(declare-function notmuch-search-find-thread-id notmuch ())
+(declare-function notmuch-search-find-subject notmuch ())
+
+;; the following variable is defined in notmuch.el
+(defvar notmuch-search-query-string)
+
+(defvar notmuch-pick-process-state nil
+  Parsing state of the search process filter.)
+
+(defgroup notmuch-pick nil
+  Showing message and thread structure.
+  :group 'notmuch)
+
+;; This is ugly. We can't run setup-show-out until it has been defined
+;; which needs the keymap to be defined. So we defer setting up to
+;; notmuch-pick-init.
+(defcustom notmuch-pick-show-out nil
+  View selected messages in new window rather than split-pane.
+  :type 'boolean
+  :group 'notmuch-pick
+  :set (lambda (symbol value)
+(set-default symbol value)
+(when (fboundp 'notmuch-pick-setup-show-out)
+  (notmuch-pick-setup-show-out
+
+(defcustom notmuch-pick-result-format
+  `((date . %12s )
+(authors . %21s )
+(subject . %-54s )
+(tags . (%s)))
+  Result formatting for Pick. Supported fields are:
+date, authors, subject, tags
+Note subject includes the tree structure graphics.
+For example:
+(setq notmuch-pick-result-format \(\(\authors\ . \%-40s\\)
+ \(\subject\ . \%s\\)\)\)
+  :type '(alist :key-type (string) :value-type (string))
+  :group 'notmuch-pick)
+
+(defcustom notmuch-pick-asynchronous-parser t
+  Use the asynchronous parser.
+  :type 'boolean
+  :group 'notmuch-pick)
+
+;; Faces for messages that match the query.
+(defface notmuch-pick-match-date-face
+  '((t :inherit default))
+  Face used in pick mode for the date in messages matching the query.
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-author-face
+  'class color)
+  (background dark))
+ (:foreground OliveDrab1))
+(((class color)
+  (background light))
+ (:foreground dark blue))
+(t
+ (:bold t)))
+  Face used in pick mode for the date in messages matching the query.
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-subject-face
+  '((t :inherit default))
+  Face used in pick mode for the subject in messages matching the query.
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-tag-face
+  'class color)
+  (background dark))
+ (:foreground OliveDrab1))
+(((class color)
+  (background light))
+ (:foreground navy blue :bold t))
+(t
+ (:bold t)))
+  Face used in pick mode for tags in messages matching the query.
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+;; Faces for messages that do not match the query.
+(defface notmuch-pick-no-match-date-face
+  '((t (:foreground gray)))
+  Face used in pick mode for non-matching dates.
+  :group 

[PATCH 04/13] show: Associate an sprinter with each format

[Austin Clements]

This associates an sprinter constructor with each show format and uses
this to construct the appropriate sprinter.  Currently nothing is done
with this sprinter, but the following patches will weave it through
the layers of notmuch show.
---
 notmuch-client.h |2 ++
 notmuch-show.c   |9 +
 2 files changed, 11 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index f930798..e4172a2 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -66,9 +66,11 @@ typedef GMimeCipherContext notmuch_crypto_context_t;
 #define STRINGIFY_(s) #s
 
 typedef struct mime_node mime_node_t;
+typedef struct sprinter sprinter_t;
 struct notmuch_show_params;
 
 typedef struct notmuch_show_format {
+sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
 const char *message_set_start;
 notmuch_status_t (*part) (const void *ctx,
  struct mime_node *node, int indent,
diff --git a/notmuch-show.c b/notmuch-show.c
index d3419e4..d04943f 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -20,12 +20,14 @@
 
 #include notmuch-client.h
 #include gmime-filter-reply.h
+#include sprinter.h
 
 static notmuch_status_t
 format_part_text (const void *ctx, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_text = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_text,
 };
 
@@ -34,6 +36,7 @@ format_part_json_entry (const void *ctx, mime_node_t *node,
int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_json = {
+.new_sprinter = sprinter_json_create,
 .message_set_start = [,
 .part = format_part_json_entry,
 .message_set_sep = , ,
@@ -46,6 +49,7 @@ format_part_mbox (const void *ctx, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_mbox = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_mbox,
 };
 
@@ -55,6 +59,7 @@ format_part_raw (unused (const void *ctx), mime_node_t *node,
 unused (const notmuch_show_params_t *params));
 
 static const notmuch_show_format_t format_raw = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_raw,
 };
 
@@ -1003,6 +1008,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
 char *query_string;
 int opt_index, ret;
 const notmuch_show_format_t *format = format_text;
+sprinter_t *sprinter;
 notmuch_show_params_t params = {
.part = -1,
.omit_excluded = TRUE,
@@ -1130,6 +1136,9 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
return 1;
 }
 
+/* Create structure printer. */
+sprinter = format-new_sprinter(ctx, stdout);
+
 /* If a single message is requested we do not use search_excludes. */
 if (params.part = 0)
ret = do_show_single (ctx, query, format, params);
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 09/13] show: Convert non-envelope format_part_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |   74 +++-
 1 file changed, 46 insertions(+), 28 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 3ff32df..afbd9d0 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -588,7 +588,6 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
return;
 }
 
-void *local = talloc_new (ctx);
 /* The disposition and content-type metadata are associated with
  * the envelope for message parts */
 GMimeObject *meta = node-envelope_part ?
@@ -597,31 +596,41 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
 const char *cid = g_mime_object_get_content_id (meta);
 const char *filename = GMIME_IS_PART (node-part) ?
g_mime_part_get_filename (GMIME_PART (node-part)) : NULL;
-const char *terminator = ;
+int nclose = 0;
 int i;
 
-if (!first)
-   printf (, );
+sp-begin_map (sp);
 
-printf ({\id\: %d, node-part_num);
+sp-map_key (sp, id);
+sp-integer (sp, node-part_num);
 
-if (node-decrypt_attempted)
-   printf (, \encstatus\: [{\status\: \%s\}],
-   node-decrypt_success ? good : bad);
+if (node-decrypt_attempted) {
+   sp-map_key (sp, encstatus);
+   sp-begin_list (sp);
+   sp-begin_map (sp);
+   sp-map_key (sp, status);
+   sp-string (sp, node-decrypt_success ? good : bad);
+   sp-end (sp);
+   sp-end (sp);
+}
 
 if (node-verify_attempted) {
-   printf (, \sigstatus\: );
+   sp-map_key (sp, sigstatus);
format_part_sigstatus_json (sp, node);
 }
 
-printf (, \content-type\: %s,
-   json_quote_str (local, g_mime_content_type_to_string 
(content_type)));
+sp-map_key (sp, content-type);
+sp-string (sp, g_mime_content_type_to_string (content_type));
 
-if (cid)
-   printf (, \content-id\: %s, json_quote_str (local, cid));
+if (cid) {
+   sp-map_key (sp, content-id);
+   sp-string (sp, cid);
+}
 
-if (filename)
-   printf (, \filename\: %s, json_quote_str (local, filename));
+if (filename) {
+   sp-map_key (sp, filename);
+   sp-string (sp, filename);
+}
 
 if (GMIME_IS_PART (node-part)) {
/* For non-HTML text parts, we include the content in the
@@ -636,35 +645,44 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
if (g_mime_content_type_is_type (content_type, text, html)) {
const char *content_charset = 
g_mime_object_get_content_type_parameter (meta, charset);
 
-   if (content_charset != NULL)
-   printf (, \content-charset\: %s, json_quote_str (local, 
content_charset));
+   if (content_charset != NULL) {
+   sp-map_key (sp, content-charset);
+   sp-string (sp, content_charset);
+   }
} else if (g_mime_content_type_is_type (content_type, text, *)) {
GMimeStream *stream_memory = g_mime_stream_mem_new ();
GByteArray *part_content;
show_text_part_content (node-part, stream_memory, 0);
part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM 
(stream_memory));
-
-   printf (, \content\: %s, json_quote_chararray (local, (char *) 
part_content-data, part_content-len));
+   sp-map_key (sp, content);
+   sp-string_len (sp, (char *) part_content-data, part_content-len);
g_object_unref (stream_memory);
}
 } else if (GMIME_IS_MULTIPART (node-part)) {
-   printf (, \content\: [);
-   terminator = ];
+   sp-map_key (sp, content);
+   sp-begin_list (sp);
+   nclose = 1;
 } else if (GMIME_IS_MESSAGE (node-part)) {
-   printf (, \content\: [{);
-   printf (\headers\: );
+   sp-map_key (sp, content);
+   sp-begin_list (sp);
+   sp-begin_map (sp);
+
+   sp-map_key (sp, headers);
format_headers_json (sp, GMIME_MESSAGE (node-part), FALSE);
 
-   printf (, \body\: [);
-   terminator = ]}];
+   sp-map_key (sp, body);
+   sp-begin_list (sp);
+   nclose = 3;
 }
 
-talloc_free (local);
-
 for (i = 0; i  node-nchildren; i++)
format_part_json (ctx, sp, mime_node_child (node, i), i == 0, TRUE);
 
-printf (%s}, terminator);
+/* Close content structures */
+for (i = 0; i  nclose; i++)
+   sp-end (sp);
+/* Close part map */
+sp-end (sp);
 }
 
 static notmuch_status_t
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 12/13] show: Convert do_show to use sprinter

[Austin Clements]

---
 notmuch-show.c |   11 ++-
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 5ee2156..a0da48a 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -961,11 +961,9 @@ do_show (void *ctx,
 notmuch_threads_t *threads;
 notmuch_thread_t *thread;
 notmuch_messages_t *messages;
-int first_toplevel = 1;
 notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
 
-if (format-message_set_start)
-   fputs (format-message_set_start, stdout);
+sp-begin_list (sp);
 
 for (threads = notmuch_query_search_threads (query);
 notmuch_threads_valid (threads);
@@ -979,10 +977,6 @@ do_show (void *ctx,
INTERNAL_ERROR (Thread %s has no toplevel messages.\n,
notmuch_thread_get_thread_id (thread));
 
-   if (!first_toplevel  format-message_set_sep)
-   fputs (format-message_set_sep, stdout);
-   first_toplevel = 0;
-
status = show_messages (ctx, format, sp, messages, 0, params);
if (status  !res)
res = status;
@@ -991,8 +985,7 @@ do_show (void *ctx,
 
 }
 
-if (format-message_set_end)
-   fputs (format-message_set_end, stdout);
+sp-end (sp);
 
 return res != NOTMUCH_STATUS_SUCCESS;
 }
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 07/13] show: Convert format_headers_json to use sprinter

[Austin Clements]

This no longer requires a talloc context (not that it really did
before since it didn't return anything), so we remove its context
argument.
---
 notmuch-client.h |3 ++-
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   58 ++
 3 files changed, 31 insertions(+), 32 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 8eeedf7..21c3231 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -183,7 +183,8 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
  notmuch_bool_t first, notmuch_bool_t output_body);
 
 void
-format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
+format_headers_json (sprinter_t *sp, GMimeMessage *message,
+notmuch_bool_t reply);
 
 typedef enum {
 NOTMUCH_SHOW_TEXT_PART_REPLY = 1  0,
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 07d4452..fa6665f 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -617,7 +617,7 @@ notmuch_reply_format_json(void *ctx,
 
 /* The headers of the reply message we've created */
 printf ({\reply-headers\: );
-format_headers_json (ctx, reply, TRUE);
+format_headers_json (sp, reply, TRUE);
 g_object_unref (G_OBJECT (reply));
 reply = NULL;
 
diff --git a/notmuch-show.c b/notmuch-show.c
index b258f65..9852119 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -199,48 +199,46 @@ _is_from_line (const char *line)
 }
 
 void
-format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply)
+format_headers_json (sprinter_t *sp, GMimeMessage *message,
+notmuch_bool_t reply)
 {
-void *local = talloc_new (ctx);
 InternetAddressList *recipients;
 const char *recipients_string;
 
-printf ({%s: %s,
-   json_quote_str (local, Subject),
-   json_quote_str (local, g_mime_message_get_subject (message)));
-printf (, %s: %s,
-   json_quote_str (local, From),
-   json_quote_str (local, g_mime_message_get_sender (message)));
+sp-begin_map (sp);
+
+sp-map_key (sp, Subject);
+sp-string (sp, g_mime_message_get_subject (message));
+
+sp-map_key (sp, From);
+sp-string (sp, g_mime_message_get_sender (message));
+
 recipients = g_mime_message_get_recipients (message, 
GMIME_RECIPIENT_TYPE_TO);
 recipients_string = internet_address_list_to_string (recipients, 0);
-if (recipients_string)
-   printf (, %s: %s,
-   json_quote_str (local, To),
-   json_quote_str (local, recipients_string));
+if (recipients_string) {
+   sp-map_key (sp, To);
+   sp-string (sp, recipients_string);
+}
+
 recipients = g_mime_message_get_recipients (message, 
GMIME_RECIPIENT_TYPE_CC);
 recipients_string = internet_address_list_to_string (recipients, 0);
-if (recipients_string)
-   printf (, %s: %s,
-   json_quote_str (local, Cc),
-   json_quote_str (local, recipients_string));
+if (recipients_string) {
+   sp-map_key (sp, Cc);
+   sp-string (sp, recipients_string);
+}
 
 if (reply) {
-   printf (, %s: %s,
-   json_quote_str (local, In-reply-to),
-   json_quote_str (local, g_mime_object_get_header (GMIME_OBJECT 
(message), In-reply-to)));
+   sp-map_key (sp, In-reply-to);
+   sp-string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
In-reply-to));
 
-   printf (, %s: %s,
-   json_quote_str (local, References),
-   json_quote_str (local, g_mime_object_get_header (GMIME_OBJECT 
(message), References)));
+   sp-map_key (sp, References);
+   sp-string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
References));
 } else {
-   printf (, %s: %s,
-   json_quote_str (local, Date),
-   json_quote_str (local, g_mime_message_get_date_as_string 
(message)));
+   sp-map_key (sp, Date);
+   sp-string (sp, g_mime_message_get_date_as_string (message));
 }
 
-printf (});
-
-talloc_free (local);
+sp-end (sp);
 }
 
 /* Write a MIME text part out to the given stream.
@@ -575,7 +573,7 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
format_message_json (ctx, node-envelope_file);
 
printf (\headers\: );
-   format_headers_json (ctx, GMIME_MESSAGE (node-part), FALSE);
+   format_headers_json (sp, GMIME_MESSAGE (node-part), FALSE);
 
if (output_body) {
printf (, \body\: [);
@@ -651,7 +649,7 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
 } else if (GMIME_IS_MESSAGE (node-part)) {
printf (, \content\: [{);
printf (\headers\: );
-   format_headers_json (local, GMIME_MESSAGE (node-part), FALSE);
+   format_headers_json (sp, GMIME_MESSAGE (node-part), FALSE);
 
printf (, \body\: [);
terminator = ]}];
-- 
1.7.10

[PATCH 13/13] show: Remove now unused fields from notmuch_show_format

[Austin Clements]

The message_set_{begin,sep,end} and null_message fields are no longer
used because we now use the structure printer provided by the format.
---
 notmuch-client.h |4 
 notmuch-show.c   |4 
 2 files changed, 8 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 21c3231..de31aa1 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -71,13 +71,9 @@ struct notmuch_show_params;
 
 typedef struct notmuch_show_format {
 sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
-const char *message_set_start;
 notmuch_status_t (*part) (const void *ctx, sprinter_t *sprinter,
  struct mime_node *node, int indent,
  const struct notmuch_show_params *params);
-const char *message_set_sep;
-const char *message_set_end;
-const char *null_message;
 } notmuch_show_format_t;
 
 typedef struct notmuch_crypto {
diff --git a/notmuch-show.c b/notmuch-show.c
index a0da48a..f5bc098 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -37,11 +37,7 @@ format_part_json_entry (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
 
 static const notmuch_show_format_t format_json = {
 .new_sprinter = sprinter_json_create,
-.message_set_start = [,
 .part = format_part_json_entry,
-.message_set_sep = , ,
-.message_set_end = ],
-.null_message = null
 };
 
 static notmuch_status_t
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 11/13] show: Convert show_message to use sprinter

[Austin Clements]

Unlike the previous patches, this function is used for all formats.
However, for formats other than the JSON format, the sprinter methods
used by show_message are all no-ops, so this code continues to
function correctly for all of the formats.

Converting show_message eliminates show_null_message in the process,
since this maps directly to an sprinter method.
---
 notmuch-show.c |   31 +--
 1 file changed, 5 insertions(+), 26 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index fa1e6e9..5ee2156 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -842,15 +842,6 @@ format_part_raw (unused (const void *ctx), unused 
(sprinter_t *sp),
 }
 
 static notmuch_status_t
-show_null_message (const notmuch_show_format_t *format)
-{
-/* Output a null message. Currently empty for all formats except Json */
-if (format-null_message)
-   printf (%s, format-null_message);
-return NOTMUCH_STATUS_SUCCESS;
-}
-
-static notmuch_status_t
 show_message (void *ctx,
  const notmuch_show_format_t *format,
  sprinter_t *sp,
@@ -884,23 +875,16 @@ show_messages (void *ctx,
 notmuch_message_t *message;
 notmuch_bool_t match;
 notmuch_bool_t excluded;
-int first_set = 1;
 int next_indent;
 notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
 
-if (format-message_set_start)
-   fputs (format-message_set_start, stdout);
+sp-begin_list (sp);
 
 for (;
 notmuch_messages_valid (messages);
 notmuch_messages_move_to_next (messages))
 {
-   if (!first_set  format-message_set_sep)
-   fputs (format-message_set_sep, stdout);
-   first_set = 0;
-
-   if (format-message_set_start)
-   fputs (format-message_set_start, stdout);
+   sp-begin_list (sp);
 
message = notmuch_messages_get (messages);
 
@@ -915,12 +899,9 @@ show_messages (void *ctx,
res = status;
next_indent = indent + 1;
} else {
-   status = show_null_message (format);
+   sp-null (sp);
}
 
-   if (!status  format-message_set_sep)
-   fputs (format-message_set_sep, stdout);
-
status = show_messages (ctx,
format, sp,
notmuch_message_get_replies (message),
@@ -931,12 +912,10 @@ show_messages (void *ctx,
 
notmuch_message_destroy (message);
 
-   if (format-message_set_end)
-   fputs (format-message_set_end, stdout);
+   sp-end (sp);
 }
 
-if (format-message_set_end)
-   fputs (format-message_set_end, stdout);
+sp-end (sp);
 
 return res;
 }
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 01/13] test: Uniformly canonicalize actual and expected JSON

[Austin Clements]

Previously, we used a variety of ad-hoc canonicalizations for JSON
output in the test suite, but were ultimately very sensitive to JSON
irrelevancies such as whitespace.  This introduces a new test
comparison function, test_expect_equal_json, that first pretty-prints
*both* the actual and expected JSON and the compares the result.

The current implementation of this simply uses Python's json.tool to
perform pretty-printing (with a fallback to the identity function if
parsing fails).  However, since the interface it introduces is
semantically high-level, we could swap in other mechanisms in the
future, such as another pretty-printer or something that does not
re-order object keys (if we decide that we care about that).

In general, this patch does not remove the existing ad-hoc
canonicalization because it does no harm.  We do have to remove the
newline-after-comma rule from notmuch_json_show_sanitize and
filter_show_json because it results in invalid JSON that cannot be
pretty-printed.

Most of this patch simply replaces test_expect_equal and
test_expect_equal_file with test_expect_equal_json.  It changes the
expected JSON in a few places where sanitizers had placed newlines
after commas inside strings.
---
 test/crypto|   37 +++--
 test/json  |   14 +++---
 test/maildir-sync  |   11 ---
 test/multipart |   34 +++---
 test/search-output |2 +-
 test/test-lib.sh   |   17 +
 6 files changed, 55 insertions(+), 60 deletions(-)

diff --git a/test/crypto b/test/crypto
index be752b1..5dd14c4 100755
--- a/test/crypto
+++ b/test/crypto
@@ -51,8 +51,7 @@ expected='[[[{id: X,
  headers: {Subject: test signed message 001,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  sigstatus: [{status: good,
  fingerprint: '$FINGERPRINT',
@@ -64,7 +63,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 
@@ -85,8 +84,7 @@ expected='[[[{id: X,
  headers: {Subject: test signed message 001,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  sigstatus: [{status: good,
  fingerprint: '$FINGERPRINT',
@@ -99,7 +97,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 
@@ -119,8 +117,7 @@ expected='[[[{id: X,
  headers: {Subject: test signed message 001,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  sigstatus: [{status: error,
  keyid: '$(echo $FINGERPRINT | cut -c 25-)',
@@ -132,7 +129,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/pgp-signature}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 mv ${GNUPGHOME}{.bak,}
@@ -193,8 +190,7 @@ expected='[[[{id: X,
  headers: {Subject: test encrypted message 001,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  encstatus: [{status: good}],
  sigstatus: [],
@@ -210,7 +206,7 @@ expected='[[[{id: X,
  content-type: application/octet-stream,
  filename: TESTATTACHMENT}]}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 
@@ -221,7 +217,7 @@ output=$(notmuch show --format=json --part=4 --decrypt 
subject:test encrypted m
 expected='{id: 4,
  content-type: text/plain,
  content: This is a test encrypted message.\n}'
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 
@@ -248,8 +244,7 @@ expected='[[[{id: X,
  headers: {Subject: test encrypted message 001,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  encstatus: [{status: bad}],
  content-type: multipart/encrypted,
@@ -258,7 +253,7 @@ expected='[[[{id: X,
  {id: 3,
  content-type: application/octet-stream}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 $output \
 $expected
 mv ${GNUPGHOME}{.bak,}
@@ -283,8 +278,7 @@ expected='[[[{id: X,
  headers: {Subject: test encrypted message 002,
  From: Notmuch Test Suite test_su...@notmuchmail.org,
  To: test_su...@notmuchmail.org,
- Date: Sat,
- 01 Jan 2000 12:00:00 +},
+ Date: Sat, 01 Jan 2000 12:00:00 +},
  body: [{id: 1,
  encstatus: [{status: 

[PATCH 10/13] show: Convert envelope format_part_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |   57 +---
 1 file changed, 34 insertions(+), 23 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index afbd9d0..fa1e6e9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -110,34 +110,44 @@ _get_one_line_summary (const void *ctx, notmuch_message_t 
*message)
 }
 
 static void
-format_message_json (const void *ctx, notmuch_message_t *message)
+format_message_json (sprinter_t *sp, notmuch_message_t *message)
 {
+void *local = talloc_new (NULL);
 notmuch_tags_t *tags;
-int first = 1;
-void *ctx_quote = talloc_new (ctx);
 time_t date;
 const char *relative_date;
 
 date = notmuch_message_get_date (message);
-relative_date = notmuch_time_relative_date (ctx, date);
+relative_date = notmuch_time_relative_date (local, date);
+
+sp-map_key (sp, id);
+sp-string (sp, notmuch_message_get_message_id (message));
+
+sp-map_key (sp, match);
+sp-boolean (sp, notmuch_message_get_flag (message, 
NOTMUCH_MESSAGE_FLAG_MATCH));
+
+sp-map_key (sp, excluded);
+sp-boolean (sp, notmuch_message_get_flag (message, 
NOTMUCH_MESSAGE_FLAG_EXCLUDED));
+
+sp-map_key (sp, filename);
+sp-string (sp, notmuch_message_get_filename (message));
+
+sp-map_key (sp, timestamp);
+date = notmuch_message_get_date (message);
+sp-integer (sp, date);
 
-printf (\id\: %s, \match\: %s, \excluded\: %s, \filename\: %s, 
\timestamp\: %ld, \date_relative\: \%s\, \tags\: [,
-   json_quote_str (ctx_quote, notmuch_message_get_message_id 
(message)),
-   notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? 
true : false,
-   notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED) ? 
true : false,
-   json_quote_str (ctx_quote, notmuch_message_get_filename (message)),
-   date, relative_date);
+sp-map_key (sp, date_relative);
+sp-string (sp, relative_date);
 
+sp-map_key (sp, tags);
+sp-begin_list (sp);
 for (tags = notmuch_message_get_tags (message);
 notmuch_tags_valid (tags);
 notmuch_tags_move_to_next (tags))
-{
- printf(%s%s, first ?  : ,,
-   json_quote_str (ctx_quote, notmuch_tags_get (tags)));
- first = 0;
-}
-printf(], );
-talloc_free (ctx_quote);
+   sp-string (sp, notmuch_tags_get (tags));
+sp-end (sp);
+
+talloc_free (local);
 }
 
 /* Extract just the email address from the contents of a From:
@@ -573,18 +583,19 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
  * devel/schemata. */
 
 if (node-envelope_file) {
-   printf ({);
-   format_message_json (ctx, node-envelope_file);
+   sp-begin_map (sp);
+   format_message_json (sp, node-envelope_file);
 
-   printf (\headers\: );
+   sp-map_key (sp, headers);
format_headers_json (sp, GMIME_MESSAGE (node-part), FALSE);
 
if (output_body) {
-   printf (, \body\: [);
+   sp-map_key (sp, body);
+   sp-begin_list (sp);
format_part_json (ctx, sp, mime_node_child (node, 0), first, TRUE);
-   printf (]);
+   sp-end (sp);
}
-   printf (});
+   sp-end (sp);
return;
 }
 
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 08/13] show: Convert format_part_sigstatus_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |  118 +---
 1 file changed, 61 insertions(+), 57 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 9852119..3ff32df 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -337,134 +337,138 @@ signer_status_to_string (GMimeSignerStatus x)
 
 #ifdef GMIME_ATLEAST_26
 static void
-format_part_sigstatus_json (mime_node_t *node)
+format_part_sigstatus_json (sprinter_t *sp, mime_node_t *node)
 {
 GMimeSignatureList *siglist = node-sig_list;
 
-printf ([);
+sp-begin_list (sp);
 
 if (!siglist) {
-   printf (]);
+   sp-end (sp);
return;
 }
 
-void *ctx_quote = talloc_new (NULL);
 int i;
 for (i = 0; i  g_mime_signature_list_length (siglist); i++) {
GMimeSignature *signature = g_mime_signature_list_get_signature 
(siglist, i);
 
-   if (i  0)
-   printf (, );
-
-   printf ({);
+   sp-begin_map (sp);
 
/* status */
GMimeSignatureStatus status = g_mime_signature_get_status (signature);
-   printf (\status\: %s,
-   json_quote_str (ctx_quote,
-   signature_status_to_string (status)));
+   sp-map_key (sp, status);
+   sp-string (sp, signature_status_to_string (status));
 
GMimeCertificate *certificate = g_mime_signature_get_certificate 
(signature);
if (status == GMIME_SIGNATURE_STATUS_GOOD) {
-   if (certificate)
-   printf (, \fingerprint\: %s, json_quote_str (ctx_quote, 
g_mime_certificate_get_fingerprint (certificate)));
+   if (certificate) {
+   sp-map_key (sp, fingerprint);
+   sp-string (sp, g_mime_certificate_get_fingerprint 
(certificate));
+   }
/* these dates are seconds since the epoch; should we
 * provide a more human-readable format string? */
time_t created = g_mime_signature_get_created (signature);
-   if (created != -1)
-   printf (, \created\: %d, (int) created);
+   if (created != -1) {
+   sp-map_key (sp, created);
+   sp-integer (sp, created);
+   }
time_t expires = g_mime_signature_get_expires (signature);
-   if (expires  0)
-   printf (, \expires\: %d, (int) expires);
+   if (expires  0) {
+   sp-map_key (sp, expires);
+   sp-integer (sp, expires);
+   }
/* output user id only if validity is FULL or ULTIMATE. */
/* note that gmime is using the term trust here, which
 * is WRONG.  It's actually user id validity. */
if (certificate) {
const char *name = g_mime_certificate_get_name (certificate);
GMimeCertificateTrust trust = g_mime_certificate_get_trust 
(certificate);
-   if (name  (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE))
-   printf (, \userid\: %s, json_quote_str (ctx_quote, 
name));
+   if (name  (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)) {
+   sp-map_key (sp, userid);
+   sp-string (sp, name);
+   }
}
} else if (certificate) {
const char *key_id = g_mime_certificate_get_key_id (certificate);
-   if (key_id)
-   printf (, \keyid\: %s, json_quote_str (ctx_quote, key_id));
+   if (key_id) {
+   sp-map_key (sp, keyid);
+   sp-string (sp, key_id);
+   }
}
 
GMimeSignatureError errors = g_mime_signature_get_errors (signature);
if (errors != GMIME_SIGNATURE_ERROR_NONE) {
-   printf (, \errors\: %d, errors);
+   sp-map_key (sp, errors);
+   sp-integer (sp, errors);
}
 
-   printf (});
+   sp-end (sp);
  }
 
-printf (]);
-
-talloc_free (ctx_quote);
+sp-end (sp);
 }
 #else
 static void
-format_part_sigstatus_json (mime_node_t *node)
+format_part_sigstatus_json (sprinter_t *sp, mime_node_t *node)
 {
 const GMimeSignatureValidity* validity = node-sig_validity;
 
-printf ([);
+sp-begin_list (sp);
 
 if (!validity) {
-   printf (]);
+   sp-end (sp);
return;
 }
 
 const GMimeSigner *signer = g_mime_signature_validity_get_signers 
(validity);
-int first = 1;
-void *ctx_quote = talloc_new (NULL);
-
 while (signer) {
-   if (first)
-   first = 0;
-   else
-   printf (, );
-
-   printf ({);
+   sp-begin_map (sp);
 
/* status */
-   printf (\status\: %s,
-   json_quote_str (ctx_quote,
-   signer_status_to_string (signer-status)));
+   sp-map_key (sp, status);
+   sp-string (sp, signer_status_to_string (signer-status));
 
if (signer-status == GMIME_SIGNER_STATUS_GOOD)
{

[PATCH 06/13] show: Feed the sprinter down to part formatters

[Austin Clements]

There are several levels of function calls between where we create the
sprinter and the call to the part formatter in show_message. This
feeds the sprinter through all of them and into the part formatters.
---
 notmuch-client.h |5 +++--
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   50 +-
 3 files changed, 33 insertions(+), 24 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index e4172a2..8eeedf7 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -72,7 +72,7 @@ struct notmuch_show_params;
 typedef struct notmuch_show_format {
 sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
 const char *message_set_start;
-notmuch_status_t (*part) (const void *ctx,
+notmuch_status_t (*part) (const void *ctx, sprinter_t *sprinter,
  struct mime_node *node, int indent,
  const struct notmuch_show_params *params);
 const char *message_set_sep;
@@ -179,7 +179,8 @@ notmuch_status_t
 show_one_part (const char *filename, int part);
 
 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body);
+format_part_json (const void *ctx, sprinter_t *sp, mime_node_t *node,
+ notmuch_bool_t first, notmuch_bool_t output_body);
 
 void
 format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
diff --git a/notmuch-reply.c b/notmuch-reply.c
index e42ba79..07d4452 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -624,7 +624,7 @@ notmuch_reply_format_json(void *ctx,
 /* Start the original */
 printf (, \original\: );
 
-format_part_json (ctx, node, TRUE, TRUE);
+format_part_json (ctx, sp, node, TRUE, TRUE);
 
 /* End */
 printf (}\n);
diff --git a/notmuch-show.c b/notmuch-show.c
index d04943f..b258f65 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -23,7 +23,7 @@
 #include sprinter.h
 
 static notmuch_status_t
-format_part_text (const void *ctx, mime_node_t *node,
+format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_text = {
@@ -32,7 +32,7 @@ static const notmuch_show_format_t format_text = {
 };
 
 static notmuch_status_t
-format_part_json_entry (const void *ctx, mime_node_t *node,
+format_part_json_entry (const void *ctx, sprinter_t *sp, mime_node_t *node,
int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_json = {
@@ -45,7 +45,7 @@ static const notmuch_show_format_t format_json = {
 };
 
 static notmuch_status_t
-format_part_mbox (const void *ctx, mime_node_t *node,
+format_part_mbox (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);
 
 static const notmuch_show_format_t format_mbox = {
@@ -54,7 +54,7 @@ static const notmuch_show_format_t format_mbox = {
 };
 
 static notmuch_status_t
-format_part_raw (unused (const void *ctx), mime_node_t *node,
+format_part_raw (unused (const void *ctx), sprinter_t *sp, mime_node_t *node,
 unused (int indent),
 unused (const notmuch_show_params_t *params));
 
@@ -471,7 +471,7 @@ format_part_sigstatus_json (mime_node_t *node)
 #endif
 
 static notmuch_status_t
-format_part_text (const void *ctx, mime_node_t *node,
+format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params)
 {
 /* The disposition and content-type metadata are associated with
@@ -553,7 +553,7 @@ format_part_text (const void *ctx, mime_node_t *node,
 }
 
 for (i = 0; i  node-nchildren; i++)
-   format_part_text (ctx, mime_node_child (node, i), indent, params);
+   format_part_text (ctx, sp, mime_node_child (node, i), indent, params);
 
 if (GMIME_IS_MESSAGE (node-part))
printf (\fbody}\n);
@@ -564,7 +564,8 @@ format_part_text (const void *ctx, mime_node_t *node,
 }
 
 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body)
+format_part_json (const void *ctx, sprinter_t *sp, mime_node_t *node,
+ notmuch_bool_t first, notmuch_bool_t output_body)
 {
 /* Any changes to the JSON format should be reflected in the file
  * devel/schemata. */
@@ -578,7 +579,7 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first, notm
 
if (output_body) {
printf (, \body\: [);
-   format_part_json (ctx, mime_node_child (node, 0), first, TRUE);
+   format_part_json (ctx, sp, mime_node_child (node, 0), first, TRUE);
printf (]);
}
printf (});
@@ -659,16 +660,17 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first, notm
 talloc_free (local);
 
 for (i = 0; i  node-nchildren; i++)
-

[PATCH 00/13] Convert notmuch show to use structure printers

[Austin Clements]

This patch series converts notmuch show to use the new structure
printer infrastructure, cleaning up the code and paving the way to
easily support other structure types.  There are a lot of patches, but
most of them are small and the conversion is very mechanical.

The first three patches set up prerequisites by making the test
infrastructure more resilient to irrelevant JSON changes and by
introducing a new method to the sprinter.  The next three patches
introduce sprinters to notmuch show and feed them through all of its
layers, though they remain unused.  The remaining patches work their
way back up from the leaves of the JSON printer all the way to
do_show, converting each layer to use the sprinter.  By doing this
conversion from the leaves of the structure up, the output remains
valid and the tests continue to pass at every step.

The full series has been tested with both GMime 2.4 and GMime 2.6.

The first two patches together and the third patch alone stand on
their own and could be pushed before the rest of the series if
desired.

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 05/13] reply: Create a JSON sprinter

[Austin Clements]

---
 notmuch-reply.c |4 
 1 file changed, 4 insertions(+)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index de21f3b..e42ba79 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -22,6 +22,7 @@
 
 #include notmuch-client.h
 #include gmime-filter-headers.h
+#include sprinter.h
 
 static void
 show_reply_headers (GMimeMessage *message)
@@ -596,6 +597,7 @@ notmuch_reply_format_json(void *ctx,
 notmuch_messages_t *messages;
 notmuch_message_t *message;
 mime_node_t *node;
+sprinter_t *sp;
 
 if (notmuch_query_count_messages (query) != 1) {
fprintf (stderr, Error: search term did not match precisely one 
message.\n);
@@ -611,6 +613,8 @@ notmuch_reply_format_json(void *ctx,
 if (!reply)
return 1;
 
+sp = sprinter_json_create (ctx, stdout);
+
 /* The headers of the reply message we've created */
 printf ({\reply-headers\: );
 format_headers_json (ctx, reply, TRUE);
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 03/13] sprinter: Add a string_len method

[Austin Clements]

This method allows callers to output strings with specific lengths.
It's useful both for strings with embedded NULs (which JSON can
represent, though parser support is apparently spotty), and
non-terminated strings.
---
 sprinter-json.c |   11 +--
 sprinter-text.c |   11 +--
 sprinter.h  |1 +
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/sprinter-json.c b/sprinter-json.c
index 4649655..2587ca6 100644
--- a/sprinter-json.c
+++ b/sprinter-json.c
@@ -89,7 +89,7 @@ json_end (struct sprinter *sp)
 }
 
 static void
-json_string (struct sprinter *sp, const char *val)
+json_string_len (struct sprinter *sp, const char *val, size_t len)
 {
 static const char *const escapes[] = {
['\'] = \\\, ['\\'] = , ['\b'] = \\b,
@@ -98,7 +98,7 @@ json_string (struct sprinter *sp, const char *val)
 struct sprinter_json *spj = json_begin_value (sp);
 
 fputc ('', spj-stream);
-for (; *val; ++val) {
+for (; len; ++val, --len) {
unsigned char ch = *val;
if (ch  ARRAY_SIZE (escapes)  escapes[ch])
fputs (escapes[ch], spj-stream);
@@ -111,6 +111,12 @@ json_string (struct sprinter *sp, const char *val)
 }
 
 static void
+json_string (struct sprinter *sp, const char *val)
+{
+json_string_len (sp, val, strlen (val));
+}
+
+static void
 json_integer (struct sprinter *sp, int val)
 {
 struct sprinter_json *spj = json_begin_value (sp);
@@ -166,6 +172,7 @@ sprinter_json_create (const void *ctx, FILE *stream)
.begin_list = json_begin_list,
.end = json_end,
.string = json_string,
+   .string_len = json_string_len,
.integer = json_integer,
.boolean = json_boolean,
.null = json_null,
diff --git a/sprinter-text.c b/sprinter-text.c
index b208840..dfa54b5 100644
--- a/sprinter-text.c
+++ b/sprinter-text.c
@@ -25,14 +25,20 @@ struct sprinter_text {
 };
 
 static void
-text_string (struct sprinter *sp, const char *val)
+text_string_len (struct sprinter *sp, const char *val, size_t len)
 {
 struct sprinter_text *sptxt = (struct sprinter_text *) sp;
 
 if (sptxt-current_prefix != NULL)
fprintf (sptxt-stream, %s:, sptxt-current_prefix);
 
-fputs(val, sptxt-stream);
+fwrite (val, len, 1, sptxt-stream);
+}
+
+static void
+text_string (struct sprinter *sp, const char *val)
+{
+text_string_len (sp, val, strlen (val));
 }
 
 static void
@@ -105,6 +111,7 @@ sprinter_text_create (const void *ctx, FILE *stream)
.begin_list = text_begin_list,
.end = text_end,
.string = text_string,
+   .string_len = text_string_len,
.integer = text_integer,
.boolean = text_boolean,
.null = text_null,
diff --git a/sprinter.h b/sprinter.h
index 6680d41..826a852 100644
--- a/sprinter.h
+++ b/sprinter.h
@@ -28,6 +28,7 @@ typedef struct sprinter {
  * For string, the char * must be UTF-8 encoded.
  */
 void (*string) (struct sprinter *, const char *);
+void (*string_len) (struct sprinter *, const char *, size_t);
 void (*integer) (struct sprinter *, int);
 void (*boolean) (struct sprinter *, notmuch_bool_t);
 void (*null) (struct sprinter *);
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 02/13] test: Remove unnecessary JSON canonicalization

[Austin Clements]

Format canonicalization of JSON output is no longer necessary, so
remove it.  Value canonicalization (e.g., normalizing thread IDs) is
still necessary, so all of the sanitization functions remain.
---
 test/json |4 ++--
 test/maildir-sync |1 -
 test/multipart|   40 ++--
 3 files changed, 12 insertions(+), 33 deletions(-)

diff --git a/test/json b/test/json
index d86ee46..ac8fa8e 100755
--- a/test/json
+++ b/test/json
@@ -18,7 +18,7 @@ test_expect_equal_json $output [[[{\id\: 
\${gen_msg_id}\, \match\: true
 
 test_begin_subtest Search message: json
 add_message [subject]=\json-search-subject\ [date]=\Sat, 01 Jan 2000 
12:00:00 -\ [body]=\json-search-message\
-output=$(notmuch search --format=json json-search-message | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
+output=$(notmuch search --format=json json-search-message | 
notmuch_search_sanitize)
 test_expect_equal_json $output [{\thread\: \XXX\,
  \timestamp\: 946728000,
  \date_relative\: \2000-01-01\,
@@ -49,7 +49,7 @@ test_expect_equal_json $output [[[{\id\: \$id\, 
\match\: true, \exclud
 
 test_begin_subtest Search message: json, utf-8
 add_message [subject]=\json-search-utf8-body-sübjéct\ [date]=\Sat, 01 
Jan 2000 12:00:00 -\ [body]=\jsön-search-méssage\
-output=$(notmuch search --format=json jsön-search-méssage | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
+output=$(notmuch search --format=json jsön-search-méssage | 
notmuch_search_sanitize)
 test_expect_equal_json $output [{\thread\: \XXX\,
  \timestamp\: 946728000,
  \date_relative\: \2000-01-01\,
diff --git a/test/maildir-sync b/test/maildir-sync
index b748d04..cd7d241 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -7,7 +7,6 @@ test_description=maildir synchronization
 # Avoid including the local value of MAIL_DIR in the result.
 filter_show_json() {
 sed -e s|${MAIL_DIR}/|MAIL_DIR/|
-echo
 }
 
 # Create the expected maildir structure
diff --git a/test/multipart b/test/multipart
index 3ccf27f..0527f84 100755
--- a/test/multipart
+++ b/test/multipart
@@ -319,10 +319,8 @@ test_expect_success \
 notmuch show --format=text --part=8 
'id:87liy5ap00@yoom.home.cworth.org'
 
 test_begin_subtest --format=json --part=0, full message
-notmuch show --format=json --part=0 'id:87liy5ap00@yoom.home.cworth.org' | 
sed 's|{id:|\n{id:|g' OUTPUT
-echo OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=0 'id:87liy5ap00@yoom.home.cworth.org' 
OUTPUT
 cat EOF EXPECTED
-
 {id: 87liy5ap00@yoom.home.cworth.org, match: true, excluded: 
false, filename: ${MAIL_DIR}/multipart, timestamp: 978709437, 
date_relative: 2001-01-05, tags: 
[attachment,inbox,signed,unread], headers: {Subject: Multipart 
message, From: Carl Worth cwo...@cworth.org, To: cwo...@cworth.org, 
Date: Fri, 05 Jan 2001 15:43:57 +}, body: [
 {id: 1, content-type: multipart/signed, content: [
 {id: 2, content-type: multipart/mixed, content: [
@@ -337,10 +335,8 @@ EOF
 test_expect_equal_json $(cat OUTPUT) $(cat EXPECTED)
 
 test_begin_subtest --format=json --part=1, message body
-notmuch show --format=json --part=1 'id:87liy5ap00@yoom.home.cworth.org' | 
sed 's|{id:|\n{id:|g' OUTPUT
-echo OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=1 'id:87liy5ap00@yoom.home.cworth.org' 
OUTPUT
 cat EOF EXPECTED
-
 {id: 1, content-type: multipart/signed, content: [
 {id: 2, content-type: multipart/mixed, content: [
 {id: 3, content-type: message/rfc822, content: [{headers: 
{Subject: html message, From: Carl Worth cwo...@cworth.org, To: 
cwo...@cworth.org, Date: Fri, 05 Jan 2001 15:42:57 +}, body: [
@@ -354,10 +350,8 @@ EOF
 test_expect_equal_json $(cat OUTPUT) $(cat EXPECTED)
 
 test_begin_subtest --format=json --part=2, multipart/mixed
-notmuch show --format=json --part=2 'id:87liy5ap00@yoom.home.cworth.org' | 
sed 's|{id:|\n{id:|g' OUTPUT
-echo OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=2 'id:87liy5ap00@yoom.home.cworth.org' 
OUTPUT
 cat EOF EXPECTED
-
 {id: 2, content-type: multipart/mixed, content: [
 {id: 3, content-type: message/rfc822, content: [{headers: 
{Subject: html message, From: Carl Worth cwo...@cworth.org, To: 
cwo...@cworth.org, Date: Fri, 05 Jan 2001 15:42:57 +}, body: [
 {id: 4, content-type: multipart/alternative, content: [
@@ -369,10 +363,8 @@ EOF
 test_expect_equal_json $(cat OUTPUT) $(cat EXPECTED)
 
 test_begin_subtest --format=json --part=3, rfc822 part
-notmuch show --format=json --part=3 'id:87liy5ap00@yoom.home.cworth.org' | 
sed 's|{id:|\n{id:|g' OUTPUT
-echo OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=3 'id:87liy5ap00@yoom.home.cworth.org' 
OUTPUT
 cat EOF EXPECTED
-
 {id: 3, content-type: message/rfc822, content: [{headers: 
{Subject: html message, From: Carl Worth cwo...@cworth.org, To: 
cwo...@cworth.org, Date: Fri, 05 Jan 2001 15:42:57 

Re: [PATCH v2] man: show: update man page for entire-thread and json.

[Austin Clements]

LGTM.

On Tue, 24 Jul 2012, Mark Walters markwalters1...@gmail.com wrote:
 Previously in notmuch show --format=json implied --entire-thread. This
 is still the default but it is now possible to disable this. Update
 the manpage to reflect this.
 ---

 Update the manpage since --format=json and --entire-thread=false is
 now supported. Reworded as Austin suggested.

 Best wishes

 Mark 

  man/man1/notmuch-show.1 |   21 +
  1 files changed, 13 insertions(+), 8 deletions(-)

 diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
 index 506583a..765b22c 100644
 --- a/man/man1/notmuch-show.1
 +++ b/man/man1/notmuch-show.1
 @@ -24,11 +24,14 @@ Supported options for
  include
  .RS 4
  .TP 4
 -.B \-\-entire\-thread
 +.B \-\-entire\-thread=(true|false)
  
 -By default only those messages that match the search terms will be
 -displayed. With this option, all messages in the same thread as any
 -matched message will be displayed.
 +If true,
 +.B notmuch show
 +outputs all messages in the thread of any message matching the search
 +terms; if false, it outputs only the matching messages. For
 +.B --format=json
 +this defaults to true.  For other formats, this defaults to false.
  .RE
  
  .RS 4
 @@ -55,11 +58,13 @@ be nested.
  The output is formatted with Javascript Object Notation (JSON). This
  format is more robust than the text format for automated
  processing. The nested structure of multipart MIME messages is
 -reflected in nested JSON output. JSON output always includes all
 -messages in a matching thread; in effect
 +reflected in nested JSON output. By default JSON output includes all
 +messages in a matching thread; that is, by default,
  .B \-\-format=json
 -implies
 -.B \-\-entire\-thread
 +sets
 +.B \-\-entire\-thread
 +The caller can disable this behaviour by setting
 +.B \-\-entire\-thread=false
  
  .RE
  .RS 4
 -- 
 1.7.9.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/5] Notmuch Pick (WIP or contrib)

[Aneesh Kumar K.V]

Mark Walters markwalters1...@gmail.com writes:

 Hello

 Notmuch pick is an emacs view which displays a threaded view of
 messages: each message has its own line and the thread structure is
 shown with UTF-8 graphics characters (so it looks vaguely similar to
 mutt's threaded view): see http://kanelephant.com/screen.png for a screenshot.

 Pick was originally written by David Edmondon and posted on irc and I
 have been developing it on and off since.  Now that almost all the
 backend changes it uses are in I would like to see about getting it
 into mainline.

 The code is not of the same standard as mainline code: in particular a
 lot of the code is written by me and is working but unidiomatic
 lisp. It has also not had widespread testing so I would expect it to
 have several bugs.

 We could try getting it into mainline using the normal review type
 approach, but the patch is necessarily large (it implements a new view
 similar in size to show or search) with the main pick.el file being
 nearly 900 lines.

 An alternative approach would be to accept it into contrib and then
 reviewers/users could submit fixes for the problems directly.

 This patch series implements the latter approach, but I am definitely
 happy to try for the former or some other approach.



I have been using this series for a long time and have not found any
issues. How about adding pick as a part of proper emacs file and have a
config value that disables pick by default. ? So only when
notmuch-enable-pick is set we will load notmuch-pick.el ?

 In its current form the user needs to copy (or link) the
 notmuch-pick.el from contrib into the emacs directory and then build
 notmuch as usual. 

With that users won't require to do the above.

There are two very small patches to mainline code:
 one to compile and load the pick file if present and one small tweak
 to notmuch-show.el. Then in contrib/notmuch-pick there are three
 files: the notmuch-pick.el file itself, a README describing
 documenting install and use, and a TODO which contains the main things
 I think need doing (and I will try to update this in light of comments
 received).




-aneesh

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/7] go: Allow notmuch objects to be garbage collected

[Adrien Bustany]

Le 20/07/2012 06:23, Austin Clements a ?crit :
> Quoth Adrien Bustany on Jul 19 at  9:25 pm:
>> Le 18/07/2012 23:40, Austin Clements a ?crit :
>>> This is subtle enough that I think it deserves a comment in the source
>>> code explaining that tracking the talloc owner reference, combined
>>> with the fact that Go finalizers are run in dependency order, ensures
>>> that the C objects will always be destroyed from the talloc leaves up.
>>
>> Definitely, I tend to comment in the commit message and forget about
>> the code...
>>
>>>
>>> Just one inline comment below.  Otherwise, I think this is all
>>> correct.
>>
>> Agree with the comment, the Database should be the parent. I guess I
>> wasn't sure of the talloc parenting.
>>
>>>
>>> Is reproducing the talloc hierarchy in all of the bindings really the
>>> right approach?  I feel like there has to be a better way (or that the
>>> way we use talloc in the library is slightly broken).  What if the
>>> bindings created an additional talloc reference to each managed
>>> object, just to keep the object alive, and used talloc_unlink instead
>>> of the destroy functions?
>>
>> Reproducing the hierarchy is probably error prone, and not that
>> simple indeed :/
>> I haven't checked at all the way you suggest, but if we use
>> talloc_reference/unlink, we get the same issue no?
>> - If we do for each new wrapped object talloc_reference(NULL,
>> wrapped_object), the the object will be kept alive until we
>> talloc_unlink(NULL, wrapped_object), but what about its parents? For
>> example will doing that on a notmuch_message_t keep the
>> notmuch_messages_t alive?
>
> Hmm.  This is what I was thinking.  You have an interesting point; I
> think it's slightly wrong, but it exposes something deeper.  I believe
> there are two different things going on here: some of the talloc
> relationships are for convenience, while some are structural.  In the
> former case, I'm pretty sure my suggestion will work, but in the
> latter case the objects should *never* be freed by the finalizer!
>
> For example, notmuch_query_search_messages returns a new
> notmuch_messages_t with the query as the talloc parent, but that
> notmuch_messages_t doesn't depend on the query object; this is just so
> you can conveniently delete everything retrieved from the query by
> deleting the query.  In this case, you can either use parent
> references like you did---which will prevent a double-free by forcing
> destruction to happen from the leaves up but at the cost of having to
> encode these relationships and of extending the parent object
> lifetimes beyond what's strictly necessary---or you can use my
> suggestion of creating an additional talloc reference.

Actually, checking the code of notmuch_query_search_messages, it seems 
that the notmuch_messages_t (and the notmuch_message_t as well) object 
*does* depend on the database and the query... So in that case I think 
we need the "owner" Object reference as I currently have (we want the 
Messages to keep the Query alive, and the Query keeps the Database alive).
That said, you example below looks valid, and it seems I'll need to add 
a flag to createMessage() (and some others) to disable the SetFinalizer 
call for certain instances (we probably want to keep it for eg. 
SearchMessageByFilename).

- The candidates I found for adding a tmalloc reference and not a "full" 
Go reference (therefore preventing to keep the parent alive too long 
needlessly) are GetAllTags, Thread.GetTags, Messages.CollectTags, and 
Message.GetTags (those are basically string lists)

- The methods for which I should remove the SetFinalizer on the wrapper 
(as you showed in the example below) while keeping the Go reference are 
Threads.Get and Messages.Get

I would also maybe remove all the Destroy() functions, since they now 
seem more dangerous than anything else...

I tried to write a test using runtime.GC to test the behaviour of the 
bindings, but for some reasons some cases which are supposed to crash 
don't, which makes me sceptical about the validity of the test :-/

Cheers

Adrien

>
> However, in your example, the notmuch_message_t's are structurally
> related to the notmuch_messages_t from whence they came.  They're all
> part of one data structure and hence it *never* makes sense for a
> caller to delete the notmuch_message_t's.  For example, even with the
> code in this patch, I think the following could lead to a crash:
>
> 1. Obtain a Messages object, say ms.
> 2. m1 := ms.Get()
> 3. m1 = nil
> 4. m2 := ms.Get()
> 5. m2.whatever()
>
> If a garbage collection happens between steps 3 and 4, the Message in
> m1 will get finalized and destroyed.  But step 4 will return the same,
> now dangling, pointer, leading to a potential crash in step 5.
>
> Maybe the answer in the structural case is to include the parent
> pointer in the Go struct and not set a finalizer on the child?  That
> way, if there's a Go reference to the parent wrapper, it won't go away
> and the children 

[PATCH v8 0/3] notmuch-search: Structured Output Formatters

[David Bremner]

craven at gmx.net writes:

> This patch series amends the situation by introducing structured
> formatters, which allow different implementations of structures like
> lists, maps, strings and numbers.
>

pushed, 

d

[PATCH v4 0/8] emacs: JSON-based search cleanups

[David Bremner]

Austin Clements  writes:

> This version fixes several bugs found in the previous version.  

pushed, 

d

[PATCH] emacs: Fix notmuch-message-mark-replied.

[David Bremner]

Ingo Lohmar  writes:

> notmuch-message-mark-replied used "apply" to change message tags
> according to notmuch-message-replied-tags after sending a reply.  This
> works if the latter is a single-element list.  But with the recently
> changed format of tag changes, it breaks for multiple-element lists.
> Use "funcall" to properly pass the list of tag changes as a single
> argument.

pushed,

d

notmuch-emacs and bbdb

[Daniel Reusche]

On 22.07.2012 02:24, Daniel Bergey wrote:
> Glad to hear I have another user!
Your welcome, nice snippet!

> My function bbdb/notmuch-snarf-to doesn't work yet on more than one
> recipient.
>
> Probably not before late August, though.

Well, if I find some time, I will at least try it too.

best,
dan

[PATCH v2 (rebased) 0/4] Add a --body option to notmuch show

[Mark Walters]

This is a rebased version of the series
id:"1343066259-22523-1-git-send-email-markwalters1009 at gmail.com"

It is otherwise unchanged. (The rebase is a change in the context part
of the test patch).

Best wishes

Mark


Mark Walters (4):
  cli: add --body=true|false option to notmuch-show.c
  test: add tests for the new --body=true|false option
  man: update man page for the new --body=true|false option
  schemata: update for --body=true|false option

 devel/schemata  |2 +-
 man/man1/notmuch-show.1 |   16 
 notmuch-client.h|3 ++-
 notmuch-reply.c |2 +-
 notmuch-show.c  |   30 ++
 test/json   |9 +
 6 files changed, 51 insertions(+), 11 deletions(-)

-- 
1.7.9.1

[PATCH v2 (rebased) 1/4] cli: add --body=true|false option to notmuch-show.c

[Mark Walters]

This option allows the caller to suppress the output of the bodies of
the messages. Currently this is only implemented for format=json.

This is used by notmuch-pick.el (although not needed) because it gives
a speed-up of at least a factor of a two (and in some cases a speed up
of more than a factor of 8); moreover it reduces the memory usage in
emacs hugely.
---
 notmuch-client.h |3 ++-
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   30 ++
 3 files changed, 25 insertions(+), 10 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 0c17b79..f930798 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -87,6 +87,7 @@ typedef struct notmuch_crypto {
 typedef struct notmuch_show_params {
 notmuch_bool_t entire_thread;
 notmuch_bool_t omit_excluded;
+notmuch_bool_t output_body;
 notmuch_bool_t raw;
 int part;
 notmuch_crypto_t crypto;
@@ -176,7 +177,7 @@ notmuch_status_t
 show_one_part (const char *filename, int part);

 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first);
+format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body);

 void
 format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 3a038ed..de21f3b 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -620,7 +620,7 @@ notmuch_reply_format_json(void *ctx,
 /* Start the original */
 printf (", \"original\": ");

-format_part_json (ctx, node, TRUE);
+format_part_json (ctx, node, TRUE, TRUE);

 /* End */
 printf ("}\n");
diff --git a/notmuch-show.c b/notmuch-show.c
index 8f3c60e..d3419e4 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -559,7 +559,7 @@ format_part_text (const void *ctx, mime_node_t *node,
 }

 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first)
+format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body)
 {
 /* Any changes to the JSON format should be reflected in the file
  * devel/schemata. */
@@ -571,10 +571,12 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first)
printf ("\"headers\": ");
format_headers_json (ctx, GMIME_MESSAGE (node->part), FALSE);

-   printf (", \"body\": [");
-   format_part_json (ctx, mime_node_child (node, 0), first);
-
-   printf ("]}");
+   if (output_body) {
+   printf (", \"body\": [");
+   format_part_json (ctx, mime_node_child (node, 0), first, TRUE);
+   printf ("]");
+   }
+   printf ("}");
return;
 }

@@ -652,16 +654,16 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first)
 talloc_free (local);

 for (i = 0; i < node->nchildren; i++)
-   format_part_json (ctx, mime_node_child (node, i), i == 0);
+   format_part_json (ctx, mime_node_child (node, i), i == 0, TRUE);

 printf ("%s}", terminator);
 }

 static notmuch_status_t
 format_part_json_entry (const void *ctx, mime_node_t *node, unused (int 
indent),
-   unused (const notmuch_show_params_t *params))
+   const notmuch_show_params_t *params)
 {
-format_part_json (ctx, node, TRUE);
+format_part_json (ctx, node, TRUE, params->output_body);

 return NOTMUCH_STATUS_SUCCESS;
 }
@@ -1004,6 +1006,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
 notmuch_show_params_t params = {
.part = -1,
.omit_excluded = TRUE,
+   .output_body = TRUE,
.crypto = {
.verify = FALSE,
.decrypt = FALSE
@@ -1032,6 +1035,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
{ NOTMUCH_OPT_INT, , "part", 'p', 0 },
{ NOTMUCH_OPT_BOOLEAN, , "decrypt", 'd', 0 },
{ NOTMUCH_OPT_BOOLEAN, , "verify", 'v', 0 },
+   { NOTMUCH_OPT_BOOLEAN, _body, "body", 'b', 0 },
{ 0, 0, 0, 0, 0 }
 };

@@ -1086,6 +1090,16 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
entire_thread = ENTIRE_THREAD_FALSE;
 }

+if (!params.output_body) {
+   if (params.part > 0) {
+   fprintf (stderr, "Warning: --body=false is incompatible with --part 
> 0. Disabling.\n");
+   params.output_body = TRUE;
+   } else {
+   if (format != _json)
+   fprintf (stderr, "Warning: --body=false only implemented for 
format=json\n");
+   }
+}
+
 if (entire_thread == ENTIRE_THREAD_TRUE)
params.entire_thread = TRUE;
 else
-- 
1.7.9.1

[PATCH v2 (rebased) 3/4] man: update man page for the new --body=true|false option

[Mark Walters]

---
 man/man1/notmuch-show.1 |   16 
 1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
index b51a54c..506583a 100644
--- a/man/man1/notmuch-show.1
+++ b/man/man1/notmuch-show.1
@@ -152,6 +152,22 @@ The default is

 .RE

+.RS 4
+.TP 4
+.B \-\-body=(true|false)
+
+If true (the default)
+.B notmuch show
+includes the bodies of the messages in the output; if false,
+bodies are omitted.
+.B --body=false
+is only implemented for the json format and it is incompatible with
+.B --part > 0.
+
+This is useful if the caller only needs the headers as body-less
+output is much faster and substantially smaller.
+.RE
+
 A common use of
 .B notmuch show
 is to display a single thread of email messages. For this, use a
-- 
1.7.9.1

[PATCH v2 (rebased) 4/4] schemata: update for --body=true|false option

[Mark Walters]

Previously body: was a compulsory field in a message. The new
--body=false option causes notmuch show to omit this field so update
schemata to reflect this.
---
 devel/schemata |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/devel/schemata b/devel/schemata
index 6677a1c..9cb25f5 100644
--- a/devel/schemata
+++ b/devel/schemata
@@ -47,7 +47,7 @@ message = {
 tags:   [string*],

 headers:headers,
-body:   [part]
+body?:  [part]# omitted if --body=false
 }

 # A MIME part (format_part_json)
-- 
1.7.9.1

[PATCH v2 (rebased) 2/4] test: add tests for the new --body=true|false option

[Mark Walters]

---
 test/json |9 +
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/test/json b/test/json
index 337b3f5..831e105 100755
--- a/test/json
+++ b/test/json
@@ -7,6 +7,15 @@ add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 
01 Jan 2000 12:00:0
 output=$(notmuch show --format=json "json-show-message")
 test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, 
\"excluded\": false, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 
946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], 
\"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test 
Suite \", \"To\": \"Notmuch Test Suite 
\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 
+\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": 
\"json-show-message\n\"}]}, ["

+# This should be the same output as above.
+test_begin_subtest "Show message: json --body=true"
+output=$(notmuch show --format=json --body=true "json-show-message")
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, 
\"excluded\": false, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 
946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], 
\"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test 
Suite \", \"To\": \"Notmuch Test Suite 
\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 
+\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": 
\"json-show-message\n\"}]}, ["
+
+test_begin_subtest "Show message: json --body=false"
+output=$(notmuch show --format=json --body=false "json-show-message")
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, 
\"excluded\": false, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 
946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], 
\"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test 
Suite \", \"To\": \"Notmuch Test Suite 
\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 
+\"}}, ["
+
 test_begin_subtest "Search message: json"
 add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 
12:00:00 -\"" "[body]=\"json-search-message\""
 output=$(notmuch search --format=json "json-search-message" | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
-- 
1.7.9.1

[PATCH v2 (rebased) 0/4] Add a --body option to notmuch show

[David Bremner]

Mark Walters  writes:

> This is a rebased version of the series
> id:"1343066259-22523-1-git-send-email-markwalters1009 at gmail.com"
>
> It is otherwise unchanged. (The rebase is a change in the context part
> of the test patch).

pushed, thanks.

d

[PATCH v2] man: show: update man page for entire-thread and json.

[Mark Walters]

Previously in notmuch show --format=json implied --entire-thread. This
is still the default but it is now possible to disable this. Update
the manpage to reflect this.
---

Update the manpage since --format=json and --entire-thread=false is
now supported. Reworded as Austin suggested.

Best wishes

Mark 

 man/man1/notmuch-show.1 |   21 +
 1 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/man/man1/notmuch-show.1 b/man/man1/notmuch-show.1
index 506583a..765b22c 100644
--- a/man/man1/notmuch-show.1
+++ b/man/man1/notmuch-show.1
@@ -24,11 +24,14 @@ Supported options for
 include
 .RS 4
 .TP 4
-.B \-\-entire\-thread
+.B \-\-entire\-thread=(true|false)

-By default only those messages that match the search terms will be
-displayed. With this option, all messages in the same thread as any
-matched message will be displayed.
+If true,
+.B notmuch show
+outputs all messages in the thread of any message matching the search
+terms; if false, it outputs only the matching messages. For
+.B --format=json
+this defaults to true.  For other formats, this defaults to false.
 .RE

 .RS 4
@@ -55,11 +58,13 @@ be nested.
 The output is formatted with Javascript Object Notation (JSON). This
 format is more robust than the text format for automated
 processing. The nested structure of multipart MIME messages is
-reflected in nested JSON output. JSON output always includes all
-messages in a matching thread; in effect
+reflected in nested JSON output. By default JSON output includes all
+messages in a matching thread; that is, by default,
 .B \-\-format=json
-implies
-.B \-\-entire\-thread
+sets
+.B "\-\-entire\-thread"
+The caller can disable this behaviour by setting
+.B \-\-entire\-thread=false

 .RE
 .RS 4
-- 
1.7.9.1

[PATCH 0/5] Notmuch Pick (WIP or contrib)

[Mark Walters]

Hello

Notmuch pick is an emacs view which displays a threaded view of
messages: each message has its own line and the thread structure is
shown with UTF-8 graphics characters (so it looks vaguely similar to
mutt's threaded view): see http://kanelephant.com/screen.png for a screenshot.

Pick was originally written by David Edmondon and posted on irc and I
have been developing it on and off since.  Now that almost all the
backend changes it uses are in I would like to see about getting it
into mainline.

The code is not of the same standard as mainline code: in particular a
lot of the code is written by me and is working but unidiomatic
lisp. It has also not had widespread testing so I would expect it to
have several bugs.

We could try getting it into mainline using the normal review type
approach, but the patch is necessarily large (it implements a new view
similar in size to show or search) with the main pick.el file being
nearly 900 lines.

An alternative approach would be to accept it into contrib and then
reviewers/users could submit fixes for the problems directly.

This patch series implements the latter approach, but I am definitely
happy to try for the former or some other approach.

In its current form the user needs to copy (or link) the
notmuch-pick.el from contrib into the emacs directory and then build
notmuch as usual. There are two very small patches to "mainline" code:
one to compile and load the pick file if present and one small tweak
to notmuch-show.el. Then in contrib/notmuch-pick there are three
files: the notmuch-pick.el file itself, a README describing
documenting install and use, and a TODO which contains the main things
I think need doing (and I will try to update this in light of comments
received).

Any comments on the code or the correct approach, patches etc
gratefully received!

Best wishes

Mark

PS If it is in contrib then I am happy to maintain it (with the aim of
an eventual move to mainline).


Mark Walters (5):
  emacs: compile and load notmuch-pick.el if present.
  emacs: make notmuch-show return its buffer
  contrib: add notmuch-pick.el file itself
  contrib: Added pick README.
  contrib: add pick TODO file

 contrib/notmuch-pick/README  |   36 ++
 contrib/notmuch-pick/TODO|   25 +
 contrib/notmuch-pick/notmuch-pick.el |  876 ++
 emacs/Makefile.local |3 +-
 emacs/notmuch-show.el|3 +-
 emacs/notmuch.el |5 +
 6 files changed, 946 insertions(+), 2 deletions(-)
 create mode 100644 contrib/notmuch-pick/README
 create mode 100644 contrib/notmuch-pick/TODO
 create mode 100644 contrib/notmuch-pick/notmuch-pick.el

-- 
1.7.9.1

[PATCH 2/5] emacs: make notmuch-show return its buffer

[Mark Walters]

notmuch-pick uses the returned buffer to try and make sure it does not
close the wrong buffer.
---
 emacs/notmuch-show.el |3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6335d45..1cabd17 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1031,7 +1031,8 @@ function is used."
  notmuch-show-parent-buffer parent-buffer
  notmuch-show-query-context query-context)
 (notmuch-show-build-buffer)
-(notmuch-show-goto-first-wanted-message)))
+(notmuch-show-goto-first-wanted-message)
+(current-buffer)))

 (defun notmuch-show-build-buffer ()
   (let ((inhibit-read-only t))
-- 
1.7.9.1

[PATCH 1/5] emacs: compile and load notmuch-pick.el if present.

[Mark Walters]

Compile and load notmuch-pick.el if present.

All the actual setup of pick is done in the function notmuch-pick-init
so we call that in the notmuch init function if it is bound. This
function will setup all extra keybinding etc.
---
 emacs/Makefile.local |3 ++-
 emacs/notmuch.el |5 +
 2 files changed, 7 insertions(+), 1 deletions(-)

diff --git a/emacs/Makefile.local b/emacs/Makefile.local
index fb82247..9f4dba6 100644
--- a/emacs/Makefile.local
+++ b/emacs/Makefile.local
@@ -15,7 +15,8 @@ emacs_sources := \
$(dir)/notmuch-crypto.el \
$(dir)/notmuch-tag.el \
$(dir)/coolj.el \
-   $(dir)/notmuch-print.el
+   $(dir)/notmuch-print.el \
+   $(wildcard $(dir)/notmuch-pick.el)

 emacs_images := \
$(srcdir)/$(dir)/notmuch-logo.png
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index fd1836f..4f3da4f 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -59,6 +59,9 @@
 (require 'notmuch-maildir-fcc)
 (require 'notmuch-message)

+;; Load notmuch-pick if available (but do not error if not present).
+(load "notmuch-pick" t)
+
 (defcustom notmuch-search-result-format
   `(("date" . "%12s ")
 ("count" . "%-7s ")
@@ -1088,6 +1091,8 @@ current search results AND that are tagged with the given 
tag."
 (defun notmuch ()
   "Run notmuch and display saved searches, known tags, etc."
   (interactive)
+  (when (fboundp 'notmuch-pick-init)
+(notmuch-pick-init))
   (notmuch-hello))

 (defun notmuch-interesting-buffer (b)
-- 
1.7.9.1

[PATCH 4/5] contrib: Added pick README.

[Mark Walters]

---
 contrib/notmuch-pick/README |   36 
 1 files changed, 36 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/README

diff --git a/contrib/notmuch-pick/README b/contrib/notmuch-pick/README
new file mode 100644
index 000..1b2399f
--- /dev/null
+++ b/contrib/notmuch-pick/README
@@ -0,0 +1,36 @@
+Notmuch Pick
+
+Notmuch pick is an experimental threaded message view for the emacs
+interface. Each message is one line in the results and the thread
+structure is shown using UFT-8 box drawing characters (similar to
+Mutt's threaded view). It comes between search and show in terms of
+amount of output and can be useful for viewing both single theads and
+multiple threads.
+
+Install
+
+To use notmuch pick copy or link the notmuch-pick.el file into the
+emacs directory of the main source and then compile and install
+notmuch as usual.
+
+Using Pick
+
+The main key entries to notmuch pick are
+
+'z' enter a query to view using notmuch pick (works in hello, search,
+show and pick itself).
+'Z' view the current query in pick (works from search and show)
+'M-RET' view the selected thread in pick (works in search mode)
+
+Once in pick mode keybinding are mostly inline with the rest of
+notmuch and are all viewable with '?' as usual.
+
+Customising Pick
+
+Pick has several customisation variables. The most significant is the
+first notmuch-pick-show-out which determines the behaviour when
+selecting a message (with RET) in the pick view. By default pick uses
+a split window showing the single message in the bottom pane. However,
+if this option is set then it views the whole thread in the complete
+window jumping to the selected message in the thread. In either case
+M-RET selects the other option.
-- 
1.7.9.1

[PATCH 5/5] contrib: add pick TODO file

[Mark Walters]

---
 contrib/notmuch-pick/TODO |   25 +
 1 files changed, 25 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/TODO

diff --git a/contrib/notmuch-pick/TODO b/contrib/notmuch-pick/TODO
new file mode 100644
index 000..2d0796e
--- /dev/null
+++ b/contrib/notmuch-pick/TODO
@@ -0,0 +1,25 @@
+TODO for notmuch-pick
+
+(These are the things I can think of: to be added to as problems get
+reported or found!)
+
+Things that need fixing before acceptance to mainline
+
+- Review lisp to make idiomatic.
+- Unify functions with search or show where appropriate.
+- Work out a fall-back if the font does not contain box graphic characters.
+- Add extra functionality?
+
+- Decide whether the asynchronous parser is satisfactory.
+- Remove debugging/timing information.
+
+- Add tests (I have some but I am not sure how to add them if pick is
+  in contrib).
+
+Bugs:
+
+- The display flickers while pick is running. I have no idea why.
+
+Other todo items
+
+?
\ No newline at end of file
-- 
1.7.9.1

[PATCH 3/5] contrib: add notmuch-pick.el file itself

[Mark Walters]

This adds the main notmuch-pick.el file.
---
 contrib/notmuch-pick/notmuch-pick.el |  876 ++
 1 files changed, 876 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-pick/notmuch-pick.el

diff --git a/contrib/notmuch-pick/notmuch-pick.el 
b/contrib/notmuch-pick/notmuch-pick.el
new file mode 100644
index 000..ded0c42
--- /dev/null
+++ b/contrib/notmuch-pick/notmuch-pick.el
@@ -0,0 +1,876 @@
+;; notmuch-pick.el --- displaying notmuch forests.
+;;
+;; Copyright ? Carl Worth
+;; Copyright ? David Edmondson
+;; Copyright ? Mark Walters
+;;
+;; This file is part of Notmuch.
+;;
+;; Notmuch is free software: you can redistribute it and/or modify it
+;; under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+;;
+;; Notmuch is distributed in the hope that it will be useful, but
+;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;; General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with Notmuch.  If not, see .
+;;
+;; Authors: David Edmondson 
+;;  Mark Walters 
+
+(require 'mail-parse)
+
+(require 'notmuch-lib)
+(require 'notmuch-query)
+(require 'notmuch-show)
+(eval-when-compile (require 'cl))
+
+(declare-function notmuch-call-notmuch-process "notmuch" ( args))
+(declare-function notmuch-show "notmuch-show" ( args))
+(declare-function notmuch-tag "notmuch" (query  tags))
+(declare-function notmuch-show-strip-re "notmuch-show" (subject))
+(declare-function notmuch-show-clean-address "notmuch-show" (parsed-address))
+(declare-function notmuch-show-spaces-n "notmuch-show" (n))
+(declare-function notmuch-read-query "notmuch" (prompt))
+(declare-function notmuch-read-tag-changes "notmuch" ( initial-input 
 search-terms))
+(declare-function notmuch-update-tags "notmuch" (current-tags tag-changes))
+(declare-function notmuch-hello-trim "notmuch-hello" (search))
+(declare-function notmuch-search-find-thread-id "notmuch" ())
+(declare-function notmuch-search-find-subject "notmuch" ())
+
+;; the following variable is defined in notmuch.el
+(defvar notmuch-search-query-string)
+
+(defvar notmuch-pick-process-state nil
+  "Parsing state of the search process filter.")
+
+(defgroup notmuch-pick nil
+  "Showing message and thread structure."
+  :group 'notmuch)
+
+;; This is ugly. We can't run setup-show-out until it has been defined
+;; which needs the keymap to be defined. So we defer setting up to
+;; notmuch-pick-init.
+(defcustom notmuch-pick-show-out nil
+  "View selected messages in new window rather than split-pane."
+  :type 'boolean
+  :group 'notmuch-pick
+  :set (lambda (symbol value)
+(set-default symbol value)
+(when (fboundp 'notmuch-pick-setup-show-out)
+  (notmuch-pick-setup-show-out
+
+(defcustom notmuch-pick-result-format
+  `(("date" . "%12s ")
+("authors" . "%21s ")
+("subject" . "%-54s ")
+("tags" . "(%s)"))
+  "Result formatting for Pick. Supported fields are:
+date, authors, subject, tags
+Note subject includes the tree structure graphics.
+For example:
+(setq notmuch-pick-result-format \(\(\"authors\" . \"%-40s\"\)
+ \(\"subject\" . \"%s\"\)\)\)"
+  :type '(alist :key-type (string) :value-type (string))
+  :group 'notmuch-pick)
+
+(defcustom notmuch-pick-asynchronous-parser t
+  "Use the asynchronous parser."
+  :type 'boolean
+  :group 'notmuch-pick)
+
+;; Faces for messages that match the query.
+(defface notmuch-pick-match-date-face
+  '((t :inherit default))
+  "Face used in pick mode for the date in messages matching the query."
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-author-face
+  'class color)
+  (background dark))
+ (:foreground "OliveDrab1"))
+(((class color)
+  (background light))
+ (:foreground "dark blue"))
+(t
+ (:bold t)))
+  "Face used in pick mode for the date in messages matching the query."
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-subject-face
+  '((t :inherit default))
+  "Face used in pick mode for the subject in messages matching the query."
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+(defface notmuch-pick-match-tag-face
+  'class color)
+  (background dark))
+ (:foreground "OliveDrab1"))
+(((class color)
+  (background light))
+ (:foreground "navy blue" :bold t))
+(t
+ (:bold t)))
+  "Face used in pick mode for tags in messages matching the query."
+  :group 'notmuch-pick
+  :group 'notmuch-faces)
+
+;; Faces for messages that do not match the query.
+(defface notmuch-pick-no-match-date-face
+  '((t (:foreground "gray")))
+  "Face used in pick mode for non-matching 

[PATCH 04/13] show: Associate an sprinter with each format

[Austin Clements]

This associates an sprinter constructor with each show format and uses
this to construct the appropriate sprinter.  Currently nothing is done
with this sprinter, but the following patches will weave it through
the layers of notmuch show.
---
 notmuch-client.h |2 ++
 notmuch-show.c   |9 +
 2 files changed, 11 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index f930798..e4172a2 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -66,9 +66,11 @@ typedef GMimeCipherContext notmuch_crypto_context_t;
 #define STRINGIFY_(s) #s

 typedef struct mime_node mime_node_t;
+typedef struct sprinter sprinter_t;
 struct notmuch_show_params;

 typedef struct notmuch_show_format {
+sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
 const char *message_set_start;
 notmuch_status_t (*part) (const void *ctx,
  struct mime_node *node, int indent,
diff --git a/notmuch-show.c b/notmuch-show.c
index d3419e4..d04943f 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -20,12 +20,14 @@

 #include "notmuch-client.h"
 #include "gmime-filter-reply.h"
+#include "sprinter.h"

 static notmuch_status_t
 format_part_text (const void *ctx, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_text = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_text,
 };

@@ -34,6 +36,7 @@ format_part_json_entry (const void *ctx, mime_node_t *node,
int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_json = {
+.new_sprinter = sprinter_json_create,
 .message_set_start = "[",
 .part = format_part_json_entry,
 .message_set_sep = ", ",
@@ -46,6 +49,7 @@ format_part_mbox (const void *ctx, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_mbox = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_mbox,
 };

@@ -55,6 +59,7 @@ format_part_raw (unused (const void *ctx), mime_node_t *node,
 unused (const notmuch_show_params_t *params));

 static const notmuch_show_format_t format_raw = {
+.new_sprinter = sprinter_text_create,
 .part = format_part_raw,
 };

@@ -1003,6 +1008,7 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
 char *query_string;
 int opt_index, ret;
 const notmuch_show_format_t *format = _text;
+sprinter_t *sprinter;
 notmuch_show_params_t params = {
.part = -1,
.omit_excluded = TRUE,
@@ -1130,6 +1136,9 @@ notmuch_show_command (void *ctx, unused (int argc), 
unused (char *argv[]))
return 1;
 }

+/* Create structure printer. */
+sprinter = format->new_sprinter(ctx, stdout);
+
 /* If a single message is requested we do not use search_excludes. */
 if (params.part >= 0)
ret = do_show_single (ctx, query, format, );
-- 
1.7.10

[PATCH 09/13] show: Convert non-envelope format_part_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |   74 +++-
 1 file changed, 46 insertions(+), 28 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 3ff32df..afbd9d0 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -588,7 +588,6 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
return;
 }

-void *local = talloc_new (ctx);
 /* The disposition and content-type metadata are associated with
  * the envelope for message parts */
 GMimeObject *meta = node->envelope_part ?
@@ -597,31 +596,41 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
 const char *cid = g_mime_object_get_content_id (meta);
 const char *filename = GMIME_IS_PART (node->part) ?
g_mime_part_get_filename (GMIME_PART (node->part)) : NULL;
-const char *terminator = "";
+int nclose = 0;
 int i;

-if (!first)
-   printf (", ");
+sp->begin_map (sp);

-printf ("{\"id\": %d", node->part_num);
+sp->map_key (sp, "id");
+sp->integer (sp, node->part_num);

-if (node->decrypt_attempted)
-   printf (", \"encstatus\": [{\"status\": \"%s\"}]",
-   node->decrypt_success ? "good" : "bad");
+if (node->decrypt_attempted) {
+   sp->map_key (sp, "encstatus");
+   sp->begin_list (sp);
+   sp->begin_map (sp);
+   sp->map_key (sp, "status");
+   sp->string (sp, node->decrypt_success ? "good" : "bad");
+   sp->end (sp);
+   sp->end (sp);
+}

 if (node->verify_attempted) {
-   printf (", \"sigstatus\": ");
+   sp->map_key (sp, "sigstatus");
format_part_sigstatus_json (sp, node);
 }

-printf (", \"content-type\": %s",
-   json_quote_str (local, g_mime_content_type_to_string 
(content_type)));
+sp->map_key (sp, "content-type");
+sp->string (sp, g_mime_content_type_to_string (content_type));

-if (cid)
-   printf (", \"content-id\": %s", json_quote_str (local, cid));
+if (cid) {
+   sp->map_key (sp, "content-id");
+   sp->string (sp, cid);
+}

-if (filename)
-   printf (", \"filename\": %s", json_quote_str (local, filename));
+if (filename) {
+   sp->map_key (sp, "filename");
+   sp->string (sp, filename);
+}

 if (GMIME_IS_PART (node->part)) {
/* For non-HTML text parts, we include the content in the
@@ -636,35 +645,44 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
if (g_mime_content_type_is_type (content_type, "text", "html")) {
const char *content_charset = 
g_mime_object_get_content_type_parameter (meta, "charset");

-   if (content_charset != NULL)
-   printf (", \"content-charset\": %s", json_quote_str (local, 
content_charset));
+   if (content_charset != NULL) {
+   sp->map_key (sp, "content-charset");
+   sp->string (sp, content_charset);
+   }
} else if (g_mime_content_type_is_type (content_type, "text", "*")) {
GMimeStream *stream_memory = g_mime_stream_mem_new ();
GByteArray *part_content;
show_text_part_content (node->part, stream_memory, 0);
part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM 
(stream_memory));
-
-   printf (", \"content\": %s", json_quote_chararray (local, (char *) 
part_content->data, part_content->len));
+   sp->map_key (sp, "content");
+   sp->string_len (sp, (char *) part_content->data, part_content->len);
g_object_unref (stream_memory);
}
 } else if (GMIME_IS_MULTIPART (node->part)) {
-   printf (", \"content\": [");
-   terminator = "]";
+   sp->map_key (sp, "content");
+   sp->begin_list (sp);
+   nclose = 1;
 } else if (GMIME_IS_MESSAGE (node->part)) {
-   printf (", \"content\": [{");
-   printf ("\"headers\": ");
+   sp->map_key (sp, "content");
+   sp->begin_list (sp);
+   sp->begin_map (sp);
+
+   sp->map_key (sp, "headers");
format_headers_json (sp, GMIME_MESSAGE (node->part), FALSE);

-   printf (", \"body\": [");
-   terminator = "]}]";
+   sp->map_key (sp, "body");
+   sp->begin_list (sp);
+   nclose = 3;
 }

-talloc_free (local);
-
 for (i = 0; i < node->nchildren; i++)
format_part_json (ctx, sp, mime_node_child (node, i), i == 0, TRUE);

-printf ("%s}", terminator);
+/* Close content structures */
+for (i = 0; i < nclose; i++)
+   sp->end (sp);
+/* Close part map */
+sp->end (sp);
 }

 static notmuch_status_t
-- 
1.7.10

[PATCH 12/13] show: Convert do_show to use sprinter

[Austin Clements]

---
 notmuch-show.c |   11 ++-
 1 file changed, 2 insertions(+), 9 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 5ee2156..a0da48a 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -961,11 +961,9 @@ do_show (void *ctx,
 notmuch_threads_t *threads;
 notmuch_thread_t *thread;
 notmuch_messages_t *messages;
-int first_toplevel = 1;
 notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;

-if (format->message_set_start)
-   fputs (format->message_set_start, stdout);
+sp->begin_list (sp);

 for (threads = notmuch_query_search_threads (query);
 notmuch_threads_valid (threads);
@@ -979,10 +977,6 @@ do_show (void *ctx,
INTERNAL_ERROR ("Thread %s has no toplevel messages.\n",
notmuch_thread_get_thread_id (thread));

-   if (!first_toplevel && format->message_set_sep)
-   fputs (format->message_set_sep, stdout);
-   first_toplevel = 0;
-
status = show_messages (ctx, format, sp, messages, 0, params);
if (status && !res)
res = status;
@@ -991,8 +985,7 @@ do_show (void *ctx,

 }

-if (format->message_set_end)
-   fputs (format->message_set_end, stdout);
+sp->end (sp);

 return res != NOTMUCH_STATUS_SUCCESS;
 }
-- 
1.7.10

[PATCH 07/13] show: Convert format_headers_json to use sprinter

[Austin Clements]

This no longer requires a talloc context (not that it really did
before since it didn't return anything), so we remove its context
argument.
---
 notmuch-client.h |3 ++-
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   58 ++
 3 files changed, 31 insertions(+), 32 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 8eeedf7..21c3231 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -183,7 +183,8 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
  notmuch_bool_t first, notmuch_bool_t output_body);

 void
-format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
+format_headers_json (sprinter_t *sp, GMimeMessage *message,
+notmuch_bool_t reply);

 typedef enum {
 NOTMUCH_SHOW_TEXT_PART_REPLY = 1 << 0,
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 07d4452..fa6665f 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -617,7 +617,7 @@ notmuch_reply_format_json(void *ctx,

 /* The headers of the reply message we've created */
 printf ("{\"reply-headers\": ");
-format_headers_json (ctx, reply, TRUE);
+format_headers_json (sp, reply, TRUE);
 g_object_unref (G_OBJECT (reply));
 reply = NULL;

diff --git a/notmuch-show.c b/notmuch-show.c
index b258f65..9852119 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -199,48 +199,46 @@ _is_from_line (const char *line)
 }

 void
-format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply)
+format_headers_json (sprinter_t *sp, GMimeMessage *message,
+notmuch_bool_t reply)
 {
-void *local = talloc_new (ctx);
 InternetAddressList *recipients;
 const char *recipients_string;

-printf ("{%s: %s",
-   json_quote_str (local, "Subject"),
-   json_quote_str (local, g_mime_message_get_subject (message)));
-printf (", %s: %s",
-   json_quote_str (local, "From"),
-   json_quote_str (local, g_mime_message_get_sender (message)));
+sp->begin_map (sp);
+
+sp->map_key (sp, "Subject");
+sp->string (sp, g_mime_message_get_subject (message));
+
+sp->map_key (sp, "From");
+sp->string (sp, g_mime_message_get_sender (message));
+
 recipients = g_mime_message_get_recipients (message, 
GMIME_RECIPIENT_TYPE_TO);
 recipients_string = internet_address_list_to_string (recipients, 0);
-if (recipients_string)
-   printf (", %s: %s",
-   json_quote_str (local, "To"),
-   json_quote_str (local, recipients_string));
+if (recipients_string) {
+   sp->map_key (sp, "To");
+   sp->string (sp, recipients_string);
+}
+
 recipients = g_mime_message_get_recipients (message, 
GMIME_RECIPIENT_TYPE_CC);
 recipients_string = internet_address_list_to_string (recipients, 0);
-if (recipients_string)
-   printf (", %s: %s",
-   json_quote_str (local, "Cc"),
-   json_quote_str (local, recipients_string));
+if (recipients_string) {
+   sp->map_key (sp, "Cc");
+   sp->string (sp, recipients_string);
+}

 if (reply) {
-   printf (", %s: %s",
-   json_quote_str (local, "In-reply-to"),
-   json_quote_str (local, g_mime_object_get_header (GMIME_OBJECT 
(message), "In-reply-to")));
+   sp->map_key (sp, "In-reply-to");
+   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"In-reply-to"));

-   printf (", %s: %s",
-   json_quote_str (local, "References"),
-   json_quote_str (local, g_mime_object_get_header (GMIME_OBJECT 
(message), "References")));
+   sp->map_key (sp, "References");
+   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"References"));
 } else {
-   printf (", %s: %s",
-   json_quote_str (local, "Date"),
-   json_quote_str (local, g_mime_message_get_date_as_string 
(message)));
+   sp->map_key (sp, "Date");
+   sp->string (sp, g_mime_message_get_date_as_string (message));
 }

-printf ("}");
-
-talloc_free (local);
+sp->end (sp);
 }

 /* Write a MIME text part out to the given stream.
@@ -575,7 +573,7 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
format_message_json (ctx, node->envelope_file);

printf ("\"headers\": ");
-   format_headers_json (ctx, GMIME_MESSAGE (node->part), FALSE);
+   format_headers_json (sp, GMIME_MESSAGE (node->part), FALSE);

if (output_body) {
printf (", \"body\": [");
@@ -651,7 +649,7 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
 } else if (GMIME_IS_MESSAGE (node->part)) {
printf (", \"content\": [{");
printf ("\"headers\": ");
-   format_headers_json (local, GMIME_MESSAGE (node->part), FALSE);
+   format_headers_json (sp, GMIME_MESSAGE (node->part), FALSE);

printf (", 

[PATCH 13/13] show: Remove now unused fields from notmuch_show_format

[Austin Clements]

The message_set_{begin,sep,end} and null_message fields are no longer
used because we now use the structure printer provided by the format.
---
 notmuch-client.h |4 
 notmuch-show.c   |4 
 2 files changed, 8 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 21c3231..de31aa1 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -71,13 +71,9 @@ struct notmuch_show_params;

 typedef struct notmuch_show_format {
 sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
-const char *message_set_start;
 notmuch_status_t (*part) (const void *ctx, sprinter_t *sprinter,
  struct mime_node *node, int indent,
  const struct notmuch_show_params *params);
-const char *message_set_sep;
-const char *message_set_end;
-const char *null_message;
 } notmuch_show_format_t;

 typedef struct notmuch_crypto {
diff --git a/notmuch-show.c b/notmuch-show.c
index a0da48a..f5bc098 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -37,11 +37,7 @@ format_part_json_entry (const void *ctx, sprinter_t *sp, 
mime_node_t *node,

 static const notmuch_show_format_t format_json = {
 .new_sprinter = sprinter_json_create,
-.message_set_start = "[",
 .part = format_part_json_entry,
-.message_set_sep = ", ",
-.message_set_end = "]",
-.null_message = "null"
 };

 static notmuch_status_t
-- 
1.7.10

[PATCH 11/13] show: Convert show_message to use sprinter

[Austin Clements]

Unlike the previous patches, this function is used for all formats.
However, for formats other than the JSON format, the sprinter methods
used by show_message are all no-ops, so this code continues to
function correctly for all of the formats.

Converting show_message eliminates show_null_message in the process,
since this maps directly to an sprinter method.
---
 notmuch-show.c |   31 +--
 1 file changed, 5 insertions(+), 26 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index fa1e6e9..5ee2156 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -842,15 +842,6 @@ format_part_raw (unused (const void *ctx), unused 
(sprinter_t *sp),
 }

 static notmuch_status_t
-show_null_message (const notmuch_show_format_t *format)
-{
-/* Output a null message. Currently empty for all formats except Json */
-if (format->null_message)
-   printf ("%s", format->null_message);
-return NOTMUCH_STATUS_SUCCESS;
-}
-
-static notmuch_status_t
 show_message (void *ctx,
  const notmuch_show_format_t *format,
  sprinter_t *sp,
@@ -884,23 +875,16 @@ show_messages (void *ctx,
 notmuch_message_t *message;
 notmuch_bool_t match;
 notmuch_bool_t excluded;
-int first_set = 1;
 int next_indent;
 notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;

-if (format->message_set_start)
-   fputs (format->message_set_start, stdout);
+sp->begin_list (sp);

 for (;
 notmuch_messages_valid (messages);
 notmuch_messages_move_to_next (messages))
 {
-   if (!first_set && format->message_set_sep)
-   fputs (format->message_set_sep, stdout);
-   first_set = 0;
-
-   if (format->message_set_start)
-   fputs (format->message_set_start, stdout);
+   sp->begin_list (sp);

message = notmuch_messages_get (messages);

@@ -915,12 +899,9 @@ show_messages (void *ctx,
res = status;
next_indent = indent + 1;
} else {
-   status = show_null_message (format);
+   sp->null (sp);
}

-   if (!status && format->message_set_sep)
-   fputs (format->message_set_sep, stdout);
-
status = show_messages (ctx,
format, sp,
notmuch_message_get_replies (message),
@@ -931,12 +912,10 @@ show_messages (void *ctx,

notmuch_message_destroy (message);

-   if (format->message_set_end)
-   fputs (format->message_set_end, stdout);
+   sp->end (sp);
 }

-if (format->message_set_end)
-   fputs (format->message_set_end, stdout);
+sp->end (sp);

 return res;
 }
-- 
1.7.10

[PATCH 01/13] test: Uniformly canonicalize actual and expected JSON

[Austin Clements]

Previously, we used a variety of ad-hoc canonicalizations for JSON
output in the test suite, but were ultimately very sensitive to JSON
irrelevancies such as whitespace.  This introduces a new test
comparison function, test_expect_equal_json, that first pretty-prints
*both* the actual and expected JSON and the compares the result.

The current implementation of this simply uses Python's json.tool to
perform pretty-printing (with a fallback to the identity function if
parsing fails).  However, since the interface it introduces is
semantically high-level, we could swap in other mechanisms in the
future, such as another pretty-printer or something that does not
re-order object keys (if we decide that we care about that).

In general, this patch does not remove the existing ad-hoc
canonicalization because it does no harm.  We do have to remove the
newline-after-comma rule from notmuch_json_show_sanitize and
filter_show_json because it results in invalid JSON that cannot be
pretty-printed.

Most of this patch simply replaces test_expect_equal and
test_expect_equal_file with test_expect_equal_json.  It changes the
expected JSON in a few places where sanitizers had placed newlines
after commas inside strings.
---
 test/crypto|   37 +++--
 test/json  |   14 +++---
 test/maildir-sync  |   11 ---
 test/multipart |   34 +++---
 test/search-output |2 +-
 test/test-lib.sh   |   17 +
 6 files changed, 55 insertions(+), 60 deletions(-)

diff --git a/test/crypto b/test/crypto
index be752b1..5dd14c4 100755
--- a/test/crypto
+++ b/test/crypto
@@ -51,8 +51,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test signed message 001",
  "From": "Notmuch Test Suite ",
  "To": "test_suite at notmuchmail.org",
- "Date": "Sat,
- 01 Jan 2000 12:00:00 +"},
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"status": "good",
  "fingerprint": "'$FINGERPRINT'",
@@ -64,7 +63,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"

@@ -85,8 +84,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test signed message 001",
  "From": "Notmuch Test Suite ",
  "To": "test_suite at notmuchmail.org",
- "Date": "Sat,
- 01 Jan 2000 12:00:00 +"},
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"status": "good",
  "fingerprint": "'$FINGERPRINT'",
@@ -99,7 +97,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"

@@ -119,8 +117,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test signed message 001",
  "From": "Notmuch Test Suite ",
  "To": "test_suite at notmuchmail.org",
- "Date": "Sat,
- 01 Jan 2000 12:00:00 +"},
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "sigstatus": [{"status": "error",
  "keyid": "'$(echo $FINGERPRINT | cut -c 25-)'",
@@ -132,7 +129,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -193,8 +190,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test encrypted message 001",
  "From": "Notmuch Test Suite ",
  "To": "test_suite at notmuchmail.org",
- "Date": "Sat,
- 01 Jan 2000 12:00:00 +"},
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "encstatus": [{"status": "good"}],
  "sigstatus": [],
@@ -210,7 +206,7 @@ expected='[[[{"id": "X",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"

@@ -221,7 +217,7 @@ output=$(notmuch show --format=json --part=4 --decrypt 
subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"

@@ -248,8 +244,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test encrypted message 001",
  "From": "Notmuch Test Suite ",
  "To": "test_suite at notmuchmail.org",
- "Date": "Sat,
- 01 Jan 2000 12:00:00 +"},
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
  "body": [{"id": 1,
  "encstatus": [{"status": "bad"}],
  "content-type": "multipart/encrypted",
@@ -258,7 +253,7 @@ expected='[[[{"id": "X",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  ['
-test_expect_equal \
+test_expect_equal_json \
 "$output" \
 "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -283,8 +278,7 @@ expected='[[[{"id": "X",
  "headers": {"Subject": "test encrypted message 002",
  "From": "Notmuch Test Suite ",
  "To": 

[PATCH 10/13] show: Convert envelope format_part_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |   57 +---
 1 file changed, 34 insertions(+), 23 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index afbd9d0..fa1e6e9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -110,34 +110,44 @@ _get_one_line_summary (const void *ctx, notmuch_message_t 
*message)
 }

 static void
-format_message_json (const void *ctx, notmuch_message_t *message)
+format_message_json (sprinter_t *sp, notmuch_message_t *message)
 {
+void *local = talloc_new (NULL);
 notmuch_tags_t *tags;
-int first = 1;
-void *ctx_quote = talloc_new (ctx);
 time_t date;
 const char *relative_date;

 date = notmuch_message_get_date (message);
-relative_date = notmuch_time_relative_date (ctx, date);
+relative_date = notmuch_time_relative_date (local, date);
+
+sp->map_key (sp, "id");
+sp->string (sp, notmuch_message_get_message_id (message));
+
+sp->map_key (sp, "match");
+sp->boolean (sp, notmuch_message_get_flag (message, 
NOTMUCH_MESSAGE_FLAG_MATCH));
+
+sp->map_key (sp, "excluded");
+sp->boolean (sp, notmuch_message_get_flag (message, 
NOTMUCH_MESSAGE_FLAG_EXCLUDED));
+
+sp->map_key (sp, "filename");
+sp->string (sp, notmuch_message_get_filename (message));
+
+sp->map_key (sp, "timestamp");
+date = notmuch_message_get_date (message);
+sp->integer (sp, date);

-printf ("\"id\": %s, \"match\": %s, \"excluded\": %s, \"filename\": %s, 
\"timestamp\": %ld, \"date_relative\": \"%s\", \"tags\": [",
-   json_quote_str (ctx_quote, notmuch_message_get_message_id 
(message)),
-   notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? 
"true" : "false",
-   notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED) ? 
"true" : "false",
-   json_quote_str (ctx_quote, notmuch_message_get_filename (message)),
-   date, relative_date);
+sp->map_key (sp, "date_relative");
+sp->string (sp, relative_date);

+sp->map_key (sp, "tags");
+sp->begin_list (sp);
 for (tags = notmuch_message_get_tags (message);
 notmuch_tags_valid (tags);
 notmuch_tags_move_to_next (tags))
-{
- printf("%s%s", first ? "" : ",",
-   json_quote_str (ctx_quote, notmuch_tags_get (tags)));
- first = 0;
-}
-printf("], ");
-talloc_free (ctx_quote);
+   sp->string (sp, notmuch_tags_get (tags));
+sp->end (sp);
+
+talloc_free (local);
 }

 /* Extract just the email address from the contents of a From:
@@ -573,18 +583,19 @@ format_part_json (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
  * devel/schemata. */

 if (node->envelope_file) {
-   printf ("{");
-   format_message_json (ctx, node->envelope_file);
+   sp->begin_map (sp);
+   format_message_json (sp, node->envelope_file);

-   printf ("\"headers\": ");
+   sp->map_key (sp, "headers");
format_headers_json (sp, GMIME_MESSAGE (node->part), FALSE);

if (output_body) {
-   printf (", \"body\": [");
+   sp->map_key (sp, "body");
+   sp->begin_list (sp);
format_part_json (ctx, sp, mime_node_child (node, 0), first, TRUE);
-   printf ("]");
+   sp->end (sp);
}
-   printf ("}");
+   sp->end (sp);
return;
 }

-- 
1.7.10

[PATCH 08/13] show: Convert format_part_sigstatus_json to use sprinter

[Austin Clements]

---
 notmuch-show.c |  118 +---
 1 file changed, 61 insertions(+), 57 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index 9852119..3ff32df 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -337,134 +337,138 @@ signer_status_to_string (GMimeSignerStatus x)

 #ifdef GMIME_ATLEAST_26
 static void
-format_part_sigstatus_json (mime_node_t *node)
+format_part_sigstatus_json (sprinter_t *sp, mime_node_t *node)
 {
 GMimeSignatureList *siglist = node->sig_list;

-printf ("[");
+sp->begin_list (sp);

 if (!siglist) {
-   printf ("]");
+   sp->end (sp);
return;
 }

-void *ctx_quote = talloc_new (NULL);
 int i;
 for (i = 0; i < g_mime_signature_list_length (siglist); i++) {
GMimeSignature *signature = g_mime_signature_list_get_signature 
(siglist, i);

-   if (i > 0)
-   printf (", ");
-
-   printf ("{");
+   sp->begin_map (sp);

/* status */
GMimeSignatureStatus status = g_mime_signature_get_status (signature);
-   printf ("\"status\": %s",
-   json_quote_str (ctx_quote,
-   signature_status_to_string (status)));
+   sp->map_key (sp, "status");
+   sp->string (sp, signature_status_to_string (status));

GMimeCertificate *certificate = g_mime_signature_get_certificate 
(signature);
if (status == GMIME_SIGNATURE_STATUS_GOOD) {
-   if (certificate)
-   printf (", \"fingerprint\": %s", json_quote_str (ctx_quote, 
g_mime_certificate_get_fingerprint (certificate)));
+   if (certificate) {
+   sp->map_key (sp, "fingerprint");
+   sp->string (sp, g_mime_certificate_get_fingerprint 
(certificate));
+   }
/* these dates are seconds since the epoch; should we
 * provide a more human-readable format string? */
time_t created = g_mime_signature_get_created (signature);
-   if (created != -1)
-   printf (", \"created\": %d", (int) created);
+   if (created != -1) {
+   sp->map_key (sp, "created");
+   sp->integer (sp, created);
+   }
time_t expires = g_mime_signature_get_expires (signature);
-   if (expires > 0)
-   printf (", \"expires\": %d", (int) expires);
+   if (expires > 0) {
+   sp->map_key (sp, "expires");
+   sp->integer (sp, expires);
+   }
/* output user id only if validity is FULL or ULTIMATE. */
/* note that gmime is using the term "trust" here, which
 * is WRONG.  It's actually user id "validity". */
if (certificate) {
const char *name = g_mime_certificate_get_name (certificate);
GMimeCertificateTrust trust = g_mime_certificate_get_trust 
(certificate);
-   if (name && (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE))
-   printf (", \"userid\": %s", json_quote_str (ctx_quote, 
name));
+   if (name && (trust == GMIME_CERTIFICATE_TRUST_FULLY || trust == 
GMIME_CERTIFICATE_TRUST_ULTIMATE)) {
+   sp->map_key (sp, "userid");
+   sp->string (sp, name);
+   }
}
} else if (certificate) {
const char *key_id = g_mime_certificate_get_key_id (certificate);
-   if (key_id)
-   printf (", \"keyid\": %s", json_quote_str (ctx_quote, key_id));
+   if (key_id) {
+   sp->map_key (sp, "keyid");
+   sp->string (sp, key_id);
+   }
}

GMimeSignatureError errors = g_mime_signature_get_errors (signature);
if (errors != GMIME_SIGNATURE_ERROR_NONE) {
-   printf (", \"errors\": %d", errors);
+   sp->map_key (sp, "errors");
+   sp->integer (sp, errors);
}

-   printf ("}");
+   sp->end (sp);
  }

-printf ("]");
-
-talloc_free (ctx_quote);
+sp->end (sp);
 }
 #else
 static void
-format_part_sigstatus_json (mime_node_t *node)
+format_part_sigstatus_json (sprinter_t *sp, mime_node_t *node)
 {
 const GMimeSignatureValidity* validity = node->sig_validity;

-printf ("[");
+sp->begin_list (sp);

 if (!validity) {
-   printf ("]");
+   sp->end (sp);
return;
 }

 const GMimeSigner *signer = g_mime_signature_validity_get_signers 
(validity);
-int first = 1;
-void *ctx_quote = talloc_new (NULL);
-
 while (signer) {
-   if (first)
-   first = 0;
-   else
-   printf (", ");
-
-   printf ("{");
+   sp->begin_map (sp);

/* status */
-   printf ("\"status\": %s",
-   json_quote_str (ctx_quote,
-   signer_status_to_string (signer->status)));
+   sp->map_key (sp, "status");
+   sp->string (sp, 

[PATCH 06/13] show: Feed the sprinter down to part formatters

[Austin Clements]

There are several levels of function calls between where we create the
sprinter and the call to the part formatter in show_message. This
feeds the sprinter through all of them and into the part formatters.
---
 notmuch-client.h |5 +++--
 notmuch-reply.c  |2 +-
 notmuch-show.c   |   50 +-
 3 files changed, 33 insertions(+), 24 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index e4172a2..8eeedf7 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -72,7 +72,7 @@ struct notmuch_show_params;
 typedef struct notmuch_show_format {
 sprinter_t *(*new_sprinter) (const void *ctx, FILE *stream);
 const char *message_set_start;
-notmuch_status_t (*part) (const void *ctx,
+notmuch_status_t (*part) (const void *ctx, sprinter_t *sprinter,
  struct mime_node *node, int indent,
  const struct notmuch_show_params *params);
 const char *message_set_sep;
@@ -179,7 +179,8 @@ notmuch_status_t
 show_one_part (const char *filename, int part);

 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body);
+format_part_json (const void *ctx, sprinter_t *sp, mime_node_t *node,
+ notmuch_bool_t first, notmuch_bool_t output_body);

 void
 format_headers_json (const void *ctx, GMimeMessage *message, notmuch_bool_t 
reply);
diff --git a/notmuch-reply.c b/notmuch-reply.c
index e42ba79..07d4452 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -624,7 +624,7 @@ notmuch_reply_format_json(void *ctx,
 /* Start the original */
 printf (", \"original\": ");

-format_part_json (ctx, node, TRUE, TRUE);
+format_part_json (ctx, sp, node, TRUE, TRUE);

 /* End */
 printf ("}\n");
diff --git a/notmuch-show.c b/notmuch-show.c
index d04943f..b258f65 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -23,7 +23,7 @@
 #include "sprinter.h"

 static notmuch_status_t
-format_part_text (const void *ctx, mime_node_t *node,
+format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_text = {
@@ -32,7 +32,7 @@ static const notmuch_show_format_t format_text = {
 };

 static notmuch_status_t
-format_part_json_entry (const void *ctx, mime_node_t *node,
+format_part_json_entry (const void *ctx, sprinter_t *sp, mime_node_t *node,
int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_json = {
@@ -45,7 +45,7 @@ static const notmuch_show_format_t format_json = {
 };

 static notmuch_status_t
-format_part_mbox (const void *ctx, mime_node_t *node,
+format_part_mbox (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params);

 static const notmuch_show_format_t format_mbox = {
@@ -54,7 +54,7 @@ static const notmuch_show_format_t format_mbox = {
 };

 static notmuch_status_t
-format_part_raw (unused (const void *ctx), mime_node_t *node,
+format_part_raw (unused (const void *ctx), sprinter_t *sp, mime_node_t *node,
 unused (int indent),
 unused (const notmuch_show_params_t *params));

@@ -471,7 +471,7 @@ format_part_sigstatus_json (mime_node_t *node)
 #endif

 static notmuch_status_t
-format_part_text (const void *ctx, mime_node_t *node,
+format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node,
  int indent, const notmuch_show_params_t *params)
 {
 /* The disposition and content-type metadata are associated with
@@ -553,7 +553,7 @@ format_part_text (const void *ctx, mime_node_t *node,
 }

 for (i = 0; i < node->nchildren; i++)
-   format_part_text (ctx, mime_node_child (node, i), indent, params);
+   format_part_text (ctx, sp, mime_node_child (node, i), indent, params);

 if (GMIME_IS_MESSAGE (node->part))
printf ("\fbody}\n");
@@ -564,7 +564,8 @@ format_part_text (const void *ctx, mime_node_t *node,
 }

 void
-format_part_json (const void *ctx, mime_node_t *node, notmuch_bool_t first, 
notmuch_bool_t output_body)
+format_part_json (const void *ctx, sprinter_t *sp, mime_node_t *node,
+ notmuch_bool_t first, notmuch_bool_t output_body)
 {
 /* Any changes to the JSON format should be reflected in the file
  * devel/schemata. */
@@ -578,7 +579,7 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first, notm

if (output_body) {
printf (", \"body\": [");
-   format_part_json (ctx, mime_node_child (node, 0), first, TRUE);
+   format_part_json (ctx, sp, mime_node_child (node, 0), first, TRUE);
printf ("]");
}
printf ("}");
@@ -659,16 +660,17 @@ format_part_json (const void *ctx, mime_node_t *node, 
notmuch_bool_t first, notm
 talloc_free (local);

 for (i = 0; i < node->nchildren; i++)

[PATCH 00/13] Convert notmuch show to use structure printers

[Austin Clements]

This patch series converts notmuch show to use the new structure
printer infrastructure, cleaning up the code and paving the way to
easily support other structure types.  There are a lot of patches, but
most of them are small and the conversion is very mechanical.

The first three patches set up prerequisites by making the test
infrastructure more resilient to irrelevant JSON changes and by
introducing a new method to the sprinter.  The next three patches
introduce sprinters to notmuch show and feed them through all of its
layers, though they remain unused.  The remaining patches work their
way back up from the leaves of the JSON printer all the way to
do_show, converting each layer to use the sprinter.  By doing this
conversion from the leaves of the structure up, the output remains
valid and the tests continue to pass at every step.

The full series has been tested with both GMime 2.4 and GMime 2.6.

The first two patches together and the third patch alone stand on
their own and could be pushed before the rest of the series if
desired.

[PATCH 05/13] reply: Create a JSON sprinter

[Austin Clements]

---
 notmuch-reply.c |4 
 1 file changed, 4 insertions(+)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index de21f3b..e42ba79 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -22,6 +22,7 @@

 #include "notmuch-client.h"
 #include "gmime-filter-headers.h"
+#include "sprinter.h"

 static void
 show_reply_headers (GMimeMessage *message)
@@ -596,6 +597,7 @@ notmuch_reply_format_json(void *ctx,
 notmuch_messages_t *messages;
 notmuch_message_t *message;
 mime_node_t *node;
+sprinter_t *sp;

 if (notmuch_query_count_messages (query) != 1) {
fprintf (stderr, "Error: search term did not match precisely one 
message.\n");
@@ -611,6 +613,8 @@ notmuch_reply_format_json(void *ctx,
 if (!reply)
return 1;

+sp = sprinter_json_create (ctx, stdout);
+
 /* The headers of the reply message we've created */
 printf ("{\"reply-headers\": ");
 format_headers_json (ctx, reply, TRUE);
-- 
1.7.10

[PATCH 03/13] sprinter: Add a string_len method

[Austin Clements]

This method allows callers to output strings with specific lengths.
It's useful both for strings with embedded NULs (which JSON can
represent, though parser support is apparently spotty), and
non-terminated strings.
---
 sprinter-json.c |   11 +--
 sprinter-text.c |   11 +--
 sprinter.h  |1 +
 3 files changed, 19 insertions(+), 4 deletions(-)

diff --git a/sprinter-json.c b/sprinter-json.c
index 4649655..2587ca6 100644
--- a/sprinter-json.c
+++ b/sprinter-json.c
@@ -89,7 +89,7 @@ json_end (struct sprinter *sp)
 }

 static void
-json_string (struct sprinter *sp, const char *val)
+json_string_len (struct sprinter *sp, const char *val, size_t len)
 {
 static const char *const escapes[] = {
['\"'] = "\\\"", ['\\'] = "", ['\b'] = "\\b",
@@ -98,7 +98,7 @@ json_string (struct sprinter *sp, const char *val)
 struct sprinter_json *spj = json_begin_value (sp);

 fputc ('"', spj->stream);
-for (; *val; ++val) {
+for (; len; ++val, --len) {
unsigned char ch = *val;
if (ch < ARRAY_SIZE (escapes) && escapes[ch])
fputs (escapes[ch], spj->stream);
@@ -111,6 +111,12 @@ json_string (struct sprinter *sp, const char *val)
 }

 static void
+json_string (struct sprinter *sp, const char *val)
+{
+json_string_len (sp, val, strlen (val));
+}
+
+static void
 json_integer (struct sprinter *sp, int val)
 {
 struct sprinter_json *spj = json_begin_value (sp);
@@ -166,6 +172,7 @@ sprinter_json_create (const void *ctx, FILE *stream)
.begin_list = json_begin_list,
.end = json_end,
.string = json_string,
+   .string_len = json_string_len,
.integer = json_integer,
.boolean = json_boolean,
.null = json_null,
diff --git a/sprinter-text.c b/sprinter-text.c
index b208840..dfa54b5 100644
--- a/sprinter-text.c
+++ b/sprinter-text.c
@@ -25,14 +25,20 @@ struct sprinter_text {
 };

 static void
-text_string (struct sprinter *sp, const char *val)
+text_string_len (struct sprinter *sp, const char *val, size_t len)
 {
 struct sprinter_text *sptxt = (struct sprinter_text *) sp;

 if (sptxt->current_prefix != NULL)
fprintf (sptxt->stream, "%s:", sptxt->current_prefix);

-fputs(val, sptxt->stream);
+fwrite (val, len, 1, sptxt->stream);
+}
+
+static void
+text_string (struct sprinter *sp, const char *val)
+{
+text_string_len (sp, val, strlen (val));
 }

 static void
@@ -105,6 +111,7 @@ sprinter_text_create (const void *ctx, FILE *stream)
.begin_list = text_begin_list,
.end = text_end,
.string = text_string,
+   .string_len = text_string_len,
.integer = text_integer,
.boolean = text_boolean,
.null = text_null,
diff --git a/sprinter.h b/sprinter.h
index 6680d41..826a852 100644
--- a/sprinter.h
+++ b/sprinter.h
@@ -28,6 +28,7 @@ typedef struct sprinter {
  * For string, the char * must be UTF-8 encoded.
  */
 void (*string) (struct sprinter *, const char *);
+void (*string_len) (struct sprinter *, const char *, size_t);
 void (*integer) (struct sprinter *, int);
 void (*boolean) (struct sprinter *, notmuch_bool_t);
 void (*null) (struct sprinter *);
-- 
1.7.10

[PATCH 02/13] test: Remove unnecessary JSON canonicalization

[Austin Clements]

Format canonicalization of JSON output is no longer necessary, so
remove it.  Value canonicalization (e.g., normalizing thread IDs) is
still necessary, so all of the sanitization functions remain.
---
 test/json |4 ++--
 test/maildir-sync |1 -
 test/multipart|   40 ++--
 3 files changed, 12 insertions(+), 33 deletions(-)

diff --git a/test/json b/test/json
index d86ee46..ac8fa8e 100755
--- a/test/json
+++ b/test/json
@@ -18,7 +18,7 @@ test_expect_equal_json "$output" "[[[{\"id\": 
\"${gen_msg_id}\", \"match\": true

 test_begin_subtest "Search message: json"
 add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 
12:00:00 -\"" "[body]=\"json-search-message\""
-output=$(notmuch search --format=json "json-search-message" | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
+output=$(notmuch search --format=json "json-search-message" | 
notmuch_search_sanitize)
 test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
  \"timestamp\": 946728000,
  \"date_relative\": \"2000-01-01\",
@@ -49,7 +49,7 @@ test_expect_equal_json "$output" "[[[{\"id\": \"$id\", 
\"match\": true, \"exclud

 test_begin_subtest "Search message: json, utf-8"
 add_message "[subject]=\"json-search-utf8-body-s?bj?ct\"" "[date]=\"Sat, 01 
Jan 2000 12:00:00 -\"" "[body]=\"js?n-search-m?ssage\""
-output=$(notmuch search --format=json "js?n-search-m?ssage" | 
notmuch_json_show_sanitize | notmuch_search_sanitize)
+output=$(notmuch search --format=json "js?n-search-m?ssage" | 
notmuch_search_sanitize)
 test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
  \"timestamp\": 946728000,
  \"date_relative\": \"2000-01-01\",
diff --git a/test/maildir-sync b/test/maildir-sync
index b748d04..cd7d241 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -7,7 +7,6 @@ test_description="maildir synchronization"
 # Avoid including the local value of MAIL_DIR in the result.
 filter_show_json() {
 sed -e "s|${MAIL_DIR}/|MAIL_DIR/|"
-echo
 }

 # Create the expected maildir structure
diff --git a/test/multipart b/test/multipart
index 3ccf27f..0527f84 100755
--- a/test/multipart
+++ b/test/multipart
@@ -319,10 +319,8 @@ test_expect_success \
 "notmuch show --format=text --part=8 'id:87liy5ap00.fsf at 
yoom.home.cworth.org'"

 test_begin_subtest "--format=json --part=0, full message"
-notmuch show --format=json --part=0 'id:87liy5ap00.fsf at 
yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
-echo >>OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=0 'id:87liy5ap00.fsf at 
yoom.home.cworth.org' >OUTPUT
 cat OUTPUT
-echo >>OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=1 'id:87liy5ap00.fsf at 
yoom.home.cworth.org' >OUTPUT
 cat OUTPUT
-echo >>OUTPUT # expect *no* newline at end of output
+notmuch show --format=json --part=2 'id:87liy5ap00.fsf at 
yoom.home.cworth.org' >OUTPUT
 cat OUTPUT
-echo >>OUTPUT # expect *no* newline at end of output
+notmuch show --format=json