[PATCH] notmuch new: add a --in-directory option

[Adam Wolfe Gordon]

On Thu, Mar 6, 2014 at 2:06 PM, Austin Clements  wrote:
> That aside, I'm curious what the use case for this is.

My usecase for this is the same as for the similar patch I worked on
previously (id:1373762746-22308-1-git-send-email-awg+notmuch at xvx.ca):

I use inotify to watch for new messages in my maildir. When I receive
a new message, I run notmuch new (after waiting a couple of seconds
since messages sometimes arrive in batches). But since I already know
exactly what changed, there's no need for notmuch new to scan the
whole directory tree. I'm not sure whether this patch lets me specify
a file, or only a directory: the former would be preferable for my
usecase, but either way it's probably an enhancement for me.

-- Adam

Re: [PATCH] notmuch new: add a --in-directory option

[Adam Wolfe Gordon]

On Thu, Mar 6, 2014 at 2:06 PM, Austin Clements  wrote:
> That aside, I'm curious what the use case for this is.

My usecase for this is the same as for the similar patch I worked on
previously (id:1373762746-22308-1-git-send-email-awg+notm...@xvx.ca):

I use inotify to watch for new messages in my maildir. When I receive
a new message, I run notmuch new (after waiting a couple of seconds
since messages sometimes arrive in batches). But since I already know
exactly what changed, there's no need for notmuch new to scan the
whole directory tree. I'm not sure whether this patch lets me specify
a file, or only a directory: the former would be preferable for my
usecase, but either way it's probably an enhancement for me.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Possible addtions to notmuch new ?

[Adam Wolfe Gordon]

On Mon, Aug 12, 2013 at 3:34 AM, Vladimir Marek
 wrote:
> [dir] would limit the scope of 'notmuch' new search to dir and it's
> subdirectories. Alternatively we could have set of include or exclude
> rules similarly to rsync (for example), but [dir] is easier to
> implement.

I sent a series recently that added a command to essentially run
'notmuch new' on a single message file. One reviewer suggested that I
make it an option to notmuch new rather than its own command, which I
think is is a good idea.

When I have some time to get back to that patch series, I'll make the
change as suggested. I think it makes sense to let it take either a
subdirectory or a single file, which would make it work for this part
of your usecase.

-- Adam

Links in email messages

[Adam Wolfe Gordon]

Hi Bart,

On Sun, Jul 14, 2013 at 11:34 PM, Bart Bunting  wrote:
> I am having trouble activating links in emails.  I guess what I
> intuitively expect to happen is that if i hit enter on a link that it
> opens up using browse-url-at-point or similar.
>
> All that appears to happen is that the message I'm viewing collapses.
>
> I would also if possible like urls to be active in text messages as
> well.
>
> Is there an easy solution to this that I'm missing?

First off, if anyone would like to implement this feature, I would
definitely appreciate it. I don't have a great solution, but there are
two workarounds I've used for this:

1. I used to use a terminal that automatically made links clickable
(with a modifier key). This worked well until I got tired of other
bugs in that terminal. (Note that this only applies if, like me, you
run emacs -nw).

2. These days I add a key to the notmuch-show keymap mapped to
browse-url-at-point, with the following:

(define-key notmuch-show-mode-map "U" 'browse-url-at-point)

So when there's a URL I want to see, I go to it and hit U. It's not as
convenient/obvious as enter, but it works well enough. I assume this
works in non-terminal emacs as well.

-- Adam

[PATCH 3/3] test: Add simple tests for the add command

[Adam Wolfe Gordon]

---
 test/insert |   35 +++
 1 file changed, 35 insertions(+)

diff --git a/test/insert b/test/insert
index 021edb6..eb23e02 100755
--- a/test/insert
+++ b/test/insert
@@ -18,6 +18,14 @@ gen_insert_msg() {
"[body]=\"insert-message\""
 }

+gen_add_msg() {
+generate_message \
+"[dir]=add-messages/cur" \
+"[subject]=\"add-subject\"" \
+"[body]=\"add-message\"" \
+"[id]=\"add-message\""
+}
+
 test_expect_code 1 "Insert zero-length file" \
 "notmuch insert < /dev/null"

@@ -118,4 +126,31 @@ gen_insert_msg
 test_expect_code 1 "Insert message, create invalid subfolder" \
 "notmuch insert --folder=../G --create-folder $gen_msg_filename"

+gen_add_msg
+tmp_filename=/tmp/$(basename $gen_msg_filename)
+mv $gen_msg_filename $tmp_filename
+notmuch add $tmp_filename
+test_expect_code 1 "Add message from outside maildir returns 1" \
+"notmuch add $tmp_filename"
+
+test_begin_subtest "Add message from outside maildir does nothing"
+output=$(notmuch count id:$gen_msg_id)
+test_expect_equal "$output" "0"
+
+test_begin_subtest "Add message from inside maildir"
+gen_add_msg
+notmuch add $gen_msg_filename
+output=$(notmuch count id:$gen_msg_id)
+test_expect_equal "$output" "1"
+
+test_begin_subtest "Add duplicate message"
+gen_add_msg
+notmuch add "$gen_msg_filename"
+output=$(notmuch search --output=files "id:$gen_msg_id" | wc -l)
+test_expect_equal "$output" 2
+
+test_begin_subtest "Adding duplicate message does not change tags"
+output=$(notmuch search --format=json --output=tags "id:$gen_msg_id")
+test_expect_equal_json "$output" '["inbox", "unread"]'
+
 test_done
-- 
1.7.9.5

[PATCH 2/3] cli: Introduce the add command

[Adam Wolfe Gordon]

Introduce a new add command, which works the same as insert, except
that it is for adding files that already exist in the maildir. This
provides an alternative to notmuch new for situations where the user
knows what has changed. For example, if the user uses inotify to check
for new files in the maildir, the new files can be indexed using
notmuch add, avoiding the need to scan the whole maildir.
---
 notmuch-client.h |3 +++
 notmuch-insert.c |   49 +
 notmuch.c|2 ++
 3 files changed, 54 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index da332f3..09d7e5c 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -183,6 +183,9 @@ int
 notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]);

 int
+notmuch_add_command (notmuch_config_t *config, int argc, char *argv[]);
+
+int
 notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);

 int
diff --git a/notmuch-insert.c b/notmuch-insert.c
index d32a535..c141450 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -479,3 +479,52 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])

 return (ret) ? 0 : 1;
 }
+
+int
+notmuch_add_command (notmuch_config_t *config, int argc, char *argv[])
+{
+notmuch_database_t *notmuch;
+const char *db_path;
+const char **new_tags;
+size_t new_tags_length;
+tag_op_list_t *tag_ops;
+const char *filename = NULL;
+unsigned int i;
+notmuch_bool_t ret;
+
+if (argc != 2) {
+   fprintf (stderr, "Error: no filename given.\n");
+   return 1;
+}
+filename = argv[1];
+
+db_path = notmuch_config_get_database_path (config);
+new_tags = notmuch_config_get_new_tags (config, &new_tags_length);
+
+tag_ops = tag_op_list_create (config);
+if (tag_ops == NULL) {
+   fprintf (stderr, "Out of memory.\n");
+   return 1;
+}
+for (i = 0; i < new_tags_length; i++) {
+   if (tag_op_list_append (tag_ops, new_tags[i], FALSE))
+   return 1;
+}
+
+/* Check that the message given on the commandline is in the maildir. */
+if (strncmp (filename, db_path, strlen(db_path)) != 0) {
+   fprintf(stderr, "Error: file %s is not in maildir %s.\n", filename,
+   db_path);
+   return 1;
+}
+
+if (notmuch_database_open (notmuch_config_get_database_path (config),
+  NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
+   return 1;
+
+ret = add_file_to_database (notmuch, filename, tag_ops);
+
+notmuch_database_destroy (notmuch);
+
+return (ret) ? 0 : 1;
+}
diff --git a/notmuch.c b/notmuch.c
index 78d29a8..b61149d 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -46,6 +46,8 @@ static command_t commands[] = {
   "Find and import new messages to the notmuch database." },
 { "insert", notmuch_insert_command, FALSE,
   "Add a new message into the maildir and notmuch database." },
+{ "add", notmuch_add_command, FALSE,
+  "Add a message from the maildir into the notmuch database." },
 { "search", notmuch_search_command, FALSE,
   "Search for messages matching the given search terms." },
 { "show", notmuch_show_command, FALSE,
-- 
1.7.9.5

[PATCH 1/3] cli: Return an error code from add_message_to_database

[Adam Wolfe Gordon]

The error code isn't necessary when we're inserting a new message from
standard input, but it will be useful for an upcoming command that
allows an existing file to be indexed.
---
 notmuch-insert.c |8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 2207b1e..d32a535 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -293,7 +293,7 @@ copy_stdin (int fdin, int fdout)

 /* Add the specified message file to the notmuch database, applying tags.
  * The file is renamed to encode notmuch tags as maildir flags. */
-static void
+static int
 add_file_to_database (notmuch_database_t *notmuch, const char *path,
  tag_op_list_t *tag_ops)
 {
@@ -318,7 +318,7 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 case NOTMUCH_STATUS_LAST_STATUS:
fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
 path, notmuch_status_to_string (status));
-   return;
+   return status;
 }

 if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
@@ -331,6 +331,8 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 }

 notmuch_message_destroy (message);
+
+return status;
 }

 static notmuch_bool_t
@@ -377,7 +379,7 @@ insert_message (void *ctx, notmuch_database_t *notmuch, int 
fdin,

 /* Even if adding the message to the notmuch database fails,
  * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops);
+(void) add_file_to_database (notmuch, newpath, tag_ops);

 return TRUE;

-- 
1.7.9.5

[PATCH 0/3] Introduce the add command

[Adam Wolfe Gordon]

Hi everyone,

The recent introduction of insert inspired me to finally add a feature I've
been wanting: a command to index a specific file in the maildir. My usecase
for this is that I have an inotify-based script that watches for new mail
and calls notmuch new when new mail shows up. Running notmuch new in this
situation is overkill, since I know exactly what's changed. A faster command
that just adds a single file reduces contention on the database lock.

This series introduces a new command, "notmuch add", which indexes a file
that already exists in the maildir. It is implemented in notmuch-insert.c
because it uses the basic infrastructure introduced for the insert command.

Missing man page for now - wanted to get the code out first for review.

-- Adam

Adam Wolfe Gordon (3):
  cli: Return an error code from add_message_to_database
  cli: Introduce the add command
  test: Add simple tests for the add command

 notmuch-client.h |3 +++
 notmuch-insert.c |   57 +++---
 notmuch.c|2 ++
 test/insert  |   35 +
 4 files changed, 94 insertions(+), 3 deletions(-)

-- 
1.7.9.5

How to find mails which are sent to 'undisclosed-recipients' ?

[Adam Wolfe Gordon]

Hi Vladimir,

On Thu, Jul 11, 2013 at 9:24 AM, Vladimir Marek
 wrote:
> Since my mail works in a way that it hides everything but what I
> selected to be shown to me, I was overlooking all mails which where sent
> to undisclosed-recipients. I tried to match such mails by
> to:undisclosed-recipients but that does not seem to work. Is there any
> workaround to find such mails?

I have some mail with this To header:

To: Undisclosed recipients 

I can find them with the search to:"Undisclosed recipients". Note the quotes.

-- Adam

[PATCH v5 0/6] emacs: show: lazy handling of hidden parts

[Adam Wolfe Gordon]

On Sun, Jun 9, 2013 at 10:57 PM, Mark Walters  
wrote:
> This is version 5 of this patch set. Version 4 is at
> id:1370074547-24677-1-git-send-email-markwalters1009 at gmail.com.

I've been using version 4 for a few days, and works great. The version
of the code I reviewed LGTM.

Since this series provides a big performance improvement for my usage,
I'd like to see it pushed soon :-).

-- Adam

[PATCH v2 0/3] emacs: show: lazy handling of hidden parts

[Adam Wolfe Gordon]

On Sun, May 26, 2013 at 1:57 AM, Mark Walters  
wrote:
> This is a slightly tweaked version of
> id:1367672478-12247-1-git-send-email-markwalters1009 at gmail.com minus
> the first two patches which have already been pushed.

>From a quick read-through, this series looks good to me. I applied the
patches to my build today and they improve the performance of show
substantially.

Re: [PATCH v2 0/3] emacs: show: lazy handling of hidden parts

[Adam Wolfe Gordon]

On Sun, May 26, 2013 at 1:57 AM, Mark Walters  wrote:
> This is a slightly tweaked version of
> id:1367672478-12247-1-git-send-email-markwalters1...@gmail.com minus
> the first two patches which have already been pushed.

>From a quick read-through, this series looks good to me. I applied the
patches to my build today and they improve the performance of show
substantially.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] lib: Fix name reordering to handle commas without spaces

[Adam Wolfe Gordon]

Notmuch automatically re-orders names of the format "Last, First" to
"First Last" when the associated email address is
First.Last at example.com. But, if a name is of the format "Last,First"
then notmuch will format the name as "irst Last". Handle any number of
spaces after the comma, including none.
---

Good idea, Tomi. This version handles any number of spaces.

lib/thread.cc |   12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index e976d64..ce6b0ef 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -186,8 +186,16 @@ _thread_cleanup_author (notmuch_thread_t *thread,
 if (comma && strlen(comma) > 1) {
/* let's assemble what we think is the correct name */
lname = comma - author;
-   fname = strlen(author) - lname - 2;
-   strncpy(clean_author, comma + 2, fname);
+
+   /* Skip all the spaces after the comma */
+   fname = strlen(author) - lname - 1;
+   comma += 1;
+   while (*comma == ' ') {
+   fname -= 1;
+   comma += 1;
+   }
+   strncpy(clean_author, comma, fname);
+
*(clean_author+fname) = ' ';
strncpy(clean_author + fname + 1, author, lname);
*(clean_author+fname+1+lname) = '\0';
-- 
1.7.9.5

[PATCH v2] lib: Fix name reordering to handle commas without spaces

[Adam Wolfe Gordon]

Notmuch automatically re-orders names of the format "Last, First" to
"First Last" when the associated email address is
first.l...@example.com. But, if a name is of the format "Last,First"
then notmuch will format the name as "irst Last". Handle any number of
spaces after the comma, including none.
---

Good idea, Tomi. This version handles any number of spaces.

lib/thread.cc |   12 ++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index e976d64..ce6b0ef 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -186,8 +186,16 @@ _thread_cleanup_author (notmuch_thread_t *thread,
 if (comma && strlen(comma) > 1) {
/* let's assemble what we think is the correct name */
lname = comma - author;
-   fname = strlen(author) - lname - 2;
-   strncpy(clean_author, comma + 2, fname);
+
+   /* Skip all the spaces after the comma */
+   fname = strlen(author) - lname - 1;
+   comma += 1;
+   while (*comma == ' ') {
+   fname -= 1;
+   comma += 1;
+   }
+   strncpy(clean_author, comma, fname);
+
*(clean_author+fname) = ' ';
strncpy(clean_author + fname + 1, author, lname);
*(clean_author+fname+1+lname) = '\0';
-- 
1.7.9.5

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] lib: Fix name reordering to handle commas without spaces

[Adam Wolfe Gordon]

Notmuch automatically re-orders names of the format "Last, First" to
"First Last" when the associated email address is
First.Last at example.com. But, if a name is of the format "Last,First"
then notmuch will format the name as "irst Last". Fix this by checking
for a space when doing the reordering.
---
 lib/thread.cc |9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index e976d64..005b355 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -186,8 +186,13 @@ _thread_cleanup_author (notmuch_thread_t *thread,
 if (comma && strlen(comma) > 1) {
/* let's assemble what we think is the correct name */
lname = comma - author;
-   fname = strlen(author) - lname - 2;
-   strncpy(clean_author, comma + 2, fname);
+   if (*(comma + 1) == ' ') {
+   fname = strlen(author) - lname - 2;
+   strncpy(clean_author, comma + 2, fname);
+   } else {
+   fname = strlen(author) - lname - 1;
+   strncpy(clean_author, comma + 1, fname);
+   }
*(clean_author+fname) = ' ';
strncpy(clean_author + fname + 1, author, lname);
*(clean_author+fname+1+lname) = '\0';
-- 
1.7.9.5

[PATCH] lib: Fix name reordering to handle commas without spaces

[Adam Wolfe Gordon]

Notmuch automatically re-orders names of the format "Last, First" to
"First Last" when the associated email address is
first.l...@example.com. But, if a name is of the format "Last,First"
then notmuch will format the name as "irst Last". Fix this by checking
for a space when doing the reordering.
---
 lib/thread.cc |9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index e976d64..005b355 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -186,8 +186,13 @@ _thread_cleanup_author (notmuch_thread_t *thread,
 if (comma && strlen(comma) > 1) {
/* let's assemble what we think is the correct name */
lname = comma - author;
-   fname = strlen(author) - lname - 2;
-   strncpy(clean_author, comma + 2, fname);
+   if (*(comma + 1) == ' ') {
+   fname = strlen(author) - lname - 2;
+   strncpy(clean_author, comma + 2, fname);
+   } else {
+   fname = strlen(author) - lname - 1;
+   strncpy(clean_author, comma + 1, fname);
+   }
*(clean_author+fname) = ' ';
strncpy(clean_author + fname + 1, author, lname);
*(clean_author+fname+1+lname) = '\0';
-- 
1.7.9.5

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] emacs: show: possible w3m/invisibility workaround

[Adam Wolfe Gordon]

On Mon, Jan 14, 2013 at 4:52 PM, David Bremner  wrote:
> Mark Walters  writes:
>
>> There is a bug in the current notmuch code with w3m and invisible
>> parts. w3m sets a keymap, and if we have a hidden [text/html] point
>> at the start of the following line still gets this w3m keymap which
>> causes some strange effects. For example, RET gives an error "No URL
>> at Point" rather than hiding the message,  goes to the next link
>> rather than just down a line.
>
> pushed to relaese and master.

I can confirm this solves the bug I was seeing.

-- Adam

Re: [PATCH v3] emacs: show: possible w3m/invisibility workaround

[Adam Wolfe Gordon]

On Mon, Jan 14, 2013 at 4:52 PM, David Bremner  wrote:
> Mark Walters  writes:
>
>> There is a bug in the current notmuch code with w3m and invisible
>> parts. w3m sets a keymap, and if we have a hidden [text/html] point
>> at the start of the following line still gets this w3m keymap which
>> causes some strange effects. For example, RET gives an error "No URL
>> at Point" rather than hiding the message,  goes to the next link
>> rather than just down a line.
>
> pushed to relaese and master.

I can confirm this solves the bug I was seeing.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs: Handling external dependencies

[Adam Wolfe Gordon]

Hi Damien,

On Sat, Nov 10, 2012 at 8:58 AM, Damien Cassou  
wrote:
> I recently sent a patch for notmuch emacs that depends on a particular
> library. What is the best way to deal with such dependencies?

First off, what's the library, and what is it used for?

I believe that currently the notmuch emacs interface only depends on
stuff that's included with emacs, which is a nice way to be. There are
some packages that can improve the notmuch emacs experience if they
are installed, like w3m. If possible, I'd encourage you to make the
new library recommended, rather than required.

> I can see different solutions:
>
> 1) distribute a rewritten version of the dependency so that the code
> now belongs to notmuch (e.g., replace the name of the library by
> 'notmuch'). This has the disadvantage of requiring maintenance when a
> new version of the library is released and can also be considered
> 'stealing' by some authors.
>
> 2) use a package manager to load the library. This has the
> disadvantage that the now standard package manager is not in
> widespread use yet and is not compatible with other OS-based package
> managers (such as apt-get in Debian).
>
> 3) distribute the dependency with the rest of notmuch and load this
> one. This has the disadvantage of possibly shadowing an already
> existing version of this library installed through a different means.
>
> 4) distribute the dependency with the rest of notmuch (in a separate
> "fallback-libs/" directory) and load it only when requiring the
> library with the standard load-path does not work. Jonas Bernoulli
> gave me a way to do that:
>
> ,
> | (or (require 'THE-LIB nil t)
> | (let ((load-path
> |   (cons (expand-file-name
> |  "fallback-libs"
> |  (file-name-directory (or load-file-name buffer-file-name)))
> | load-path)))
> |   (require 'THE-LIB)))
> `
>
> What do you think?

I'm not big on any of these solutions. I'd suggest just using the
package, documenting it as a dependency, and letting users install it
however they like. This means adding it as a dependency to the
distro-specific packaging (easy for Debian and friends, not sure about
others).

Just my thoughts - offering them mostly since no one else has replied.

-- Adam

Re: emacs: Handling external dependencies

[Adam Wolfe Gordon]

Hi Damien,

On Sat, Nov 10, 2012 at 8:58 AM, Damien Cassou  wrote:
> I recently sent a patch for notmuch emacs that depends on a particular
> library. What is the best way to deal with such dependencies?

First off, what's the library, and what is it used for?

I believe that currently the notmuch emacs interface only depends on
stuff that's included with emacs, which is a nice way to be. There are
some packages that can improve the notmuch emacs experience if they
are installed, like w3m. If possible, I'd encourage you to make the
new library recommended, rather than required.

> I can see different solutions:
>
> 1) distribute a rewritten version of the dependency so that the code
> now belongs to notmuch (e.g., replace the name of the library by
> 'notmuch'). This has the disadvantage of requiring maintenance when a
> new version of the library is released and can also be considered
> 'stealing' by some authors.
>
> 2) use a package manager to load the library. This has the
> disadvantage that the now standard package manager is not in
> widespread use yet and is not compatible with other OS-based package
> managers (such as apt-get in Debian).
>
> 3) distribute the dependency with the rest of notmuch and load this
> one. This has the disadvantage of possibly shadowing an already
> existing version of this library installed through a different means.
>
> 4) distribute the dependency with the rest of notmuch (in a separate
> "fallback-libs/" directory) and load it only when requiring the
> library with the standard load-path does not work. Jonas Bernoulli
> gave me a way to do that:
>
> ,
> | (or (require 'THE-LIB nil t)
> | (let ((load-path
> |   (cons (expand-file-name
> |  "fallback-libs"
> |  (file-name-directory (or load-file-name buffer-file-name)))
> | load-path)))
> |   (require 'THE-LIB)))
> `
>
> What do you think?

I'm not big on any of these solutions. I'd suggest just using the
package, documenting it as a dependency, and letting users install it
however they like. This means adding it as a dependency to the
distro-specific packaging (easy for Debian and friends, not sure about
others).

Just my thoughts - offering them mostly since no one else has replied.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

miraculous from: field

[Adam Wolfe Gordon]

On Wed, Jun 20, 2012 at 5:33 AM, Jani Nikula  wrote:
> The reply template comes from 'notmuch reply' cli command, while the
> forwarding is internal to the emacs ui. I don't have any quick solutions,
> but perhaps in the long run we should do the forwarding template in the cli
> too.

I think this is the right approach. Someone suggested a while ago that
we have a notmuch compose, which would do new message creation (for
frontends that can't do it themselves), reply, and forward in one
place. It would probably support raw and JSON like the current reply
does.

Something I'll do if I find some time, but would be happy to see
someone else work on :-).

-- Adam

Re: miraculous from: field

[Adam Wolfe Gordon]

On Wed, Jun 20, 2012 at 5:33 AM, Jani Nikula  wrote:
> The reply template comes from 'notmuch reply' cli command, while the
> forwarding is internal to the emacs ui. I don't have any quick solutions,
> but perhaps in the long run we should do the forwarding template in the cli
> too.

I think this is the right approach. Someone suggested a while ago that
we have a notmuch compose, which would do new message creation (for
frontends that can't do it themselves), reply, and forward in one
place. It would probably support raw and JSON like the current reply
does.

Something I'll do if I find some time, but would be happy to see
someone else work on :-).

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs complains about encoding?

[Adam Wolfe Gordon]

On Wed, May 23, 2012 at 4:15 AM, Michal Sojka  wrote:
>> I think the current plan is to use the same decoding lookup table that
>> notmuch-show is using in reply too.
>
> Which table do you refer to? notmuch-show-handlers-for?

Yep, that looks like the right thing.

I've been particularly busy with work and other things lately, so I
haven't had time to get this change done. If someone else wants to
work on this I'll be happy to review it; otherwise I'll try to get it
done in the next few weeks. Tomi's gnus-decoded fix went into 0.13.1
anyway, so I don't think the rest is particularly urgent.

-- Adam

Can't compile notmuch-delivery

[Adam Wolfe Gordon]

On Fri, Jun 1, 2012 at 11:01 AM, Jameson Graef Rollins
 wrote:
> On Fri, Jun 01 2012, David Bremner  wrote:
>> I guess we should clarify what it means to accept some code into
>> contrib. Do we accept to maintain it even after the original contributor
>> loses interest?
>
> I don't know, but this is one of the reasons I'm against having
> "contrib" stuff in the notmuch repo. ?If it's not part of the stuff
> we're willing to release in source tarballs or binary packages then it
> should probably be in a separate repo.

+1.

I think having the bindings in the tree and including them in build
system is reasonable, since they're interfaces into the core of the
notmuch library. We should be careful of what bindings we accept in to
the tree (I don't want to fix Fortran bindings ;-) ), but the small
set we have seems maintainable.

It's harder to make a case for contrib. Keeping stuff that relies on
notmuch up-to-date is a nice idea, but doing so creates more work for
developers working on the library and core, since they have to fix a
bunch of code they're not that familiar with. Things that are actively
used (e.g. alot) will be updated quickly anyway since their developers
will tend to keep up with what notmuch is doing.

-- Adam

Re: emacs complains about encoding?

[Adam Wolfe Gordon]

On Wed, May 23, 2012 at 4:15 AM, Michal Sojka  wrote:
>> I think the current plan is to use the same decoding lookup table that
>> notmuch-show is using in reply too.
>
> Which table do you refer to? notmuch-show-handlers-for?

Yep, that looks like the right thing.

I've been particularly busy with work and other things lately, so I
haven't had time to get this change done. If someone else wants to
work on this I'll be happy to review it; otherwise I'll try to get it
done in the next few weeks. Tomi's gnus-decoded fix went into 0.13.1
anyway, so I don't think the rest is particularly urgent.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Can't compile notmuch-delivery

[Adam Wolfe Gordon]

On Fri, Jun 1, 2012 at 11:01 AM, Jameson Graef Rollins
 wrote:
> On Fri, Jun 01 2012, David Bremner  wrote:
>> I guess we should clarify what it means to accept some code into
>> contrib. Do we accept to maintain it even after the original contributor
>> loses interest?
>
> I don't know, but this is one of the reasons I'm against having
> "contrib" stuff in the notmuch repo.  If it's not part of the stuff
> we're willing to release in source tarballs or binary packages then it
> should probably be in a separate repo.

+1.

I think having the bindings in the tree and including them in build
system is reasonable, since they're interfaces into the core of the
notmuch library. We should be careful of what bindings we accept in to
the tree (I don't want to fix Fortran bindings ;-) ), but the small
set we have seems maintainable.

It's harder to make a case for contrib. Keeping stuff that relies on
notmuch up-to-date is a nice idea, but doing so creates more work for
developers working on the library and core, since they have to fix a
bunch of code they're not that familiar with. Things that are actively
used (e.g. alot) will be updated quickly anyway since their developers
will tend to keep up with what notmuch is doing.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs complains about encoding?

[Adam Wolfe Gordon]

On Wed, May 16, 2012 at 3:24 AM, Tomi Ollila  wrote:
> Haa, It doesn't matter which is the original encoding of the message;
>
> notmuch reply id:20120515194455.B7AD5100646 at guru.guru-group.fi
>
> where ?notmuch show --format=raw ^^^ ?outputs (among other lines):
>
> ?Content-Type: text/plain; charset="iso-8859-1"
> ?Content-Transfer-Encoding: quoted-printable
>
> and
>
> notmuch reply id:"878vgsbprq.fsf at nikula.org"
>
> where ?notmuch show --format=raw ^^^ ?outputs (among other lines):
>
> ?Content-Type: text/plain; charset="utf-8"
> ?Content-Transfer-Encoding: base64
>
> produce correct reply content, both in utf-8.
>
> So it is the emacs side which breaks replies.

It turns out it's actually not the emacs side, but an interaction
between our JSON reply format and emacs.

The JSON reply (and show) code includes part content for all text/*
parts except text/html. Because all JSON is required to be UTF-8, it
handles the encoding itself, puts UTF-8 text in, and omits a
content-charset field from the output. Emacs passes on the
content-charset field to mm-display-part-inline if it's available, but
for text/plain parts it's not, leaving mm-display-part-inline to its
own devices for figuring out what the charset is. It seems
mm-display-part-inline correctly figures out that it's UTF-8, and puts
in the series of ugly \nnn characters because that's what emacs does
with UTF-8 sometimes.

In the original reply stuff (pre-JSON reply format) emacs used the
output of notmuch reply verbatim, so all the charset stuff was handled
in notmuch. Before f6c170fabca8f39e74705e3813504137811bf162, emacs was
using the JSON reply format, but was inserting the text itself instead
of using mm-display-part-inline, so emacs still wasn't trying to do
any charset manipulation. Using mm-display-part-inline is desirable
because it lets us handle non-text/plain (e.g. text/html) parts
correctly in reply, and makes the display more consistent (since we
use it for show). But, it leads to this problem.

So, there are a couple of solutions I can see:

1) Have the JSON formats include the original content-charset even
though they're actually outputting UTF-8. Of the solutions I tried,
this is the best, even though it doesn't sound like a good thing to
do.

2) Have the JSON formats include content only if it's actually UTF-8.
This means that for non-UTF-8 parts (including ASCII parts), the emacs
interface has to do more work to display the part content, since it
must fetch it from outside first. When I tried this, it worked but
caused the \nnn to show up when viewing messages in emacs. I suspect
this is because it sets a charset for the whole buffer, and can't
accommodate messages with different charsets in the same buffer
properly. Reply works correctly, though.

3) Have the JSON formats include the charset for all parts, but make
it UTF-8 for all parts they include content for (since we're actually
outputting UTF-8). This doesn't seem to fix the problem, even though
it seems like it should.

If no one has a better idea or a strong reason not to, I'll send a
patch for solution (1).

-- Adam

Re: emacs complains about encoding?

[Adam Wolfe Gordon]

On Wed, May 16, 2012 at 3:24 AM, Tomi Ollila  wrote:
> Haa, It doesn't matter which is the original encoding of the message;
>
> notmuch reply id:20120515194455.b7ad5100...@guru.guru-group.fi
>
> where  notmuch show --format=raw ^^^  outputs (among other lines):
>
>  Content-Type: text/plain; charset="iso-8859-1"
>  Content-Transfer-Encoding: quoted-printable
>
> and
>
> notmuch reply id:"878vgsbprq@nikula.org"
>
> where  notmuch show --format=raw ^^^  outputs (among other lines):
>
>  Content-Type: text/plain; charset="utf-8"
>  Content-Transfer-Encoding: base64
>
> produce correct reply content, both in utf-8.
>
> So it is the emacs side which breaks replies.

It turns out it's actually not the emacs side, but an interaction
between our JSON reply format and emacs.

The JSON reply (and show) code includes part content for all text/*
parts except text/html. Because all JSON is required to be UTF-8, it
handles the encoding itself, puts UTF-8 text in, and omits a
content-charset field from the output. Emacs passes on the
content-charset field to mm-display-part-inline if it's available, but
for text/plain parts it's not, leaving mm-display-part-inline to its
own devices for figuring out what the charset is. It seems
mm-display-part-inline correctly figures out that it's UTF-8, and puts
in the series of ugly \nnn characters because that's what emacs does
with UTF-8 sometimes.

In the original reply stuff (pre-JSON reply format) emacs used the
output of notmuch reply verbatim, so all the charset stuff was handled
in notmuch. Before f6c170fabca8f39e74705e3813504137811bf162, emacs was
using the JSON reply format, but was inserting the text itself instead
of using mm-display-part-inline, so emacs still wasn't trying to do
any charset manipulation. Using mm-display-part-inline is desirable
because it lets us handle non-text/plain (e.g. text/html) parts
correctly in reply, and makes the display more consistent (since we
use it for show). But, it leads to this problem.

So, there are a couple of solutions I can see:

1) Have the JSON formats include the original content-charset even
though they're actually outputting UTF-8. Of the solutions I tried,
this is the best, even though it doesn't sound like a good thing to
do.

2) Have the JSON formats include content only if it's actually UTF-8.
This means that for non-UTF-8 parts (including ASCII parts), the emacs
interface has to do more work to display the part content, since it
must fetch it from outside first. When I tried this, it worked but
caused the \nnn to show up when viewing messages in emacs. I suspect
this is because it sets a charset for the whole buffer, and can't
accommodate messages with different charsets in the same buffer
properly. Reply works correctly, though.

3) Have the JSON formats include the charset for all parts, but make
it UTF-8 for all parts they include content for (since we're actually
outputting UTF-8). This doesn't seem to fix the problem, even though
it seems like it should.

If no one has a better idea or a strong reason not to, I'll send a
patch for solution (1).

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: Force reply to use html2text for consistency

[Adam Wolfe Gordon]

The output of the HTML reply test in the emacs suite can vary
depending on which HTML renderers are installed on the machine running
the tests. The renderer that is always available is emacs's builtin
html2text function. In order to get consistency, force the test to use
html2text even if other renderers are available.
---

As discussed on IRC, here's a patch that forces the HTML reply test to use
html2text, and adjusts the output accordingly. This should fix the test
failure on the buildbot.

 test/emacs |7 ++-
 1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/test/emacs b/test/emacs
index a615b39..e9f954c 100755
--- a/test/emacs
+++ b/test/emacs
@@ -447,7 +447,7 @@ test_expect_equal_file OUTPUT EXPECTED
 test_begin_subtest "Reply within emacs to an html-only message"
 add_message '[content-type]="text/html"' \
'[body]="Hi,This is an HTML test message.OK?"'
-test_emacs "(let ((message-hidden-headers '()))
+test_emacs "(let ((message-hidden-headers '()) (mm-text-html-renderer 
'html2text))
(notmuch-show \"id:${gen_msg_id}\")
(notmuch-show-reply)
(test-output))"
@@ -463,10 +463,7 @@ User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Notmuch Test Suite  writes:

-> Hi,
-> This is an HTML test message.
->
-> OK?
+> Hi,This is an HTML test message.OK?
 EOF
 test_expect_equal_file OUTPUT EXPECTED

-- 
1.7.5.4

[PATCH v2 0/2] Replying to HTML-only emails, v2

[Adam Wolfe Gordon]

On Sun, May 6, 2012 at 6:12 AM, David Bremner  wrote:
> I pushed this series, but it seems to have a test failure on the
> buildbot (it's fine on my Debian wheezey box)
>
> http://buildbot.notmuchmail.org/builders/master/builds/198/steps/tests/logs/problems
>
> Maybe you and Tom could try to figure out what is different about his
> environment?

The failure is on the new HTML reply test, and the difference in the
output is just indentation - the quoted HTML text has an extra three
spaces at the beginning of each line. So, it's almost certainly a
difference in emacs versions or packages. I'm running Ubuntu Oneiric
with emacs 23.3.

It looks like the relevant variable in mm is
mm-text-html-renderer-alist, which defines various methods of parsing
HTML. For me it's:

((w3 . mm-inline-text-html-render-with-w3)
 (w3m . mm-inline-text-html-render-with-w3m)
 (w3m-standalone . mm-inline-text-html-render-with-w3m-standalone)
 (links mm-inline-render-with-file mm-links-remove-leading-blank
"links" "-dump" file)
 (lynx mm-inline-render-with-stdin nil "lynx" "-dump" "-force_html"
"-stdin" "-nolist")
 (html2text mm-inline-render-with-function html2text))

I get the behavior seen on the buildbot if the only option available
on my machine is lynx, so I'm guessing that's what's happening.

I'm not sure what the best option is for fixing this. A few possible options:

* Only run the test if links, w3m, or w3m.el is available.
* Make the test accept differences in whitespace.
* Try to produce HTML for the test that renders the same in every method.

Opinions from anyone?

-- Adam

[PATCH] test: Force reply to use html2text for consistency

[Adam Wolfe Gordon]

The output of the HTML reply test in the emacs suite can vary
depending on which HTML renderers are installed on the machine running
the tests. The renderer that is always available is emacs's builtin
html2text function. In order to get consistency, force the test to use
html2text even if other renderers are available.
---

As discussed on IRC, here's a patch that forces the HTML reply test to use
html2text, and adjusts the output accordingly. This should fix the test
failure on the buildbot.

 test/emacs |7 ++-
 1 files changed, 2 insertions(+), 5 deletions(-)

diff --git a/test/emacs b/test/emacs
index a615b39..e9f954c 100755
--- a/test/emacs
+++ b/test/emacs
@@ -447,7 +447,7 @@ test_expect_equal_file OUTPUT EXPECTED
 test_begin_subtest "Reply within emacs to an html-only message"
 add_message '[content-type]="text/html"' \
'[body]="Hi,This is an HTML test message.OK?"'
-test_emacs "(let ((message-hidden-headers '()))
+test_emacs "(let ((message-hidden-headers '()) (mm-text-html-renderer 
'html2text))
(notmuch-show \"id:${gen_msg_id}\")
(notmuch-show-reply)
(test-output))"
@@ -463,10 +463,7 @@ User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Notmuch Test Suite  writes:
 
-> Hi,
-> This is an HTML test message.
->
-> OK?
+> Hi,This is an HTML test message.OK?
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 0/2] Replying to HTML-only emails, v2

[Adam Wolfe Gordon]

On Sun, May 6, 2012 at 6:12 AM, David Bremner  wrote:
> I pushed this series, but it seems to have a test failure on the
> buildbot (it's fine on my Debian wheezey box)
>
> http://buildbot.notmuchmail.org/builders/master/builds/198/steps/tests/logs/problems
>
> Maybe you and Tom could try to figure out what is different about his
> environment?

The failure is on the new HTML reply test, and the difference in the
output is just indentation - the quoted HTML text has an extra three
spaces at the beginning of each line. So, it's almost certainly a
difference in emacs versions or packages. I'm running Ubuntu Oneiric
with emacs 23.3.

It looks like the relevant variable in mm is
mm-text-html-renderer-alist, which defines various methods of parsing
HTML. For me it's:

((w3 . mm-inline-text-html-render-with-w3)
 (w3m . mm-inline-text-html-render-with-w3m)
 (w3m-standalone . mm-inline-text-html-render-with-w3m-standalone)
 (links mm-inline-render-with-file mm-links-remove-leading-blank
"links" "-dump" file)
 (lynx mm-inline-render-with-stdin nil "lynx" "-dump" "-force_html"
"-stdin" "-nolist")
 (html2text mm-inline-render-with-function html2text))

I get the behavior seen on the buildbot if the only option available
on my machine is lynx, so I'm guessing that's what's happening.

I'm not sure what the best option is for fixing this. A few possible options:

* Only run the test if links, w3m, or w3m.el is available.
* Make the test accept differences in whitespace.
* Try to produce HTML for the test that renders the same in every method.

Opinions from anyone?

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2.1 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.

Mark the test for this feature as not broken.
---
I had removed the narrowing from this patch to check whether it was
actually necessary (it is), and forgot to put it back in. Here is 
a corrected patch.

 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   15 ++-
 emacs/notmuch-show.el |   19 +--
 test/emacs|1 -
 4 files changed, 30 insertions(+), 24 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 6907a5f..7fa441a 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -21,6 +21,8 @@

 ;; This is an part of an emacs-based interface to the notmuch mail system.

+(require 'mm-view)
+(require 'mm-decode)
 (eval-when-compile (require 'cl))

 (defvar notmuch-command "notmuch"
@@ -237,6 +239,23 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))

+(defun notmuch-mm-display-part-inline (msg part nth content-type 
process-crypto)
+  "Use the mm-decode/mm-view functions to display a part in the
+current buffer, if possible."
+  (let ((display-buffer (current-buffer)))
+(with-temp-buffer
+  (let* ((charset (plist-get part :content-charset))
+(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
+   ;; If the user wants the part inlined, insert the content and
+   ;; test whether we are able to inline it (which includes both
+   ;; capability and suitability tests).
+   (when (mm-inlined-p handle)
+ (insert (notmuch-get-bodypart-content msg part nth process-crypto))
+ (when (mm-inlinable-p handle)
+   (set-buffer display-buffer)
+   (mm-display-part handle)
+   t))
+
 ;; Converts a plist of headers to an alist of headers. The input plist should
 ;; have symbols of the form :Header as keys, and the resulting alist will have
 ;; symbols of the form 'Header as keys.
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..fc7ae07 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@

 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)

 (require 'notmuch-lib)
@@ -90,6 +91,14 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))

+(defun notmuch-mua-insert-quotable-part (message part)
+  (save-restriction
+(narrow-to-region (point) (point))
+(notmuch-mm-display-part-inline message part (plist-get part :id)
+   (plist-get part :content-type)
+   notmuch-show-process-crypto)
+(goto-char (point-max
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +178,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))

(set-mark (point))
(goto-char start)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 37f0ebb..d318430 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -524,23 +524,6 @@ message at DEPTH in the current thread."
 (let ((handle (mm-make-handle (current-buffer) (list content-type
   (mm-interactively-view-part handle

-(defun notmuch-show-mm-display-part-inline (msg part nth content-type)
-  "Use the mm-decode/mm-view functions to display a part in the
-current buffer, if possible."
-  (let ((display-buffer (current-buffer)))
-(with-temp-buffer
-  (let* ((charset (plist-get part :content-charset))
-(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
-   ;; If the user wants the part inlined, insert the content and
-   ;; test whether we are able to inline it (which includes both
-   ;; capability and suitability tests).
-   (when (mm-inlined-p handle)
- (insert (notmuch-get-bodypart-content msg part nth

[PATCH v2 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.

Mark the test for this feature as not broken.
---
 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   13 -
 emacs/notmuch-show.el |   19 +--
 test/emacs|1 -
 4 files changed, 28 insertions(+), 24 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 6907a5f..7fa441a 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -21,6 +21,8 @@

 ;; This is an part of an emacs-based interface to the notmuch mail system.

+(require 'mm-view)
+(require 'mm-decode)
 (eval-when-compile (require 'cl))

 (defvar notmuch-command "notmuch"
@@ -237,6 +239,23 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))

+(defun notmuch-mm-display-part-inline (msg part nth content-type 
process-crypto)
+  "Use the mm-decode/mm-view functions to display a part in the
+current buffer, if possible."
+  (let ((display-buffer (current-buffer)))
+(with-temp-buffer
+  (let* ((charset (plist-get part :content-charset))
+(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
+   ;; If the user wants the part inlined, insert the content and
+   ;; test whether we are able to inline it (which includes both
+   ;; capability and suitability tests).
+   (when (mm-inlined-p handle)
+ (insert (notmuch-get-bodypart-content msg part nth process-crypto))
+ (when (mm-inlinable-p handle)
+   (set-buffer display-buffer)
+   (mm-display-part handle)
+   t))
+
 ;; Converts a plist of headers to an alist of headers. The input plist should
 ;; have symbols of the form :Header as keys, and the resulting alist will have
 ;; symbols of the form 'Header as keys.
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..88d7505 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@

 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)

 (require 'notmuch-lib)
@@ -90,6 +91,12 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))

+(defun notmuch-mua-insert-quotable-part (message part)
+  (notmuch-mm-display-part-inline message part (plist-get part :id)
+ (plist-get part :content-type)
+ notmuch-show-process-crypto)
+  (goto-char (point-max)))
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +176,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))

(set-mark (point))
(goto-char start)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 37f0ebb..d318430 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -524,23 +524,6 @@ message at DEPTH in the current thread."
 (let ((handle (mm-make-handle (current-buffer) (list content-type
   (mm-interactively-view-part handle

-(defun notmuch-show-mm-display-part-inline (msg part nth content-type)
-  "Use the mm-decode/mm-view functions to display a part in the
-current buffer, if possible."
-  (let ((display-buffer (current-buffer)))
-(with-temp-buffer
-  (let* ((charset (plist-get part :content-charset))
-(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
-   ;; If the user wants the part inlined, insert the content and
-   ;; test whether we are able to inline it (which includes both
-   ;; capability and suitability tests).
-   (when (mm-inlined-p handle)
- (insert (notmuch-get-bodypart-content msg part nth 
notmuch-show-process-crypto))
- (when (mm-inlinable-p handle)
-   (set-buffer display-buffer)
-   (mm-display-part handle)
-   t))
-
 (defun notmuch-show-multipart/*-to-list (part)
   (mapc

[PATCH v2 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

With the latest reply infrastructure, we should be able to nicely
quote HTML-only emails. But currently emacs quotes the raw HTML
instead of parsing it first. This commit adds a test for this case.

This test currently marked as broken.
---
 test/emacs |   27 +++
 1 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 38decd0..5f238d9 100755
--- a/test/emacs
+++ b/test/emacs
@@ -444,6 +444,33 @@ Alex Botero-Lowry  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED

+test_begin_subtest "Reply within emacs to an html-only message"
+test_subtest_known_broken
+add_message '[content-type]="text/html"' \
+   '[body]="Hi,This is an HTML test message.OK?"'
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-show \"id:${gen_msg_id}\")
+   (notmuch-show-reply)
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
+cat 
+Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
+--text follows this line--
+Notmuch Test Suite  writes:
+
+> Hi,
+> This is an HTML test message.
+>
+> OK?
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Quote MML tags in reply"
 message_id='test-emacs-mml-quoting at message.id'
 add_message [id]="$message_id" \
-- 
1.7.5.4

[PATCH v2 0/2] Replying to HTML-only emails, v2

[Adam Wolfe Gordon]

Hi everyone,

Here is a new version of my HTML reply series [1], addressing Autin's review
comments. The reply code now shares more with the similar notmuch-show code,
which is a good thing.

Reviews welcome, but I'm hoping this can be in 0.13.

-- Adam

[1] id:"1335056093-17621-1-git-send-email-awg+notmuch at xvx.ca"

Adam Wolfe Gordon (2):
  test: Replying to an HTML-only message in emacs
  emacs: Correctly quote non-text/plain parts in reply

 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   13 -
 emacs/notmuch-show.el |   19 +--
 test/emacs|   26 ++
 4 files changed, 54 insertions(+), 23 deletions(-)

-- 
1.7.5.4

[PATCH 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

On Fri, May 4, 2012 at 1:05 PM, Austin Clements  wrote:
> Personally I think the narrowing trick is clever, but I worry that it
> assumes too much about how mm-display-part works, since mm-display-part
> just takes a buffer in the handle. ?Is there a reason this doesn't
> simply use notmuch-show-mm-display-part-inline? ?Something like
> (untested)
>
> (defun notmuch-mua-insert-quotable-part (message part)
> ?(notmuch-show-mm-display-part-inline
> ? message part (plist-get part :id) (plist-get part :content-type)))
>
> You might not even need notmuch-mua-insert-quotable-part. ?(Why does
> notmuch-show-mm-display-part-inline take all of those redundant
> arguments?)

This almost works - patch forthcoming. The trouble with just using
notmuch-show-mm-display-part-inline is that it doesn't move the point
to the end of the text, which we rely on for message-cite-original to
quote the right region. But, I can use
notmuch-show-mm-display-part-inline (moved to the lib) by narrowing
first, and setting the point after using it. If we don't narrow, the
point ends up after the user's signature (which is inserted by message
mode before we insert the quoted text), and the signature ends up
being quoted.

> In general, I feel like the reply code should share more structure with
> the notmuch-show code. ?I worry that the quoted text people wind up with
> may not resemble the text they saw in the show buffer because the two
> code paths use different rules. ?But addressing that (if it's
> addressable) should be done in a later series.

Yeah, I agree with this. Maybe before 0.14 I will try to do some
refactoring to share code between reply and show, at least in emacs.

-- Adam

[PATCH 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

On Fri, May 4, 2012 at 12:47 PM, Austin Clements  wrote:
> Does the test output depend on the HTML renderer used? ?(And should
> there be a test dependency on w3m or something? ?I must admit, I don't
> know how mm's HTML rendering works.)

That's a good question. HTML output does depend on what renderer is
used, but I think the test message is simple enough that they'll all
produce the same output. I believe mm prefers w3m.el, and falls back
to using an external w3m binary, and then emacs's built-in HTML
parsing (I'm significantly less sure whether the last part is true).

I've tried removing w3m and w3m.el from my system, and the test still
passes, so I think we're OK.

-- Adam

[PATCH v2.1 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.

Mark the test for this feature as not broken.
---
I had removed the narrowing from this patch to check whether it was
actually necessary (it is), and forgot to put it back in. Here is 
a corrected patch.

 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   15 ++-
 emacs/notmuch-show.el |   19 +--
 test/emacs|1 -
 4 files changed, 30 insertions(+), 24 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 6907a5f..7fa441a 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -21,6 +21,8 @@
 
 ;; This is an part of an emacs-based interface to the notmuch mail system.
 
+(require 'mm-view)
+(require 'mm-decode)
 (eval-when-compile (require 'cl))
 
 (defvar notmuch-command "notmuch"
@@ -237,6 +239,23 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))
 
+(defun notmuch-mm-display-part-inline (msg part nth content-type 
process-crypto)
+  "Use the mm-decode/mm-view functions to display a part in the
+current buffer, if possible."
+  (let ((display-buffer (current-buffer)))
+(with-temp-buffer
+  (let* ((charset (plist-get part :content-charset))
+(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
+   ;; If the user wants the part inlined, insert the content and
+   ;; test whether we are able to inline it (which includes both
+   ;; capability and suitability tests).
+   (when (mm-inlined-p handle)
+ (insert (notmuch-get-bodypart-content msg part nth process-crypto))
+ (when (mm-inlinable-p handle)
+   (set-buffer display-buffer)
+   (mm-display-part handle)
+   t))
+
 ;; Converts a plist of headers to an alist of headers. The input plist should
 ;; have symbols of the form :Header as keys, and the resulting alist will have
 ;; symbols of the form 'Header as keys.
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..fc7ae07 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@
 
 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)
 
 (require 'notmuch-lib)
@@ -90,6 +91,14 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))
 
+(defun notmuch-mua-insert-quotable-part (message part)
+  (save-restriction
+(narrow-to-region (point) (point))
+(notmuch-mm-display-part-inline message part (plist-get part :id)
+   (plist-get part :content-type)
+   notmuch-show-process-crypto)
+(goto-char (point-max
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +178,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))
 
(set-mark (point))
(goto-char start)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 37f0ebb..d318430 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -524,23 +524,6 @@ message at DEPTH in the current thread."
 (let ((handle (mm-make-handle (current-buffer) (list content-type
   (mm-interactively-view-part handle
 
-(defun notmuch-show-mm-display-part-inline (msg part nth content-type)
-  "Use the mm-decode/mm-view functions to display a part in the
-current buffer, if possible."
-  (let ((display-buffer (current-buffer)))
-(with-temp-buffer
-  (let* ((charset (plist-get part :content-charset))
-(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
-   ;; If the user wants the part inlined, insert the content and
-   ;; test whether we are able to inline it (which includes both
-   ;; capability and suitability tests).
-   (when (mm-inlined-p handle)
- (insert (notmuch-get-bodypart-content msg

[PATCH v2 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly. We re-use some code from notmuch-show
(notmuch-show-mm-display-part-inline), which has been moved to
notmuch-lib.el.

Mark the test for this feature as not broken.
---
 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   13 -
 emacs/notmuch-show.el |   19 +--
 test/emacs|1 -
 4 files changed, 28 insertions(+), 24 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 6907a5f..7fa441a 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -21,6 +21,8 @@
 
 ;; This is an part of an emacs-based interface to the notmuch mail system.
 
+(require 'mm-view)
+(require 'mm-decode)
 (eval-when-compile (require 'cl))
 
 (defvar notmuch-command "notmuch"
@@ -237,6 +239,23 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))
 
+(defun notmuch-mm-display-part-inline (msg part nth content-type 
process-crypto)
+  "Use the mm-decode/mm-view functions to display a part in the
+current buffer, if possible."
+  (let ((display-buffer (current-buffer)))
+(with-temp-buffer
+  (let* ((charset (plist-get part :content-charset))
+(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
+   ;; If the user wants the part inlined, insert the content and
+   ;; test whether we are able to inline it (which includes both
+   ;; capability and suitability tests).
+   (when (mm-inlined-p handle)
+ (insert (notmuch-get-bodypart-content msg part nth process-crypto))
+ (when (mm-inlinable-p handle)
+   (set-buffer display-buffer)
+   (mm-display-part handle)
+   t))
+
 ;; Converts a plist of headers to an alist of headers. The input plist should
 ;; have symbols of the form :Header as keys, and the resulting alist will have
 ;; symbols of the form 'Header as keys.
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..88d7505 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@
 
 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)
 
 (require 'notmuch-lib)
@@ -90,6 +91,12 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))
 
+(defun notmuch-mua-insert-quotable-part (message part)
+  (notmuch-mm-display-part-inline message part (plist-get part :id)
+ (plist-get part :content-type)
+ notmuch-show-process-crypto)
+  (goto-char (point-max)))
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +176,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))
 
(set-mark (point))
(goto-char start)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 37f0ebb..d318430 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -524,23 +524,6 @@ message at DEPTH in the current thread."
 (let ((handle (mm-make-handle (current-buffer) (list content-type
   (mm-interactively-view-part handle
 
-(defun notmuch-show-mm-display-part-inline (msg part nth content-type)
-  "Use the mm-decode/mm-view functions to display a part in the
-current buffer, if possible."
-  (let ((display-buffer (current-buffer)))
-(with-temp-buffer
-  (let* ((charset (plist-get part :content-charset))
-(handle (mm-make-handle (current-buffer) `(,content-type (charset 
. ,charset)
-   ;; If the user wants the part inlined, insert the content and
-   ;; test whether we are able to inline it (which includes both
-   ;; capability and suitability tests).
-   (when (mm-inlined-p handle)
- (insert (notmuch-get-bodypart-content msg part nth 
notmuch-show-process-crypto))
- (when (mm-inlinable-p handle)
-   (set-buffer display-buffer)
-   (mm-display-part handle)
-   t))
-
 (defun notmuch-show-multipart/*-to-list (part)

[PATCH v2 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

With the latest reply infrastructure, we should be able to nicely
quote HTML-only emails. But currently emacs quotes the raw HTML
instead of parsing it first. This commit adds a test for this case.

This test currently marked as broken.
---
 test/emacs |   27 +++
 1 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 38decd0..5f238d9 100755
--- a/test/emacs
+++ b/test/emacs
@@ -444,6 +444,33 @@ Alex Botero-Lowry  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "Reply within emacs to an html-only message"
+test_subtest_known_broken
+add_message '[content-type]="text/html"' \
+   '[body]="Hi,This is an HTML test message.OK?"'
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-show \"id:${gen_msg_id}\")
+   (notmuch-show-reply)
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
+cat 
+Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
+--text follows this line--
+Notmuch Test Suite  writes:
+
+> Hi,
+> This is an HTML test message.
+>
+> OK?
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Quote MML tags in reply"
 message_id='test-emacs-mml-quot...@message.id'
 add_message [id]="$message_id" \
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 0/2] Replying to HTML-only emails, v2

[Adam Wolfe Gordon]

Hi everyone,

Here is a new version of my HTML reply series [1], addressing Autin's review
comments. The reply code now shares more with the similar notmuch-show code,
which is a good thing.

Reviews welcome, but I'm hoping this can be in 0.13.

-- Adam

[1] id:"1335056093-17621-1-git-send-email-awg+notm...@xvx.ca"

Adam Wolfe Gordon (2):
  test: Replying to an HTML-only message in emacs
  emacs: Correctly quote non-text/plain parts in reply

 emacs/notmuch-lib.el  |   19 +++
 emacs/notmuch-mua.el  |   13 -
 emacs/notmuch-show.el |   19 +--
 test/emacs|   26 ++
 4 files changed, 54 insertions(+), 23 deletions(-)

-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

On Fri, May 4, 2012 at 1:05 PM, Austin Clements  wrote:
> Personally I think the narrowing trick is clever, but I worry that it
> assumes too much about how mm-display-part works, since mm-display-part
> just takes a buffer in the handle.  Is there a reason this doesn't
> simply use notmuch-show-mm-display-part-inline?  Something like
> (untested)
>
> (defun notmuch-mua-insert-quotable-part (message part)
>  (notmuch-show-mm-display-part-inline
>   message part (plist-get part :id) (plist-get part :content-type)))
>
> You might not even need notmuch-mua-insert-quotable-part.  (Why does
> notmuch-show-mm-display-part-inline take all of those redundant
> arguments?)

This almost works - patch forthcoming. The trouble with just using
notmuch-show-mm-display-part-inline is that it doesn't move the point
to the end of the text, which we rely on for message-cite-original to
quote the right region. But, I can use
notmuch-show-mm-display-part-inline (moved to the lib) by narrowing
first, and setting the point after using it. If we don't narrow, the
point ends up after the user's signature (which is inserted by message
mode before we insert the quoted text), and the signature ends up
being quoted.

> In general, I feel like the reply code should share more structure with
> the notmuch-show code.  I worry that the quoted text people wind up with
> may not resemble the text they saw in the show buffer because the two
> code paths use different rules.  But addressing that (if it's
> addressable) should be done in a later series.

Yeah, I agree with this. Maybe before 0.14 I will try to do some
refactoring to share code between reply and show, at least in emacs.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

On Fri, May 4, 2012 at 12:47 PM, Austin Clements  wrote:
> Does the test output depend on the HTML renderer used?  (And should
> there be a test dependency on w3m or something?  I must admit, I don't
> know how mm's HTML rendering works.)

That's a good question. HTML output does depend on what renderer is
used, but I think the test message is simple enough that they'll all
produce the same output. I believe mm prefers w3m.el, and falls back
to using an external w3m binary, and then emacs's built-in HTML
parsing (I'm significantly less sure whether the last part is true).

I've tried removing w3m and w3m.el from my system, and the test still
passes, so I think we're OK.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: Do not pass stderr of notmuch reply to JSON parser

[Adam Wolfe Gordon]

On Tue, May 1, 2012 at 3:10 PM, Michal Sojka  wrote:
> Sometimes, notmuch reply outputs something to stderr, for example:
> "Failed to verify signed part: Cannot verify multipart/signed part:
> unsupported signature protocol". When this happens, replying in emacs
> fails, because emacs cannot parse the error message as JSON.
>
> This patch causes emacs to ignore stderr when reading reply from
> notmuch.

LGTM, as I said in the other thread. Thanks for the fix.

-- Adam

Re: [PATCH] emacs: Do not pass stderr of notmuch reply to JSON parser

[Adam Wolfe Gordon]

On Tue, May 1, 2012 at 3:10 PM, Michal Sojka  wrote:
> Sometimes, notmuch reply outputs something to stderr, for example:
> "Failed to verify signed part: Cannot verify multipart/signed part:
> unsupported signature protocol". When this happens, replying in emacs
> fails, because emacs cannot parse the error message as JSON.
>
> This patch causes emacs to ignore stderr when reading reply from
> notmuch.

LGTM, as I said in the other thread. Thanks for the fix.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

JSON readtable error when replying

[Adam Wolfe Gordon]

On Mon, Apr 30, 2012 at 10:50, Michal Sojka  wrote:
> json_xs didn't complain, but I've found that notmuch outputs
> ?Failed to verify signed part: Cannot verify multipart/signed part: 
> unsupported signature protocol 'application/x-pkcs7-signature'.
> to stderr before the json output and emacs tries to parse
> stderr. Emacs then complains about not finding "F" in json-readtable.
>
> The following patch fixes the problem for me, but I do not know if it is
> a good thing to throw away the stderr output of notmuch.
>
> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
> index 87bd88d..49db603 100644
> --- a/emacs/notmuch-mua.el
> +++ b/emacs/notmuch-mua.el
> @@ -113,7 +113,7 @@ list."
>
> ? ? ;; Get the reply object as JSON, and parse it into an elisp object.
> ? ? (with-temp-buffer
> - ? ? ?(apply 'call-process (append (list notmuch-command nil (list t t) nil) 
> args))
> + ? ? ?(apply 'call-process (append (list notmuch-command nil (list t nil) 
> nil) args))
> ? ? ? (goto-char (point-min))
> ? ? ? (let ((json-object-type 'plist)
> ? ? ? ? ? ?(json-array-type 'list)
>
> Comments?

Good catch. I'm not sure what (if anything) we want to do with the
stderr output, but putting it in the buffer with the JSON is
definitely not the right thing. Your fix looks fine to me - would you
mind doing a commit and sending a patch to the list?

David, it would probably be good to get this (pretty trivial) fix in
before 0.13, since it will include the JSON reply stuff.

-- Adam

Re: JSON readtable error when replying

[Adam Wolfe Gordon]

On Mon, Apr 30, 2012 at 10:50, Michal Sojka  wrote:
> json_xs didn't complain, but I've found that notmuch outputs
>  Failed to verify signed part: Cannot verify multipart/signed part: 
> unsupported signature protocol 'application/x-pkcs7-signature'.
> to stderr before the json output and emacs tries to parse
> stderr. Emacs then complains about not finding "F" in json-readtable.
>
> The following patch fixes the problem for me, but I do not know if it is
> a good thing to throw away the stderr output of notmuch.
>
> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
> index 87bd88d..49db603 100644
> --- a/emacs/notmuch-mua.el
> +++ b/emacs/notmuch-mua.el
> @@ -113,7 +113,7 @@ list."
>
>     ;; Get the reply object as JSON, and parse it into an elisp object.
>     (with-temp-buffer
> -      (apply 'call-process (append (list notmuch-command nil (list t t) nil) 
> args))
> +      (apply 'call-process (append (list notmuch-command nil (list t nil) 
> nil) args))
>       (goto-char (point-min))
>       (let ((json-object-type 'plist)
>            (json-array-type 'list)
>
> Comments?

Good catch. I'm not sure what (if anything) we want to do with the
stderr output, but putting it in the buffer with the JSON is
definitely not the right thing. Your fix looks fine to me - would you
mind doing a commit and sending a patch to the list?

David, it would probably be good to get this (pretty trivial) fix in
before 0.13, since it will include the JSON reply stuff.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

JSON readtable error when replying

[Adam Wolfe Gordon]

Hi Michal,

On Mon, Apr 30, 2012 at 09:00, Michal Sojka  wrote:
> when I try to reply within emacs to a particular message, I get "JSON
> readtable error". This happens with with the current git master as well
> as with a version from about a month ago. Is this a known problem or
> shall I dig into it and find what wrong?

Sounds like notmuch reply is producing bad JSON for that particular
message. Could you try the following, replacing  with the
message ID of the message you're replying to:

notmuch reply --format=json id:"" | json_xs -t json-pretty

and see if it complains? The json_xs tool is in the libjson-xs-perl
package in Ubuntu. If you can't get it, run just the notmuch reply
command, and see what it does - the JSON will be hard to read, but if
it's producing an error of some sort you'll see it.

-- Adam

Re: JSON readtable error when replying

[Adam Wolfe Gordon]

Hi Michal,

On Mon, Apr 30, 2012 at 09:00, Michal Sojka  wrote:
> when I try to reply within emacs to a particular message, I get "JSON
> readtable error". This happens with with the current git master as well
> as with a version from about a month ago. Is this a known problem or
> shall I dig into it and find what wrong?

Sounds like notmuch reply is producing bad JSON for that particular
message. Could you try the following, replacing  with the
message ID of the message you're replying to:

notmuch reply --format=json id:"" | json_xs -t json-pretty

and see if it complains? The json_xs tool is in the libjson-xs-perl
package in Ubuntu. If you can't get it, run just the notmuch reply
command, and see what it does - the JSON will be hard to read, but if
it's producing an error of some sort you'll see it.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[RFC] Use JSON in emacs interface

[Adam Wolfe Gordon]

Hi Chris,

On Sun, Apr 29, 2012 at 10:22, Chris Gray  wrote:
> I first thought of changing the regex so that it looked for the last
> semicolon in the string or something like that, but that would just move
> the problem. ?(Semicolons are probably more frequent in subject lines
> than in author names.) ?So it seems to me that what is needed is for
> notmuch and emacs to talk with each other in a format that is
> unambiguously parseable. ?Since notmuch search already has the option of
> outputting to JSON, that seems like a natural fit.
>
> Emacs has an existing JSON parser,
> ,
> but it doesn't appear that it is able to parse progressively, meaning
> that it wouldn't be able to display results as they come in from notmuch
> search if used as-is. ?My guess is that its parts could be hacked
> together to overcome this limitation though.
>
> Anyway, if others think this is a good idea, I'm willing to do the
> coding.

I think this is a great idea. If you look at notmuch-mua.el and
notmuch-query.el, you'll see that we already use json.el for reply and
parts of show.

As for parsing progressively to show search results as they arrive,
I'd be inclined to instead implement paging, so emacs could ask for
just the first n results, display them, then ask for the next n
results, etc. Or maybe ask for the results grouped, so that there
would be a valid JSON object for the first n results, then another for
the next n, etc. This might be a tricky thing to design and implement,
but I think it's been discussed before as something that would be nice
to have.

Personally, I think it would be fine to implement the JSON search
first, then deal with progressive parsing and/or grouping, but others
may differ here. Either way, I'd be happy to review patches for this
and offer any suggestions.

-- Adam

Re: [RFC] Use JSON in emacs interface

[Adam Wolfe Gordon]

Hi Chris,

On Sun, Apr 29, 2012 at 10:22, Chris Gray  wrote:
> I first thought of changing the regex so that it looked for the last
> semicolon in the string or something like that, but that would just move
> the problem.  (Semicolons are probably more frequent in subject lines
> than in author names.)  So it seems to me that what is needed is for
> notmuch and emacs to talk with each other in a format that is
> unambiguously parseable.  Since notmuch search already has the option of
> outputting to JSON, that seems like a natural fit.
>
> Emacs has an existing JSON parser,
> ,
> but it doesn't appear that it is able to parse progressively, meaning
> that it wouldn't be able to display results as they come in from notmuch
> search if used as-is.  My guess is that its parts could be hacked
> together to overcome this limitation though.
>
> Anyway, if others think this is a good idea, I'm willing to do the
> coding.

I think this is a great idea. If you look at notmuch-mua.el and
notmuch-query.el, you'll see that we already use json.el for reply and
parts of show.

As for parsing progressively to show search results as they arrive,
I'd be inclined to instead implement paging, so emacs could ask for
just the first n results, display them, then ask for the next n
results, etc. Or maybe ask for the results grouped, so that there
would be a valid JSON object for the first n results, then another for
the next n, etc. This might be a tricky thing to design and implement,
but I think it's been discussed before as something that would be nice
to have.

Personally, I think it would be fine to implement the JSON search
first, then deal with progressive parsing and/or grouping, but others
may differ here. Either way, I'd be happy to review patches for this
and offer any suggestions.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

priorities for 0.13

[Adam Wolfe Gordon]

Hi David,

On Wed, Apr 25, 2012 at 06:31, David Bremner  wrote:
> I'd like to have a feature freeze for 0.13 sometime in the first week of
> May. ?What do people feel are priorities to try to get reviewed and
> pushed for 0.13?

I would like to get the HTML reply series [1] in before 0.13, since
otherwise the reply stuff that's already pushed can cause some ugly
(but not buggy) behavior, quoting raw HTML in reply. There's been one
review, and it's a small patch, but I think it could use another set
of eyes.

[1] id:"1335056093-17621-1-git-send-email-awg+notmuch at xvx.ca"

-- Adam

Re: priorities for 0.13

[Adam Wolfe Gordon]

Hi David,

On Wed, Apr 25, 2012 at 06:31, David Bremner  wrote:
> I'd like to have a feature freeze for 0.13 sometime in the first week of
> May.  What do people feel are priorities to try to get reviewed and
> pushed for 0.13?

I would like to get the HTML reply series [1] in before 0.13, since
otherwise the reply stuff that's already pushed can cause some ugly
(but not buggy) behavior, quoting raw HTML in reply. There's been one
review, and it's a small patch, but I think it could use another set
of eyes.

[1] id:"1335056093-17621-1-git-send-email-awg+notm...@xvx.ca"

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[ANN] New awesome vim plug-in using Ruby bindings

[Adam Wolfe Gordon]

On Mon, Apr 23, 2012 at 05:53, Felipe Contreras
 wrote:
> I do have w3.el installed. I can read the HTML, but when scrolling and
> selecting there are glitches, and when replying nothing gets quoted.

Ah, I see. As I said before, reply will work soon. The other issues
sound like emacs/w3 problems, out of our control. Thanks for
explaining.

-- Adam

Re: [ANN] New awesome vim plug-in using Ruby bindings

[Adam Wolfe Gordon]

On Mon, Apr 23, 2012 at 05:53, Felipe Contreras
 wrote:
> I do have w3.el installed. I can read the HTML, but when scrolling and
> selecting there are glitches, and when replying nothing gets quoted.

Ah, I see. As I said before, reply will work soon. The other issues
sound like emacs/w3 problems, out of our control. Thanks for
explaining.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[ANN] New awesome vim plug-in using Ruby bindings

[Adam Wolfe Gordon]

Hi Felipe,

This work sounds nice - it's good to have lots of interface choices.
One question below:

On Sun, Apr 22, 2012 at 19:12, Felipe Contreras
 wrote:
> doesn't have support for that), and even learning emacs, to use what
> most people here use (but it turns out the HTML messages don't work
> correctly there either). I also tried the various mutt+notmuch

HTML emails should work properly in emacs, and work even better if you
have w3.el installed. Replying to HTML emails isn't in git yet, but
there are patches for it in progress [1]. If there's a bug in the HTML
support, we should probably fix it, so could you explain what doesn't
work for you?

Thanks,

-- Adam

[1] id:"1335056093-17621-1-git-send-email-awg+notmuch at xvx.ca"

Re: [ANN] New awesome vim plug-in using Ruby bindings

[Adam Wolfe Gordon]

Hi Felipe,

This work sounds nice - it's good to have lots of interface choices.
One question below:

On Sun, Apr 22, 2012 at 19:12, Felipe Contreras
 wrote:
> doesn't have support for that), and even learning emacs, to use what
> most people here use (but it turns out the HTML messages don't work
> correctly there either). I also tried the various mutt+notmuch

HTML emails should work properly in emacs, and work even better if you
have w3.el installed. Replying to HTML emails isn't in git yet, but
there are patches for it in progress [1]. If there's a bug in the HTML
support, we should probably fix it, so could you explain what doesn't
work for you?

Thanks,

-- Adam

[1] id:"1335056093-17621-1-git-send-email-awg+notm...@xvx.ca"
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly.

Mark the test for this feature as not broken.
---
 emacs/notmuch-mua.el |   20 +++-
 test/emacs   |1 -
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..f7af789 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@

 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)

 (require 'notmuch-lib)
@@ -90,6 +91,19 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))

+(defun notmuch-mua-insert-quotable-part (message part)
+  (save-restriction
+(narrow-to-region (point) (point))
+(insert (notmuch-get-bodypart-content message part
+ (plist-get part :id)
+ notmuch-show-process-crypto))
+(let ((handle (mm-make-handle (current-buffer)
+ (list (plist-get part :content-type
+ (end-of-orig (point-max)))
+  (mm-display-part handle)
+  (kill-region (point-min) end-of-orig))
+(goto-char (point-max
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +183,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))

(set-mark (point))
(goto-char start)
diff --git a/test/emacs b/test/emacs
index e648f80..579844f 100755
--- a/test/emacs
+++ b/test/emacs
@@ -445,7 +445,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to an html-only message"
-test_subtest_known_broken
 add_message '[content-type]="text/html"' \
'[body]="Hi,This is an HTML test message.OK?"'
 test_emacs "(let ((message-hidden-headers '()))
-- 
1.7.5.4

[PATCH 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

With the latest reply infrastructure, we should be able to nicely
quote HTML-only emails. But currently emacs quotes the raw HTML
instead of parsing it first. This commit adds a test for this case.

This test currently marked as broken.
---
 test/emacs |   27 +++
 1 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index c7510e9..e648f80 100755
--- a/test/emacs
+++ b/test/emacs
@@ -444,6 +444,33 @@ Alex Botero-Lowry  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED

+test_begin_subtest "Reply within emacs to an html-only message"
+test_subtest_known_broken
+add_message '[content-type]="text/html"' \
+   '[body]="Hi,This is an HTML test message.OK?"'
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-show \"id:${gen_msg_id}\")
+   (notmuch-show-reply)
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
+cat 
+Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
+--text follows this line--
+Notmuch Test Suite  writes:
+
+> Hi,
+> This is an HTML test message.
+>
+> OK?
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Quote MML tags in reply"
 message_id='test-emacs-mml-quoting at message.id'
 add_message [id]="$message_id" \
-- 
1.7.5.4

[PATCH 0/2] Replying to HTML-only emails

[Adam Wolfe Gordon]

Hi all,

My recent reply enhancements were originally intended to allow proper
quoting of HTML-only email in reply. While the final version was a big
improvement on reply in general, it didn't actually acheive this goal.

So, this series finishes that work, using mm-display-part to render the
content of the original message correctly before inserting it into the reply
message. I've also added a test for this feature.

I'm not 100% happy with how the code looks for this, but I couldn't figure
out a way to make it work without the kind of ugly narrow-to-region. If
anyone has suggestions here I'd like to hear them.

Adam Wolfe Gordon (2):
  test: Replying to an HTML-only message in emacs
  emacs: Correctly quote non-text/plain parts in reply

 emacs/notmuch-mua.el |   20 +++-
 test/emacs   |   26 ++
 2 files changed, 41 insertions(+), 5 deletions(-)

-- 
1.7.5.4

[PATCH 0/2] Replying to HTML-only emails

[Adam Wolfe Gordon]

Hi all,

My recent reply enhancements were originally intended to allow proper
quoting of HTML-only email in reply. While the final version was a big
improvement on reply in general, it didn't actually acheive this goal.

So, this series finishes that work, using mm-display-part to render the
content of the original message correctly before inserting it into the reply
message. I've also added a test for this feature.

I'm not 100% happy with how the code looks for this, but I couldn't figure
out a way to make it work without the kind of ugly narrow-to-region. If
anyone has suggestions here I'd like to hear them.

Adam Wolfe Gordon (2):
  test: Replying to an HTML-only message in emacs
  emacs: Correctly quote non-text/plain parts in reply

 emacs/notmuch-mua.el |   20 +++-
 test/emacs   |   26 ++
 2 files changed, 41 insertions(+), 5 deletions(-)

-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] emacs: Correctly quote non-text/plain parts in reply

[Adam Wolfe Gordon]

Quote non-text parts nicely by displaying them with mm-display-part
before calling message-cite-original to quote them. HTML-only emails
can now be quoted correctly.

Mark the test for this feature as not broken.
---
 emacs/notmuch-mua.el |   20 +++-
 test/emacs   |1 -
 2 files changed, 15 insertions(+), 6 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..f7af789 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -21,6 +21,7 @@
 
 (require 'json)
 (require 'message)
+(require 'mm-view)
 (require 'format-spec)
 
 (require 'notmuch-lib)
@@ -90,6 +91,19 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))
 
+(defun notmuch-mua-insert-quotable-part (message part)
+  (save-restriction
+(narrow-to-region (point) (point))
+(insert (notmuch-get-bodypart-content message part
+ (plist-get part :id)
+ notmuch-show-process-crypto))
+(let ((handle (mm-make-handle (current-buffer)
+ (list (plist-get part :content-type
+ (end-of-orig (point-max)))
+  (mm-display-part handle)
+  (kill-region (point-min) end-of-orig))
+(goto-char (point-max
+
 ;; There is a bug in emacs 23's message.el that results in a newline
 ;; not being inserted after the References header, so the next header
 ;; is concatenated to the end of it. This function fixes the problem,
@@ -169,11 +183,7 @@ list."
;; Get the parts of the original message that should be quoted; this 
includes
;; all the text parts, except the non-preferred ones in a 
multipart/alternative.
(let ((quotable-parts (notmuch-mua-get-quotable-parts (plist-get 
original :body
- (mapc (lambda (part)
- (insert (notmuch-get-bodypart-content original part
-   (plist-get part :id)
-   
notmuch-show-process-crypto)))
-   quotable-parts))
+ (mapc (apply-partially 'notmuch-mua-insert-quotable-part original) 
quotable-parts))
 
(set-mark (point))
(goto-char start)
diff --git a/test/emacs b/test/emacs
index e648f80..579844f 100755
--- a/test/emacs
+++ b/test/emacs
@@ -445,7 +445,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to an html-only message"
-test_subtest_known_broken
 add_message '[content-type]="text/html"' \
'[body]="Hi,This is an HTML test message.OK?"'
 test_emacs "(let ((message-hidden-headers '()))
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/2] test: Replying to an HTML-only message in emacs

[Adam Wolfe Gordon]

With the latest reply infrastructure, we should be able to nicely
quote HTML-only emails. But currently emacs quotes the raw HTML
instead of parsing it first. This commit adds a test for this case.

This test currently marked as broken.
---
 test/emacs |   27 +++
 1 files changed, 27 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index c7510e9..e648f80 100755
--- a/test/emacs
+++ b/test/emacs
@@ -444,6 +444,33 @@ Alex Botero-Lowry  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "Reply within emacs to an html-only message"
+test_subtest_known_broken
+add_message '[content-type]="text/html"' \
+   '[body]="Hi,This is an HTML test message.OK?"'
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-show \"id:${gen_msg_id}\")
+   (notmuch-show-reply)
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
+cat 
+Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
+--text follows this line--
+Notmuch Test Suite  writes:
+
+> Hi,
+> This is an HTML test message.
+>
+> OK?
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Quote MML tags in reply"
 message_id='test-emacs-mml-quot...@message.id'
 add_message [id]="$message_id" \
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 1/2] cli: make --entire-thread=false work for format=json.

[Adam Wolfe Gordon]

On Sat, Apr 21, 2012 at 03:15, Mark Walters  
wrote:
> The --entire-thread option in notmuch-show.c defaults to true when
> format=json. Previously there was no way to turn this off. This patch
> makes it respect --entire-thread=false.
>
> The one subtlety is that we initialise a notmuch_bool_t to -1 to
> indicate that the option parsing has not set it. This allows the code
> to distinguish between the option being omitted from the command line,
> and the option being set to false on the command line.
>
> Finally, all formats except Json can output empty messages for non
> entire-thread, but in Json format we need to output {} to keep the
> other elements (e.g. the replies to this message) in the correct
> place.

LGTM. You can probably remove the relevant item from devel/TODO as well!

-- Adam

Re: [PATCH v3 1/2] cli: make --entire-thread=false work for format=json.

[Adam Wolfe Gordon]

On Sat, Apr 21, 2012 at 03:15, Mark Walters  wrote:
> The --entire-thread option in notmuch-show.c defaults to true when
> format=json. Previously there was no way to turn this off. This patch
> makes it respect --entire-thread=false.
>
> The one subtlety is that we initialise a notmuch_bool_t to -1 to
> indicate that the option parsing has not set it. This allows the code
> to distinguish between the option being omitted from the command line,
> and the option being set to false on the command line.
>
> Finally, all formats except Json can output empty messages for non
> entire-thread, but in Json format we need to output {} to keep the
> other elements (e.g. the replies to this message) in the correct
> place.

LGTM. You can probably remove the relevant item from devel/TODO as well!

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: Add configurable wrapping width for notmuch-wash-wrap-long-lines

[Adam Wolfe Gordon]

On Wed, Apr 18, 2012 at 09:46, Daniel Schoepe  wrote:
> the reason for not making this a defcustom was that all of the other
> functions in notmuch-wash.el use defvar for customization too, so
> perhaps they should all be defcustom'd in a separate patch?

Ah, yes, I see now. Most of those variables (the regexps and
button-formats) probably don't need to be customizable - it seems
unlikely that anyone will want to change them except for
internationalization, which should probably be done differently
anyway. I think it makes sense for the others to be customs, so doing
that in a separate patch is a good idea.

Thanks again for this patch - it makes my mail read reading much more pleasant.

-- Adam

Re: [PATCH] emacs: Add configurable wrapping width for notmuch-wash-wrap-long-lines

[Adam Wolfe Gordon]

On Wed, Apr 18, 2012 at 09:46, Daniel Schoepe  wrote:
> the reason for not making this a defcustom was that all of the other
> functions in notmuch-wash.el use defvar for customization too, so
> perhaps they should all be defcustom'd in a separate patch?

Ah, yes, I see now. Most of those variables (the regexps and
button-formats) probably don't need to be customizable - it seems
unlikely that anyone will want to change them except for
internationalization, which should probably be done differently
anyway. I think it makes sense for the others to be customs, so doing
that in a separate patch is a good idea.

Thanks again for this patch - it makes my mail read reading much more pleasant.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 0/3] composing patches

[Adam Wolfe Gordon]

Hi Felipe,

On Wed, Apr 18, 2012 at 06:39, Felipe Contreras
 wrote:
> I don't know how it works in gnus, but at least on the vim mode, the output
> generated by 'notmuch reply' is not ready to be sent, at least the Message-ID
> field is needed, and also is nice to have the User-Agent.

In the emacs interface, the Message-ID header is generated when the
message is sent, so it never shows up in the reply buffer. The
User-Agent header is created by the emacs reply code.

> Besides, in order to avoid creating a new message by hand (possibly fetching
> the info from notmuch config), it's more straightforward to have 'notmuch
> compose' command.
>
> In the future 'notmuch compose' might be used to replace 'notmuch reply' and
> possily add a forward option too. It might also be possible to add mail 
> aliases
> when composing a message, and these aliases might be used while generating the
> 'notmuch search' output.

I can see how this functionality is useful, and I like the idea of
consolidating message creation, forwarding, and reply. However, if
this is intended to replace or share code with reply (and I think it
should), it should support all the existing reply formats: default,
json, and headers-only.

In fact, I think it might make more sense to make this work an
extension of the existing reply code rather than a rewrite.
notmuch-reply.c could become notmuch-compose.c, and provide the
compose command (with flags for creating a reply), and also the reply
command (which would just be an alias to the compose functionality).

> ?Makefile.local ? ?| ? ?1 +
> ?notmuch-client.h ?| ? ?3 ++
> ?notmuch-compose.c | ?111 
> +
> ?notmuch-reply.c ? | ? 17 
> ?notmuch.c ? ? ? ? | ? ?5 +++

Such a big new features should definitely have some tests.

-- Adam

[PATCH v2 2/3] reply: add message-id header

[Adam Wolfe Gordon]

On Wed, Apr 18, 2012 at 07:09, Jani Nikula  wrote:
> I do wonder whether the emacs ui / message mode will add a duplicate
> message-id rather than override this.

I think the emacs reply code will replace Message-ID and User-Agent
headers correctly, as it does with the From header when selecting a
different identity within emacs. But, this should be tested.

Ideally, the emacs tests should catch any problems here. As things are
right now, I think we would catch problems with the User-Agent header
in the reply tests: the tests replace the emacs and notmuch versions
with XXX but otherwise leave the header intact, so it should differ
from the one generated by notmuch compose.

The Message-ID is trickier, since in emacs it's generated when the
message is sent, not when the message is created, and thus the reply
tests don't see it at all. The "Sending a message via (fake) SMTP"
test could potentially catch problems, but currently the tests don't
try to predict the Message-ID emacs will give the message, so they
replace it with XXX.

-- Adam

Re: [PATCH v2 0/3] composing patches

[Adam Wolfe Gordon]

Hi Felipe,

On Wed, Apr 18, 2012 at 06:39, Felipe Contreras
 wrote:
> I don't know how it works in gnus, but at least on the vim mode, the output
> generated by 'notmuch reply' is not ready to be sent, at least the Message-ID
> field is needed, and also is nice to have the User-Agent.

In the emacs interface, the Message-ID header is generated when the
message is sent, so it never shows up in the reply buffer. The
User-Agent header is created by the emacs reply code.

> Besides, in order to avoid creating a new message by hand (possibly fetching
> the info from notmuch config), it's more straightforward to have 'notmuch
> compose' command.
>
> In the future 'notmuch compose' might be used to replace 'notmuch reply' and
> possily add a forward option too. It might also be possible to add mail 
> aliases
> when composing a message, and these aliases might be used while generating the
> 'notmuch search' output.

I can see how this functionality is useful, and I like the idea of
consolidating message creation, forwarding, and reply. However, if
this is intended to replace or share code with reply (and I think it
should), it should support all the existing reply formats: default,
json, and headers-only.

In fact, I think it might make more sense to make this work an
extension of the existing reply code rather than a rewrite.
notmuch-reply.c could become notmuch-compose.c, and provide the
compose command (with flags for creating a reply), and also the reply
command (which would just be an alias to the compose functionality).

>  Makefile.local    |    1 +
>  notmuch-client.h  |    3 ++
>  notmuch-compose.c |  111 
> +
>  notmuch-reply.c   |   17 
>  notmuch.c         |    5 +++

Such a big new features should definitely have some tests.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 2/3] reply: add message-id header

[Adam Wolfe Gordon]

On Wed, Apr 18, 2012 at 07:09, Jani Nikula  wrote:
> I do wonder whether the emacs ui / message mode will add a duplicate
> message-id rather than override this.

I think the emacs reply code will replace Message-ID and User-Agent
headers correctly, as it does with the From header when selecting a
different identity within emacs. But, this should be tested.

Ideally, the emacs tests should catch any problems here. As things are
right now, I think we would catch problems with the User-Agent header
in the reply tests: the tests replace the emacs and notmuch versions
with XXX but otherwise leave the header intact, so it should differ
from the one generated by notmuch compose.

The Message-ID is trickier, since in emacs it's generated when the
message is sent, not when the message is created, and thus the reply
tests don't see it at all. The "Sending a message via (fake) SMTP"
test could potentially catch problems, but currently the tests don't
try to predict the Message-ID emacs will give the message, so they
replace it with XXX.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: Add configurable wrapping width for notmuch-wash-wrap-long-lines

[Adam Wolfe Gordon]

Hi Daniel,

On Fri, Feb 17, 2012 at 11:34, Daniel Schoepe  wrote:
> This introduces a variable to control after how many characters a line
> is wrapped by notmuch-wash-wrap-long-lines (still wrapping at the
> window width if it is lower).

I really like this idea, and I'm using this patch now. I've been
missing this mechanism in notmuch.

> +(defvar notmuch-wash-wrap-lines-length nil
> + ?"Wrap line after at most this many characters.
> +
> +If this is nil, lines in messages will be wrapped to fit in the
> +current window. If this is a number, lines will be wrapped after
> +this many characters or at the window width (whichever one is
> +lower).")

This should probably be a defcustom, in the notmuch-show section.
Otherwise, LGTM.

-- Adam

Re: [PATCH] emacs: Add configurable wrapping width for notmuch-wash-wrap-long-lines

[Adam Wolfe Gordon]

Hi Daniel,

On Fri, Feb 17, 2012 at 11:34, Daniel Schoepe  wrote:
> This introduces a variable to control after how many characters a line
> is wrapped by notmuch-wash-wrap-long-lines (still wrapping at the
> window width if it is lower).

I really like this idea, and I'm using this patch now. I've been
missing this mechanism in notmuch.

> +(defvar notmuch-wash-wrap-lines-length nil
> +  "Wrap line after at most this many characters.
> +
> +If this is nil, lines in messages will be wrapped to fit in the
> +current window. If this is a number, lines will be wrapped after
> +this many characters or at the window width (whichever one is
> +lower).")

This should probably be a defcustom, in the notmuch-show section.
Otherwise, LGTM.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

incrontab?

[Adam Wolfe Gordon]

Hi David,

On Thu, Apr 12, 2012 at 02:25, David Belohrad  wrote:
> is somebody using incrontab to issue 'notmuch new'? I've tried but with
> only partial success. I have setup incrotab to run 'notmuch new' when
> something changes in my Maildir. However it is not
> reliable. E.g. sometimes it works out of the box, sometimes it seems
> that 'notmuch new' is simply not invoked at all even if I see in
> /var/log/mail.log, that a new mail was delivered correctly to the
> folder. Anyone really uses this setup?

I don't use incrontab, but I do use my own inotify-based script for
updating notmuch: https://gist.github.com/1952483 . I haven't had any
trouble with it.

> I have reverted back to crontab to issue 'notmuch new' every 5
> minutes. And frankly speaking, I'm rather thinking to run this command
> from emacs directly everytime I either start notmuch, or refresh view
> using '=' on notmuch-hello buffer.

You could probably do this with notmuch-hello-refresh-hook, but it
will be a bit tricky: the hook is executed after the notmuch-hello
buffer is refreshed, so you'd have to have it refresh after notmuch
new completes, without running the hook infinitely.

A better approach might be to use advice. Something like (completely untested):

(defadvice notmuch-hello-update (before notmuch-new) (call-process
"notmuch" nil nil nil "new"))

Hope that helps,
-- Adam

Re: incrontab?

[Adam Wolfe Gordon]

Hi David,

On Thu, Apr 12, 2012 at 02:25, David Belohrad  wrote:
> is somebody using incrontab to issue 'notmuch new'? I've tried but with
> only partial success. I have setup incrotab to run 'notmuch new' when
> something changes in my Maildir. However it is not
> reliable. E.g. sometimes it works out of the box, sometimes it seems
> that 'notmuch new' is simply not invoked at all even if I see in
> /var/log/mail.log, that a new mail was delivered correctly to the
> folder. Anyone really uses this setup?

I don't use incrontab, but I do use my own inotify-based script for
updating notmuch: https://gist.github.com/1952483 . I haven't had any
trouble with it.

> I have reverted back to crontab to issue 'notmuch new' every 5
> minutes. And frankly speaking, I'm rather thinking to run this command
> from emacs directly everytime I either start notmuch, or refresh view
> using '=' on notmuch-hello buffer.

You could probably do this with notmuch-hello-refresh-hook, but it
will be a bit tricky: the hook is executed after the notmuch-hello
buffer is refreshed, so you'd have to have it refresh after notmuch
new completes, without running the hook infinitely.

A better approach might be to use advice. Something like (completely untested):

(defadvice notmuch-hello-update (before notmuch-new) (call-process
"notmuch" nil nil nil "new"))

Hope that helps,
-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/2] cli: make --entire-thread=false work for format=json.

[Adam Wolfe Gordon]

Hi Mark,

This looks good to me, but I haven't tested it. It's probably worth
adding a test for the new functionality.

One style issue below, which is a matter of taste and I'll defer to
others if they disagree:

On Tue, Apr 10, 2012 at 11:04, Mark Walters  
wrote:
> @@ -895,10 +905,11 @@ show_messages (void *ctx,
> ? ? ? ? ? ?if (status && !res)
> ? ? ? ? ? ? ? ?res = status;
> ? ? ? ? ? ?next_indent = indent + 1;
> + ? ? ? } else
> + ? ? ? ? ? status = show_null_message (format);

I accept, but don't particularly like, the notmuch style of omitting
braces where they aren't required. However, an else with a brace on
only one side just looks weird. If they're like this everywhere else
then I guess it's best to be consistent, but to me it's a lot more
readable as } else {.

As I said above, I'll defer to others' judgement here, just thought it
was worth pointing out.

-- Adam

Re: [PATCH 1/2] cli: make --entire-thread=false work for format=json.

[Adam Wolfe Gordon]

Hi Mark,

This looks good to me, but I haven't tested it. It's probably worth
adding a test for the new functionality.

One style issue below, which is a matter of taste and I'll defer to
others if they disagree:

On Tue, Apr 10, 2012 at 11:04, Mark Walters  wrote:
> @@ -895,10 +905,11 @@ show_messages (void *ctx,
>            if (status && !res)
>                res = status;
>            next_indent = indent + 1;
> +       } else
> +           status = show_null_message (format);

I accept, but don't particularly like, the notmuch style of omitting
braces where they aren't required. However, an else with a brace on
only one side just looks weird. If they're like this everywhere else
then I guess it's best to be consistent, but to me it's a lot more
readable as } else {.

As I said above, I'll defer to others' judgement here, just thought it
was worth pointing out.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: get rid of trailing spaces in notmuch-hello view

[Adam Wolfe Gordon]

On Fri, Mar 9, 2012 at 20:54, Dmitry Kurochkin
 wrote:
> This patch removes trailing spaces in notmuch-hello view.
>
> A side effect of this change is that tag/query buttons no longer
> include a space at the end. ?This means that pressing RET when the
> point is at the first character after the tag/query button no longer
> works (note that this is the standard behavior for buttons). ?We may
> change this behavior in the future (without adding trailing spaces
> back) if people would find this change inconvenient.

LGTM, and works as expected.

(Sorry if you get this twice, Dmitry.)

Re: [PATCH] emacs: get rid of trailing spaces in notmuch-hello view

[Adam Wolfe Gordon]

On Fri, Mar 9, 2012 at 20:54, Dmitry Kurochkin
 wrote:
> This patch removes trailing spaces in notmuch-hello view.
>
> A side effect of this change is that tag/query buttons no longer
> include a space at the end.  This means that pressing RET when the
> point is at the first character after the tag/query button no longer
> works (note that this is the standard behavior for buttons).  We may
> change this behavior in the future (without adding trailing spaces
> back) if people would find this change inconvenient.

LGTM, and works as expected.

(Sorry if you get this twice, Dmitry.)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/4] dirent->d_type not available on Soalris

[Adam Wolfe Gordon]

On Mon, Apr 9, 2012 at 09:12, Vladimir Marek  
wrote:
> Fair enough. Is there some performance test suite? Another way would be
> to make this compile time option set by configure. Used only when the
> system in question does not have dirent->d_type member.

I like the idea of making it configurable. From the Linux readdir(3) man page:

> Under glibc, programs can check  for  the  availability
> of  the  fields  not defined in POSIX.1 by testing whether
> the macros _DIRENT_HAVE_D_NAMLEN,
> _DIRENT_HAVE_D_RECLEN, _DIRENT_HAVE_D_OFF,
> or  _DIRENT_HAVE_D_TYPE  are defined.

I read this as meaning that we can just test for _DIRENT_HAVE_D_TYPE
instead of adding our own define.

I expect the stat system calls will provide some overhead, and there's
no real need to pay that overhead on systems where it isn't necessary.

-- Adam

Re: [PATCH 2/4] dirent->d_type not available on Soalris

[Adam Wolfe Gordon]

On Mon, Apr 9, 2012 at 09:12, Vladimir Marek  wrote:
> Fair enough. Is there some performance test suite? Another way would be
> to make this compile time option set by configure. Used only when the
> system in question does not have dirent->d_type member.

I like the idea of making it configurable. From the Linux readdir(3) man page:

> Under glibc, programs can check  for  the  availability
> of  the  fields  not defined in POSIX.1 by testing whether
> the macros _DIRENT_HAVE_D_NAMLEN,
> _DIRENT_HAVE_D_RECLEN, _DIRENT_HAVE_D_OFF,
> or  _DIRENT_HAVE_D_TYPE  are defined.

I read this as meaning that we can just test for _DIRENT_HAVE_D_TYPE
instead of adding our own define.

I expect the stat system calls will provide some overhead, and there's
no real need to pay that overhead on systems where it isn't necessary.

-- Adam
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Why not tags inside messages?

[Adam Wolfe Gordon]

Hi David,

On Sat, Apr 7, 2012 at 10:46, David Belohrad  wrote:
> I'd love to use notmuch with offline imap ?to work rather on local copy of
> messages, than using remote notmuch, which is slightly slower due to
> bandwidth limitation of my vdsl line. There is however fundamental problem
> of syncing flags between two instances of notmuch. So my question is,
> whether it would be possible, when tagging message, to store the tag as well
> in the message itself. Might this help to sync across multiple instances?

One solution that various people have used (I've used it for backup)
is to make the .notmuch directory in the maildir into a git
repository, and sync it to various machines using git push/git pull.
You can do the push/pull manually or in a notmuch hook to keep things
synced. I'm not sure if this is any better for your purposes than
using remote notmuch, but it would keep things local most of the time.

> Another thing is, that I'd love to have native android app working over
> remote notmuch to work as my email agent. Is there anything like this? I
> guess not...

I started working on one once, using python to write a little notmuch
server that produced JSON for various things, and an Android app that
contacted the server and displayed things nicely. It only got as far
as being able to list tags and count messages. It's probably something
I'll pick up again at some point, and of course I'd be happy to share
code if people are willing to help.

I know another Android client has been mentioned on IRC, but I'm not
sure if it got any further than mine.

Re: Why not tags inside messages?

[Adam Wolfe Gordon]

Hi David,

On Sat, Apr 7, 2012 at 10:46, David Belohrad  wrote:
> I'd love to use notmuch with offline imap  to work rather on local copy of
> messages, than using remote notmuch, which is slightly slower due to
> bandwidth limitation of my vdsl line. There is however fundamental problem
> of syncing flags between two instances of notmuch. So my question is,
> whether it would be possible, when tagging message, to store the tag as well
> in the message itself. Might this help to sync across multiple instances?

One solution that various people have used (I've used it for backup)
is to make the .notmuch directory in the maildir into a git
repository, and sync it to various machines using git push/git pull.
You can do the push/pull manually or in a notmuch hook to keep things
synced. I'm not sure if this is any better for your purposes than
using remote notmuch, but it would keep things local most of the time.

> Another thing is, that I'd love to have native android app working over
> remote notmuch to work as my email agent. Is there anything like this? I
> guess not...

I started working on one once, using python to write a little notmuch
server that produced JSON for various things, and an Android app that
contacted the server and displayed things nicely. It only got as far
as being able to list tags and count messages. It's probably something
I'll pick up again at some point, and of course I'd be happy to share
code if people are willing to help.

I know another Android client has been mentioned on IRC, but I'm not
sure if it got any further than mine.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch-poll and OfflineIMAP

[Adam Wolfe Gordon]

Hi Jacek,

On Tue, Apr 3, 2012 at 09:27, Jacek Generowicz  
wrote:
> Would you have any advice on how to construct a notmuch-poll script
> that would work well in concert with OfflineIMAP? ?In particlular, how
> can you avoid having to re-issue the IMAP account passwords?
>
> The OfflineIMAP docs state that SIGUSR1 forces an immediate resync of
> all accounts, so something based around
>
> ? ?kill -SIGUSR1 `cat ~/.offlineimap/pid`
>
> or
>
> ? ?pkill -SIGUSR1 -u `whoami` offlineimap
>
> (with an already-running OfflineIMAP process in autorefresh mode)
> could work, but it's not clear to me how to discover when the resync
> is done, and sleeping for an arbitrary time before calling 'notmuch
> new' seems unsatisfactory.
>
> Any words of wisdom?

I run offlineimap in autorefresh mode (with IDLE too, actually) and
use a script based on inotify to update notmuch whenever new mail
comes in: https://gist.github.com/1952483 . It sounds like some
variant of this would probably work for you.

As an added bonus, I have my post-new script run emacsclient -n -e
'(notmuch-hello-update t)', so if I flip to my mail desktop I can see
right away if there's new mail.

Re: notmuch-poll and OfflineIMAP

[Adam Wolfe Gordon]

Hi Jacek,

On Tue, Apr 3, 2012 at 09:27, Jacek Generowicz  wrote:
> Would you have any advice on how to construct a notmuch-poll script
> that would work well in concert with OfflineIMAP?  In particlular, how
> can you avoid having to re-issue the IMAP account passwords?
>
> The OfflineIMAP docs state that SIGUSR1 forces an immediate resync of
> all accounts, so something based around
>
>    kill -SIGUSR1 `cat ~/.offlineimap/pid`
>
> or
>
>    pkill -SIGUSR1 -u `whoami` offlineimap
>
> (with an already-running OfflineIMAP process in autorefresh mode)
> could work, but it's not clear to me how to discover when the resync
> is done, and sleeping for an arbitrary time before calling 'notmuch
> new' seems unsatisfactory.
>
> Any words of wisdom?

I run offlineimap in autorefresh mode (with IDLE too, actually) and
use a script based on inotify to update notmuch whenever new mail
comes in: https://gist.github.com/1952483 . It sounds like some
variant of this would probably work for you.

As an added bonus, I have my post-new script run emacsclient -n -e
'(notmuch-hello-update t)', so if I flip to my mail desktop I can see
right away if there's new mail.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[BUG/PATCH v4 4/4] emacs: Fix the References header in reply

[Adam Wolfe Gordon]

In the new reply code, the References header gets inserted by
message.el using a function called message-shorten-references. Unlike
all the other header-inserting functions, it doesn't put a newline
after the header, causing the next header to end up on the same
line. In our case, this header happened to be User-Agent, so it's hard
to notice. This is probably a bug in message.el, but we need to work
around it.

This fixes the problem by wrapping message-shorten-references in a
function that inserts a newline after if necessary. This should
protect against the message.el bug being fixed in the future.
---
 emacs/notmuch-mua.el |   28 +---
 test/emacs   |6 --
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index cfa3d61..9f279d9 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -90,6 +90,15 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))

+;; There is a bug in emacs 23's message.el that results in a newline
+;; not being inserted after the References header, so the next header
+;; is concatenated to the end of it. This function fixes the problem,
+;; while guarding against the possibility that some current or future
+;; version of emacs has the bug fixed.
+(defun notmuch-mua-insert-references (original-func header references)
+  (funcall original-func header references)
+  (unless (bolp) (insert "\n")))
+
 (defun notmuch-mua-reply (query-string &optional sender reply-all)
   (let ((args '("reply" "--format=json"))
reply
@@ -125,9 +134,22 @@ list."
  ;; Overlay the composition window on that being used to read
  ;; the original message.
  ((same-window-regexps '("\\*mail .*")))
-   (notmuch-mua-mail (plist-get reply-headers :To)
- (plist-get reply-headers :Subject)
- (notmuch-headers-plist-to-alist reply-headers)))
+
+   ;; We modify message-header-format-alist to get around a bug in 
message.el.
+   ;; See the comment above on notmuch-mua-insert-references.
+   (let ((message-header-format-alist
+  (loop for pair in message-header-format-alist
+if (eq (car pair) 'References)
+collect (cons 'References 
+  (apply-partially
+   'notmuch-mua-insert-references
+   (cdr pair)))
+else
+collect pair)))
+ (notmuch-mua-mail (plist-get reply-headers :To)
+   (plist-get reply-headers :Subject)
+   (notmuch-headers-plist-to-alist reply-headers
+
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
diff --git a/test/emacs b/test/emacs
index 15cc778..c7510e9 100755
--- a/test/emacs
+++ b/test/emacs
@@ -267,7 +267,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
@@ -292,7 +291,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_other at notmuchmail.org

@@ -318,7 +316,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from address in named group list within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
  [cc]=test_suite_other at notmuchmail.org
@@ -345,7 +342,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
@@ -409,7 +405,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
(notmuch-show-reply)
@@ -450,7 +445,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Quote MML tags in reply"
-test_subtest_known_broken
 message_id='test-emacs-mml-quoting at message.id'
 add_message [id]="$message_id" \
"[subject]='$test_subtest_name'" \
-- 
1.7.5.4

[BUG/PATCH v4 3/4] test: Show all headers in emacs reply tests

[Adam Wolfe Gordon]

By default, emacs hides the User-Agent and References headers when
composing mail. This is a good thing for users, but a bad thing for
testing, since we can create ugly or invalid headers and not have it
show up in the tests.

By setting message-hidden-headers to an empty list, we force emacs to
show all the headers, so we can check that they're correct. Users
won't see this, but it will let us catch future bugs.

As a side-effect, this breaks all the reply tests, since there is a
bug with the References and User-Agent headers, fixed in the next commit.
---
 test/emacs |   55 +++
 1 files changed, 43 insertions(+), 12 deletions(-)

diff --git a/test/emacs b/test/emacs
index 30654bb..15cc778 100755
--- a/test/emacs
+++ b/test/emacs
@@ -267,17 +267,23 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs"
-test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)'
+   (test-output))'
 sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: /' OUTPUT
+sed -i -e 's/^References: <.*>$/References: /' OUTPUT
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:

@@ -307,20 +318,25 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from address in named group list within emacs"
+test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
  [cc]=test_suite_other at notmuchmail.org

-test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)"
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:

@@ -329,15 +345,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_emacs '(notmuch-show "id:20091118002059.067214ed at hikari")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
-   (test-output)'
+   (test-output))'
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <20091118002059.067214ed at hikari>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Adrian Perez de Castro  writes:

@@ -388,15 +409,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_emacs '(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
(notmuch-show-reply)
-   (test-output)'
+ 

[BUG/PATCH v4 2/4] emacs: Fix two bugs in reply

[Adam Wolfe Gordon]

Bug 1: Replying from alternate addresses


The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
>From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.

This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.

This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.

Bug 2: Duplicate headers in emacs 23.2
--

The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.

Converting everything to symbols also fixes this issue.
---
 emacs/notmuch-lib.el |7 +--
 emacs/notmuch-mua.el |   10 +-
 test/emacs   |1 -
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index c159dda..6907a5f 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -237,9 +237,12 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))

-(defun notmuch-plist-to-alist (plist)
+;; Converts a plist of headers to an alist of headers. The input plist should
+;; have symbols of the form :Header as keys, and the resulting alist will have
+;; symbols of the form 'Header as keys.
+(defun notmuch-headers-plist-to-alist (plist)
   (loop for (key value . rest) on plist by #'cddr
-   collect (cons (substring (symbol-name key) 1) value)))
+   collect (cons (intern (substring (symbol-name key) 1)) value)))

 ;; Compatibility functions for versions of emacs before emacs 23.
 ;;
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 6aae3a0..cfa3d61 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -127,7 +127,7 @@ list."
  ((same-window-regexps '("\\*mail .*")))
(notmuch-mua-mail (plist-get reply-headers :To)
  (plist-get reply-headers :Subject)
- (notmuch-plist-to-alist reply-headers)))
+ (notmuch-headers-plist-to-alist reply-headers)))
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
@@ -185,11 +185,11 @@ OTHER-ARGS are passed through to `message-mail'."
   (when notmuch-mua-user-agent-function
 (let ((user-agent (funcall notmuch-mua-user-agent-function)))
   (when (not (string= "" user-agent))
-   (push (cons "User-Agent" user-agent) other-headers
+   (push (cons 'User-Agent user-agent) other-headers

-  (unless (mail-header 'From other-headers)
-(push (cons "From" (concat
-   (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))
+  (unless (assq 'From other-headers)
+(push (cons 'From (concat
+  (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))

   (apply #'message-mail to subject other-headers other-args)
   (message-sort-headers)
diff --git a/test/emacs b/test/emacs
index 576bc1f..30654bb 100755
--- a/test/emacs
+++ b/test/emacs
@@ -286,7 +286,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_other at notmuchmail.org

-- 
1.7.5.4

[BUG/PATCH v4 1/4] test: Tests for reply from alternate addresses in emacs

[Adam Wolfe Gordon]

Since the recent reply changes were pushed, there has been a bug that
causes emacs to always reply from the primary address, even if the
JSON or default CLI reply output uses an alternate address.

This adds two tests to the emacs test library based on the two "Reply
form..." tests in the reply test library. One is currently marked
broken.
---
 test/emacs |   44 
 1 files changed, 44 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 8b92d0a..576bc1f 100755
--- a/test/emacs
+++ b/test/emacs
@@ -285,6 +285,50 @@ Notmuch Test Suite  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED

+test_begin_subtest "Reply from alternate address within emacs"
+test_subtest_known_broken
+add_message '[from]="Sender "' \
+[to]=test_suite_other at notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Reply from address in named group list within emacs"
+add_message '[from]="Sender "' \
+'[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
+ [cc]=test_suite_other at notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
 test_emacs '(notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
-- 
1.7.5.4

[BUG/PATCH v4 0/4] Bug fixes for reply, rebased

[Adam Wolfe Gordon]

This series is the same as the one from yesterday [1], rebased onto the
current master to resolve some conflicts with Austin's chnages.

[1] id:"1333240404-13076-1-git-send-email-awg+notmuch at xvx.ca"

Adam Wolfe Gordon (4):
  test: Tests for reply from alternate addresses in emacs
  emacs: Fix two bugs in reply
  test: Show all headers in emacs reply tests
  emacs: Fix the References header in reply

 emacs/notmuch-lib.el |7 +++-
 emacs/notmuch-mua.el |   36 +
 test/emacs   |   84 +-
 3 files changed, 110 insertions(+), 17 deletions(-)

-- 
1.7.5.4

[BUG/PATCH v4 1/4] test: Tests for reply from alternate addresses in emacs

[Adam Wolfe Gordon]

Since the recent reply changes were pushed, there has been a bug that
causes emacs to always reply from the primary address, even if the
JSON or default CLI reply output uses an alternate address.

This adds two tests to the emacs test library based on the two "Reply
form..." tests in the reply test library. One is currently marked
broken.
---
 test/emacs |   44 
 1 files changed, 44 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 8b92d0a..576bc1f 100755
--- a/test/emacs
+++ b/test/emacs
@@ -285,6 +285,50 @@ Notmuch Test Suite  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "Reply from alternate address within emacs"
+test_subtest_known_broken
+add_message '[from]="Sender "' \
+[to]=test_suite_ot...@notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Reply from address in named group list within emacs"
+add_message '[from]="Sender "' \
+'[to]=group:test_su...@notmuchmail.org,some...@example.com\;' \
+ [cc]=test_suite_ot...@notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
 test_emacs '(notmuch-show "id:20091118002059.067214ed@hikari")
(notmuch-show-reply)
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[BUG/PATCH v4 4/4] emacs: Fix the References header in reply

[Adam Wolfe Gordon]

In the new reply code, the References header gets inserted by
message.el using a function called message-shorten-references. Unlike
all the other header-inserting functions, it doesn't put a newline
after the header, causing the next header to end up on the same
line. In our case, this header happened to be User-Agent, so it's hard
to notice. This is probably a bug in message.el, but we need to work
around it.

This fixes the problem by wrapping message-shorten-references in a
function that inserts a newline after if necessary. This should
protect against the message.el bug being fixed in the future.
---
 emacs/notmuch-mua.el |   28 +---
 test/emacs   |6 --
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index cfa3d61..9f279d9 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -90,6 +90,15 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))
 
+;; There is a bug in emacs 23's message.el that results in a newline
+;; not being inserted after the References header, so the next header
+;; is concatenated to the end of it. This function fixes the problem,
+;; while guarding against the possibility that some current or future
+;; version of emacs has the bug fixed.
+(defun notmuch-mua-insert-references (original-func header references)
+  (funcall original-func header references)
+  (unless (bolp) (insert "\n")))
+
 (defun notmuch-mua-reply (query-string &optional sender reply-all)
   (let ((args '("reply" "--format=json"))
reply
@@ -125,9 +134,22 @@ list."
  ;; Overlay the composition window on that being used to read
  ;; the original message.
  ((same-window-regexps '("\\*mail .*")))
-   (notmuch-mua-mail (plist-get reply-headers :To)
- (plist-get reply-headers :Subject)
- (notmuch-headers-plist-to-alist reply-headers)))
+
+   ;; We modify message-header-format-alist to get around a bug in 
message.el.
+   ;; See the comment above on notmuch-mua-insert-references.
+   (let ((message-header-format-alist
+  (loop for pair in message-header-format-alist
+if (eq (car pair) 'References)
+collect (cons 'References 
+  (apply-partially
+   'notmuch-mua-insert-references
+   (cdr pair)))
+else
+collect pair)))
+ (notmuch-mua-mail (plist-get reply-headers :To)
+   (plist-get reply-headers :Subject)
+   (notmuch-headers-plist-to-alist reply-headers
+
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
diff --git a/test/emacs b/test/emacs
index 15cc778..c7510e9 100755
--- a/test/emacs
+++ b/test/emacs
@@ -267,7 +267,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
@@ -292,7 +291,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_ot...@notmuchmail.org
 
@@ -318,7 +316,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply from address in named group list within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_su...@notmuchmail.org,some...@example.com\;' \
  [cc]=test_suite_ot...@notmuchmail.org
@@ -345,7 +342,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show "id:20091118002059.067214ed@hikari")
(notmuch-show-reply)
@@ -409,7 +405,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show 
"id:cf0c4d610911171136h1713aa59w9cf9aa31f052a...@mail.gmail.com")
(notmuch-show-reply)
@@ -450,7 +445,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Quote MML tags in reply"
-test_subtest_known_broken
 message_id='test-emacs-mml-quot...@message.id'
 add_message [id]="$message_id" \
"[subject]='$test_subtest_name'" \
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[BUG/PATCH v4 3/4] test: Show all headers in emacs reply tests

[Adam Wolfe Gordon]

By default, emacs hides the User-Agent and References headers when
composing mail. This is a good thing for users, but a bad thing for
testing, since we can create ugly or invalid headers and not have it
show up in the tests.

By setting message-hidden-headers to an empty list, we force emacs to
show all the headers, so we can check that they're correct. Users
won't see this, but it will let us catch future bugs.

As a side-effect, this breaks all the reply tests, since there is a
bug with the References and User-Agent headers, fixed in the next commit.
---
 test/emacs |   55 +++
 1 files changed, 43 insertions(+), 12 deletions(-)

diff --git a/test/emacs b/test/emacs
index 30654bb..15cc778 100755
--- a/test/emacs
+++ b/test/emacs
@@ -267,17 +267,23 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs"
-test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)'
+   (test-output))'
 sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: /' OUTPUT
+sed -i -e 's/^References: <.*>$/References: /' OUTPUT
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:
 
@@ -307,20 +318,25 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply from address in named group list within emacs"
+test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_su...@notmuchmail.org,some...@example.com\;' \
  [cc]=test_suite_ot...@notmuchmail.org
 
-test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)"
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:
 
@@ -329,15 +345,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_emacs '(notmuch-show "id:20091118002059.067214ed@hikari")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show "id:20091118002059.067214ed@hikari")
(notmuch-show-reply)
-   (test-output)'
+   (test-output))'
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <20091118002059.067214ed@hikari>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Adrian Perez de Castro  writes:
 
@@ -388,15 +409,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_emacs '(notmuch-show 
"id:cf0c4d610911171136h1713aa59w9cf9aa31f052a...@mail.gmail.com")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show 
"id:cf0c4d610911171136h1713aa59w9cf9aa31f052a...@mail.gmail.com")
(notmuch-show-reply)
-   (test-output)'
+   (test-output))'
+se

[BUG/PATCH v4 2/4] emacs: Fix two bugs in reply

[Adam Wolfe Gordon]

Bug 1: Replying from alternate addresses


The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
>From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.

This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.

This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.

Bug 2: Duplicate headers in emacs 23.2
--

The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.

Converting everything to symbols also fixes this issue.
---
 emacs/notmuch-lib.el |7 +--
 emacs/notmuch-mua.el |   10 +-
 test/emacs   |1 -
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index c159dda..6907a5f 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -237,9 +237,12 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (notmuch-id-to-query (plist-get msg :id)) 
nth process-crypto)))
 
-(defun notmuch-plist-to-alist (plist)
+;; Converts a plist of headers to an alist of headers. The input plist should
+;; have symbols of the form :Header as keys, and the resulting alist will have
+;; symbols of the form 'Header as keys.
+(defun notmuch-headers-plist-to-alist (plist)
   (loop for (key value . rest) on plist by #'cddr
-   collect (cons (substring (symbol-name key) 1) value)))
+   collect (cons (intern (substring (symbol-name key) 1)) value)))
 
 ;; Compatibility functions for versions of emacs before emacs 23.
 ;;
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 6aae3a0..cfa3d61 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -127,7 +127,7 @@ list."
  ((same-window-regexps '("\\*mail .*")))
(notmuch-mua-mail (plist-get reply-headers :To)
  (plist-get reply-headers :Subject)
- (notmuch-plist-to-alist reply-headers)))
+ (notmuch-headers-plist-to-alist reply-headers)))
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
@@ -185,11 +185,11 @@ OTHER-ARGS are passed through to `message-mail'."
   (when notmuch-mua-user-agent-function
 (let ((user-agent (funcall notmuch-mua-user-agent-function)))
   (when (not (string= "" user-agent))
-   (push (cons "User-Agent" user-agent) other-headers
+   (push (cons 'User-Agent user-agent) other-headers
 
-  (unless (mail-header 'From other-headers)
-(push (cons "From" (concat
-   (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))
+  (unless (assq 'From other-headers)
+(push (cons 'From (concat
+  (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))
 
   (apply #'message-mail to subject other-headers other-args)
   (message-sort-headers)
diff --git a/test/emacs b/test/emacs
index 576bc1f..30654bb 100755
--- a/test/emacs
+++ b/test/emacs
@@ -286,7 +286,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_ot...@notmuchmail.org
 
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[BUG/PATCH v4 0/4] Bug fixes for reply, rebased

[Adam Wolfe Gordon]

This series is the same as the one from yesterday [1], rebased onto the
current master to resolve some conflicts with Austin's chnages.

[1] id:"1333240404-13076-1-git-send-email-awg+notm...@xvx.ca"

Adam Wolfe Gordon (4):
  test: Tests for reply from alternate addresses in emacs
  emacs: Fix two bugs in reply
  test: Show all headers in emacs reply tests
  emacs: Fix the References header in reply

 emacs/notmuch-lib.el |7 +++-
 emacs/notmuch-mua.el |   36 +
 test/emacs   |   84 +-
 3 files changed, 110 insertions(+), 17 deletions(-)

-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[BUG/PATCH v3 4/4] emacs: Fix the References header in reply

[Adam Wolfe Gordon]

In the new reply code, the References header gets inserted by
message.el using a function called message-shorten-references. Unlike
all the other header-inserting functions, it doesn't put a newline
after the header, causing the next header to end up on the same
line. In our case, this header happened to be User-Agent, so it's hard
to notice. This is probably a bug in message.el, but we need to work
around it.

This fixes the problem by wrapping message-shorten-references in a
function that inserts a newline after if necessary. This should
protect against the message.el bug being fixed in the future.
---
 emacs/notmuch-mua.el |   28 +---
 test/emacs   |6 --
 2 files changed, 25 insertions(+), 9 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index cfa3d61..9f279d9 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -90,6 +90,15 @@ list."
else if (notmuch-match-content-type (plist-get part :content-type) 
"text/*")
  collect part))

+;; There is a bug in emacs 23's message.el that results in a newline
+;; not being inserted after the References header, so the next header
+;; is concatenated to the end of it. This function fixes the problem,
+;; while guarding against the possibility that some current or future
+;; version of emacs has the bug fixed.
+(defun notmuch-mua-insert-references (original-func header references)
+  (funcall original-func header references)
+  (unless (bolp) (insert "\n")))
+
 (defun notmuch-mua-reply (query-string &optional sender reply-all)
   (let ((args '("reply" "--format=json"))
reply
@@ -125,9 +134,22 @@ list."
  ;; Overlay the composition window on that being used to read
  ;; the original message.
  ((same-window-regexps '("\\*mail .*")))
-   (notmuch-mua-mail (plist-get reply-headers :To)
- (plist-get reply-headers :Subject)
- (notmuch-headers-plist-to-alist reply-headers)))
+
+   ;; We modify message-header-format-alist to get around a bug in 
message.el.
+   ;; See the comment above on notmuch-mua-insert-references.
+   (let ((message-header-format-alist
+  (loop for pair in message-header-format-alist
+if (eq (car pair) 'References)
+collect (cons 'References 
+  (apply-partially
+   'notmuch-mua-insert-references
+   (cdr pair)))
+else
+collect pair)))
+ (notmuch-mua-mail (plist-get reply-headers :To)
+   (plist-get reply-headers :Subject)
+   (notmuch-headers-plist-to-alist reply-headers
+
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
diff --git a/test/emacs b/test/emacs
index 7d00cae..e319f83 100755
--- a/test/emacs
+++ b/test/emacs
@@ -256,7 +256,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
@@ -281,7 +280,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_other at notmuchmail.org

@@ -307,7 +305,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from address in named group list within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
  [cc]=test_suite_other at notmuchmail.org
@@ -334,7 +331,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
@@ -398,7 +394,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_subtest_known_broken
 test_emacs '(let ((message-hidden-headers ''()))
(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
(notmuch-show-reply)
@@ -439,7 +434,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Quote MML tags in reply"
-test_subtest_known_broken
 message_id='test-emacs-mml-quoting at message.id'
 add_message [id]="$message_id" \
"[subject]='$test_subtest_name'" \
-- 
1.7.5.4

[BUG/PATCH v3 3/4] test: Show all headers in emacs reply tests

[Adam Wolfe Gordon]

By default, emacs hides the User-Agent and References headers when
composing mail. This is a good thing for users, but a bad thing for
testing, since we can create ugly or invalid headers and not have it
show up in the tests.

By setting message-hidden-headers to an empty list, we force emacs to
show all the headers, so we can check that they're correct. Users
won't see this, but it will let us catch future bugs.

As a side-effect, this breaks all the reply tests, since there is a
bug with the References and User-Agent headers, fixed in the next commit.
---
 test/emacs |   55 +++
 1 files changed, 43 insertions(+), 12 deletions(-)

diff --git a/test/emacs b/test/emacs
index 06291d3..7d00cae 100755
--- a/test/emacs
+++ b/test/emacs
@@ -256,17 +256,23 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs"
-test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-search "subject:\"testing message sent via SMTP\"")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)'
+   (test-output))'
 sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: /' OUTPUT
+sed -i -e 's/^References: <.*>$/References: /' OUTPUT
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:

@@ -296,20 +307,25 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from address in named group list within emacs"
+test_subtest_known_broken
 add_message '[from]="Sender "' \
 '[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
  [cc]=test_suite_other at notmuchmail.org

-test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+test_emacs "(let ((message-hidden-headers '()))
+   (notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
(notmuch-test-wait)
(notmuch-search-reply-to-thread)
-   (test-output)"
+   (test-output))"
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <${gen_msg_id}>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Sender  writes:

@@ -318,15 +334,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_emacs '(notmuch-show "id:20091118002059.067214ed at hikari")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
-   (test-output)'
+   (test-output))'
+sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat 
 Fcc: ${MAIL_DIR}/sent
+References: <20091118002059.067214ed at hikari>
+User-Agent: Notmuch/XXX Emacs/XXX
 --text follows this line--
 Adrian Perez de Castro  writes:

@@ -377,15 +398,20 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/alternative message"
-test_emacs '(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
+test_subtest_known_broken
+test_emacs '(let ((message-hidden-headers ''()))
+   (notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a at 
mail.gmail.com")
(notmuch-show-reply)
-   (test-output)'
+ 

[BUG/PATCH v3 2/4] emacs: Fix two bugs in reply

[Adam Wolfe Gordon]

Bug 1: Replying from alternate addresses


The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
>From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.

This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.

This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.

Bug 2: Duplicate headers in emacs 23.2
--

The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.

Converting everything to symbols also fixes this issue.
---
 emacs/notmuch-lib.el |7 +--
 emacs/notmuch-mua.el |   10 +-
 test/emacs   |1 -
 3 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index c146748..0effe24 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -232,9 +232,12 @@ the given type."
   (or (plist-get part :content)
   (notmuch-get-bodypart-internal (concat "id:" (plist-get msg :id)) nth 
process-crypto)))

-(defun notmuch-plist-to-alist (plist)
+;; Converts a plist of headers to an alist of headers. The input plist should
+;; have symbols of the form :Header as keys, and the resulting alist will have
+;; symbols of the form 'Header as keys.
+(defun notmuch-headers-plist-to-alist (plist)
   (loop for (key value . rest) on plist by #'cddr
-   collect (cons (substring (symbol-name key) 1) value)))
+   collect (cons (intern (substring (symbol-name key) 1)) value)))

 ;; Compatibility functions for versions of emacs before emacs 23.
 ;;
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 6aae3a0..cfa3d61 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -127,7 +127,7 @@ list."
  ((same-window-regexps '("\\*mail .*")))
(notmuch-mua-mail (plist-get reply-headers :To)
  (plist-get reply-headers :Subject)
- (notmuch-plist-to-alist reply-headers)))
+ (notmuch-headers-plist-to-alist reply-headers)))
   ;; Insert the message body - but put it in front of the signature
   ;; if one is present
   (goto-char (point-max))
@@ -185,11 +185,11 @@ OTHER-ARGS are passed through to `message-mail'."
   (when notmuch-mua-user-agent-function
 (let ((user-agent (funcall notmuch-mua-user-agent-function)))
   (when (not (string= "" user-agent))
-   (push (cons "User-Agent" user-agent) other-headers
+   (push (cons 'User-Agent user-agent) other-headers

-  (unless (mail-header 'From other-headers)
-(push (cons "From" (concat
-   (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))
+  (unless (assq 'From other-headers)
+(push (cons 'From (concat
+  (notmuch-user-name) " <" (notmuch-user-primary-email) 
">")) other-headers))

   (apply #'message-mail to subject other-headers other-args)
   (message-sort-headers)
diff --git a/test/emacs b/test/emacs
index 3402efb..06291d3 100755
--- a/test/emacs
+++ b/test/emacs
@@ -275,7 +275,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply from alternate address within emacs"
-test_subtest_known_broken
 add_message '[from]="Sender "' \
 [to]=test_suite_other at notmuchmail.org

-- 
1.7.5.4

[BUG/PATCH v3 1/4] test: Tests for reply from alternate addresses in emacs

[Adam Wolfe Gordon]

Since the recent reply changes were pushed, there has been a bug that
causes emacs to always reply from the primary address, even if the
JSON or default CLI reply output uses an alternate address.

This adds two tests to the emacs test library based on the two "Reply
form..." tests in the reply test library. One is currently marked
broken.
---
 test/emacs |   44 
 1 files changed, 44 insertions(+), 0 deletions(-)

diff --git a/test/emacs b/test/emacs
index 8a28705..3402efb 100755
--- a/test/emacs
+++ b/test/emacs
@@ -274,6 +274,50 @@ Notmuch Test Suite  writes:
 EOF
 test_expect_equal_file OUTPUT EXPECTED

+test_begin_subtest "Reply from alternate address within emacs"
+test_subtest_known_broken
+add_message '[from]="Sender "' \
+[to]=test_suite_other at notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Reply from address in named group list within emacs"
+add_message '[from]="Sender "' \
+'[to]=group:test_suite at notmuchmail.org,someone at 
example.com\;' \
+ [cc]=test_suite_other at notmuchmail.org
+
+test_emacs "(notmuch-search \"id:\\\"${gen_msg_id}\\\"\")
+   (notmuch-test-wait)
+   (notmuch-search-reply-to-thread)
+   (test-output)"
+cat 
+Fcc: ${MAIL_DIR}/sent
+--text follows this line--
+Sender  writes:
+
+> This is just a test message (#${gen_msg_cnt})
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
 test_emacs '(notmuch-show "id:20091118002059.067214ed at hikari")
(notmuch-show-reply)
-- 
1.7.5.4