Re: all intermediate drafts appear in threads (v0.24.2, Emacs 25.2)

2017-08-27 Thread Sanjoy Mahajan 
On 2017-07-18 07:42, David Bremner  wrote:

> Two things you might find useful to know:
>
> - you can resume editing a previous draft with 'e'
> - autosaving does not create drafts
>
> If it really bugs you, you could override the binding C-x C-s in
> notmuch-show mode.

I think that's the best solution for me.  I can more easily reprogram
Emacs than I can my fingers (which have been programmed by Emacs, and
use C-x C-s reflexively).

-- 
-Sanjoy


Save Long Wharf Park in Boston Harbor!


Six reasoning tools to make hard problems easy.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/5] test: known broken test for subject after reindexing

2017-08-27 Thread David Bremner 
In [1], Mark gave a test that was behaving strangly. This turns out to
be specific to reindexing.  I suppose one could argue that picking the
lexicographically last file name is a defensible choice, but it's
almost as easy to take the first, which seems more intuitive. So mark
the current situation as broken.

[1]: id:1503859703-2973-1-git-send-email-markwalters1...@gmail.com
---
 test/T670-duplicate-mid.sh | 8 
 1 file changed, 8 insertions(+)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index d2f89432..d4c1d1c2 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -40,6 +40,14 @@ notmuch reindex '*'
 notmuch search --output=files "sekrit" | notmuch_dir_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest 'reindex choses subject from first filename'
+test_subtest_known_broken
+cat < EXPECTED
+thread:XXX   2001-01-05 [1/1(3)] Notmuch Test Suite; message 0 (inbox unread)
+EOF
+notmuch search id:duplicate | notmuch_search_sanitize > OUTPUT
+test_expect_equal_file EXPECTED OUTPUT
+
 rm ${MAIL_DIR}/copy0
 test_begin_subtest 'Deleted first duplicate file does not stop notmuch show 
from working'
 output=$(notmuch show --body=false --format=json id:duplicate |
-- 
2.13.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/5] lib: enforce that n_message_reindex takes headers from first file

2017-08-27 Thread David Bremner 
This is still a bit stopgap to be only choosing one set of headers,
but this seems like a more defensible set of headers to choose.
---
 lib/message.cc | 4 +++-
 test/T670-duplicate-mid.sh | 1 -
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 98a88d3a..051362d4 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -2012,7 +2012,9 @@ notmuch_message_reindex (notmuch_message_t *message,
thread_id = orig_thread_id;
 
_notmuch_message_add_term (message, "thread", thread_id);
-   _notmuch_message_set_header_values (message, date, from, subject);
+   /* Take header values only from first filename */
+   if (found == 0)
+   _notmuch_message_set_header_values (message, date, from, subject);
 
ret = _notmuch_message_index_file (message, message_file);
 
diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index d4c1d1c2..ce010cf7 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -41,7 +41,6 @@ notmuch search --output=files "sekrit" | notmuch_dir_sanitize 
> OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest 'reindex choses subject from first filename'
-test_subtest_known_broken
 cat < EXPECTED
 thread:XXX   2001-01-05 [1/1(3)] Notmuch Test Suite; message 0 (inbox unread)
 EOF
-- 
2.13.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/5] test: make fallback to duplicate test more robust.

2017-08-27 Thread David Bremner 
The original intent of this test was to verify that notmuch show was
not crashing when the first file (where headers are being read from)
was deleted. Run the output through some sanitization so that as we
add and delete copies we don't have to update this test.
---
 test/T670-duplicate-mid.sh | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index decbc0a4..3495e63d 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -39,23 +39,24 @@ test_expect_equal_file EXPECTED OUTPUT
 
 rm ${MAIL_DIR}/copy1
 test_begin_subtest 'Deleted first duplicate file does not stop notmuch show 
from working'
-output=$(notmuch show --body=false --format=json id:duplicate)
+output=$(notmuch show --body=false --format=json id:duplicate |
+notmuch_json_show_sanitize | sed 's/message [0-9]/A_SUBJECT/')
 expected='[[[{
-"id": "'duplicate'",
+"id": "X",
 "match": true,
 "excluded": false,
 "filename": [
 "'"${MAIL_DIR}"/copy1'",
 "'"${MAIL_DIR}"/copy2'"
 ],
-"timestamp": 978709435,
+"timestamp": 42,
 "date_relative": "2001-01-05",
 "tags": ["inbox","unread"],
 "headers": {
-"Subject": "message 2",
+"Subject": "A_SUBJECT",
 "From": "Notmuch Test Suite ",
 "To": "Notmuch Test Suite ",
-"Date": "Fri, 05 Jan 2001 15:43:55 +"
+"Date": "GENERATED_DATE"
 }
  },
 ['
-- 
2.13.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] test/duplicate-mid: check for subject with notmuch-show

2017-08-27 Thread David Bremner 
In [1] Mark showed that the the current code (d7a49e81) is not
consistent in it's handling of subjects of messages with duplicate
message-ids (or in notmuch-speak, of messages with multiple files).
notmuch-search uses indexing order and explicitedly preserves the
first. notmuch-show (apparently) uses alphabetical (or at least xapian
term order) of filenames. In a perfect world we would probably report
all subjects in the json output; at the very least we should be
consistent.

[1]: id:87378dny3d@qmul.ac.uk
---
 test/T670-duplicate-mid.sh | 25 +
 1 file changed, 25 insertions(+)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index ce010cf7..21a9689a 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -13,6 +13,31 @@ EOF
 notmuch search id:duplicate | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest 'First subject preserved in notmuch-show (json)'
+test_subtest_known_broken
+output=$(notmuch show --body=false --format=json id:duplicate | 
notmuch_json_show_sanitize)
+expected='[[[{
+"id": "X",
+"match": true,
+"excluded": false,
+"filename": [
+"'"${MAIL_DIR}"/copy0'",
+"'"${MAIL_DIR}"/copy1'",
+"'"${MAIL_DIR}"/copy2'"
+],
+"timestamp": 42,
+"date_relative": "2001-01-05",
+"tags": ["inbox","unread"],
+"headers": {
+"Subject": "message 1",
+"From": "Notmuch Test Suite ",
+"To": "Notmuch Test Suite ",
+"Date": "GENERATED_DATE"
+}
+ },
+['
+test_expect_equal_json "$output" "$expected"
+
 test_begin_subtest 'Search for second subject'
 cat

[PATCH 2/5] test/duplicate-mid: clarify index order vs filename order

2017-08-27 Thread David Bremner 
The existing test for notmuch search had the first in filename order
the same as the first indexed, which made it harder to understand what
the underlying behaviour is. Add a file with a lexicographically
smaller name, but later index time to clarify this.
---
 test/T670-duplicate-mid.sh | 10 +++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index 3495e63d..d2f89432 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -5,15 +5,17 @@ test_description="duplicate message ids"
 add_message '[id]="duplicate"' '[subject]="message 1" [filename]=copy1'
 add_message '[id]="duplicate"' '[subject]="message 2" [filename]=copy2'
 
-test_begin_subtest 'First subject preserved'
+add_message '[id]="duplicate"' '[subject]="message 0" [filename]=copy0'
+test_begin_subtest 'search: first indexed subject preserved'
 cat < EXPECTED
-thread:XXX   2001-01-05 [1/1(2)] Notmuch Test Suite; message 1 (inbox unread)
+thread:XXX   2001-01-05 [1/1(3)] Notmuch Test Suite; message 1 (inbox unread)
 EOF
 notmuch search id:duplicate | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest 'Search for second subject'
 cat  OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
-rm ${MAIL_DIR}/copy1
+rm ${MAIL_DIR}/copy0
 test_begin_subtest 'Deleted first duplicate file does not stop notmuch show 
from working'
 output=$(notmuch show --body=false --format=json id:duplicate |
 notmuch_json_show_sanitize | sed 's/message [0-9]/A_SUBJECT/')
@@ -46,6 +49,7 @@ expected='[[[{
 "match": true,
 "excluded": false,
 "filename": [
+"'"${MAIL_DIR}"/copy0'",
 "'"${MAIL_DIR}"/copy1'",
 "'"${MAIL_DIR}"/copy2'"
 ],
-- 
2.13.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: duplicate mids: add an extra broken search test

2017-08-27 Thread Mark Walters 
---

Hi

Many thanks to bremner for the parent patch. I thought it might be
worth adding a search test after the broken show test demonstrating
the distinction between show and search. When I added it I found it
actually gives the other file as the answer!

This applies on top of the parent patch.

Best wishes

Mark


test/T670-duplicate-mid.sh | 8 
 1 file changed, 8 insertions(+)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index 55ebde3..0de2bc0 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -86,6 +86,14 @@ expected='[[[{
 ['
 test_expect_equal_json "$output" "$expected"
 
+test_begin_subtest 'First subject preserved in notmuch-search file order'
+test_subtest_known_broken
+notmuch search id:duplicate | notmuch_search_sanitize > OUTPUT
+cat < EXPECTED
+thread:XXX   2001-01-05 [1/1(3)] Notmuch Test Suite; message 1 (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 rm ${MAIL_DIR}/00-copy4
 test_begin_subtest 'Deleted first duplicate file does not stop notmuch show 
from working'
 output=$(notmuch show --body=false --format=json id:duplicate | 
notmuch_json_show_sanitize)
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 4/4] emacs: reply: Allow washing of inserted text parts during reply

2017-08-27 Thread David Edmondson 
When inserting a text/plain part during reply generation, add a hook
that allows the user to improve the display of the part.

As an example, the function `notmuch-wash-tidy-citations' might be
called.
---
 emacs/notmuch-mua.el | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index fd64b362..4d352626 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -115,6 +115,13 @@ mutiple parts get a header."
(function :tag "Other"))
   :group 'notmuch-reply)
 
+(defcustom notmuch-mua-reply-text/plain-hook nil
+  "Functions used to improve the display of text/plain parts in replies.
+
+This is analogous to `notmuch-show-insert-text/plain-hook'."
+  :type 'hook
+  :group 'notmuch-reply)
+
 ;;
 
 (defun notmuch-mua-get-switch-function ()
@@ -249,7 +256,7 @@ mutiple parts get a header."
  ;; Don't attempt to clean up messages, excerpt
  ;; citations, etc. in the original message before
  ;; quoting.
- ((notmuch-show-insert-text/plain-hook nil)
+ ((notmuch-show-insert-text/plain-hook 
notmuch-mua-reply-text/plain-hook)
   ;; Don't omit long parts.
   (notmuch-show-max-text-part-size 0)
   ;; Insert headers for parts as appropriate for replying.
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 0/4] Improved citation washing.

2017-08-27 Thread David Edmondson 

Improved citation washing.

- More aggressive citation washing.
- Allow washing during text/plain reply generation.

--

I acknowledge that some of the citation washing can seem aggressive at
times, but it does make my life more pleasant (no more shouting at the
screen "what was he thinking!" when someone uses a weird quoting
style).

David Edmondson (4):
  emacs: wash: Wrap long lines after tidying citations
  emacs: wash: Improved citation tidying
  emacs: test: Update expected output
  emacs: reply: Allow washing of inserted text parts during reply

 emacs/notmuch-mua.el   |  9 -
 emacs/notmuch-show.el  |  6 ++--
 emacs/notmuch-wash.el  | 42 +-
 .../notmuch-show-elide-non-matching-messages-off   |  2 ++
 .../notmuch-show-elide-non-matching-messages-on|  2 ++
 .../notmuch-show-indent-thread-content-off |  2 ++
 .../notmuch-show-process-crypto-mime-parts-off |  1 +
 .../notmuch-show-process-crypto-mime-parts-on  |  1 +
 .../notmuch-show-message-with-headers-hidden   |  4 +--
 .../notmuch-show-message-with-headers-visible  |  4 +--
 .../notmuch-show-thread-maildir-storage|  7 +++-
 ...hread-maildir-storage-with-fourfold-indentation |  7 +++-
 ...show-thread-maildir-storage-without-indentation |  7 +++-
 ...tmuch-show-thread-with-all-messages-uncollapsed |  5 +--
 14 files changed, 78 insertions(+), 21 deletions(-)

-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 3/4] emacs: test: Update expected output

2017-08-27 Thread David Edmondson 
Update the expected output of the tests to conform with the new
citation tidying.
---
 .../notmuch-show-elide-non-matching-messages-off   | 2 ++
 .../notmuch-show-elide-non-matching-messages-on| 2 ++
 .../notmuch-show-indent-thread-content-off | 2 ++
 .../notmuch-show-process-crypto-mime-parts-off | 1 +
 .../notmuch-show-process-crypto-mime-parts-on  | 1 +
 .../emacs.expected-output/notmuch-show-message-with-headers-hidden | 4 ++--
 .../notmuch-show-message-with-headers-visible  | 4 ++--
 test/emacs.expected-output/notmuch-show-thread-maildir-storage | 7 ++-
 .../notmuch-show-thread-maildir-storage-with-fourfold-indentation  | 7 ++-
 .../notmuch-show-thread-maildir-storage-without-indentation| 7 ++-
 .../notmuch-show-thread-with-all-messages-uncollapsed  | 5 +++--
 11 files changed, 33 insertions(+), 9 deletions(-)

diff --git 
a/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-off 
b/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-off
index e0bd2c73..7b588e76 100644
--- 
a/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-off
+++ 
b/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-off
@@ -42,6 +42,7 @@ Cheers,
   [ multipart/signed ]
   [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
   [ text/plain ]
+
   > See the patch just posted here.
 
   Is the list archived anywhere?  The obvious archives
@@ -69,6 +70,7 @@ Cheers,
 [ multipart/signed ]
 [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
 [ text/plain ]
+
 > I've also pushed a slightly more complicated (and complete) fix to my
 > private notmuch repository
 
diff --git 
a/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-on 
b/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-on
index d76d0952..79633344 100644
--- 
a/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-on
+++ 
b/test/emacs-show.expected-output/notmuch-show-elide-non-matching-messages-on
@@ -41,6 +41,7 @@ Cheers,
   [ multipart/signed ]
   [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
   [ text/plain ]
+
   > See the patch just posted here.
 
   Is the list archived anywhere?  The obvious archives
@@ -66,6 +67,7 @@ Cheers,
 [ multipart/signed ]
 [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
 [ text/plain ]
+
 > I've also pushed a slightly more complicated (and complete) fix to my
 > private notmuch repository
 
diff --git 
a/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off 
b/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off
index 1a06374d..79f1e73b 100644
--- a/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off
+++ b/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off
@@ -42,6 +42,7 @@ Date: Tue, 17 Nov 2009 15:33:01 -0500
 [ multipart/signed ]
 [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
 [ text/plain ]
+
 > See the patch just posted here.
 
 Is the list archived anywhere?  The obvious archives
@@ -69,6 +70,7 @@ Date: Tue, 17 Nov 2009 19:50:40 -0500
 [ multipart/signed ]
 [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
 [ text/plain ]
+
 > I've also pushed a slightly more complicated (and complete) fix to my
 > private notmuch repository
 
diff --git 
a/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-off 
b/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-off
index ce2892a0..c88522ca 100644
--- a/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-off
+++ b/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-off
@@ -11,6 +11,7 @@ Subject: [notmuch] Working with Maildir storage?
   [ multipart/signed ]
   [ Unknown signature status ]
   [ text/plain ]
+
   > See the patch just posted here.
 
   Is the list archived anywhere?  The obvious archives
diff --git 
a/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-on 
b/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-on
index eaa557a0..c5ebca86 100644
--- a/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-on
+++ b/test/emacs-show.expected-output/notmuch-show-process-crypto-mime-parts-on
@@ -11,6 +11,7 @@ Subject: [notmuch] Working with Maildir storage?
   [ multipart/signed ]
   [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ]
   [ text/plain ]
+
   > See the patch just posted here.
 
   Is the list archived anywhere?  The obvious archives
diff --git 
a/test/emacs.expected-output/notmuch-show-message-with-headers-hidden 
b/test/emacs.expected-output/notmuch-show-message-with-headers-hidden
index 9d7f91b6..1d05574e 100644
--- a/test/emacs.expected-outpu

[PATCH v1 2/4] emacs: wash: Improved citation tidying

2017-08-27 Thread David Edmondson 
Improve the citation tidying by being more aggressive about the
changes made:
- collapse all citation leaders to the canonical form (stripping out
  intervening spaces, for example),
- ensure that a blank line exists before and after every block of
  cited text.
---
 emacs/notmuch-wash.el | 42 ++
 1 file changed, 34 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-wash.el b/emacs/notmuch-wash.el
index 5f8b9267..c449163f 100644
--- a/emacs/notmuch-wash.el
+++ b/emacs/notmuch-wash.el
@@ -300,29 +300,55 @@ that PREFIX should not include a newline."
 
 Perform several transformations on the message body:
 
+- Remove any leading space from lines with citation leaders,
+- Collapse citation leaders to their minimal form (repeated >
+  followed by a space),
 - Remove lines of repeated citation leaders with no other
   content,
-- Remove citation leaders standing alone before a block of cited
-  text,
-- Remove citation trailers standing alone after a block of cited
-  text."
+- Remove citation leaders standing alone before and after a block
+  of cited text,
+- Ensure that a blank line separates a block of cited text from
+  non-cited text."
+
+  ;; Remove any leading spaces on citation lines.
+  (goto-char (point-min))
+  (while (re-search-forward "^ +>" nil t)
+(replace-match ">"))
+
+  ;; Collapse all citation leaders to the minimal form.
+  (goto-char (point-min))
+  (while (re-search-forward "^>[> ]*>" nil t)
+(replace-match (remove ?  (match-string 0
 
   ;; Remove lines of repeated citation leaders with no other content.
   (goto-char (point-min))
-  (while (re-search-forward "\\(^>[> ]*\n\\)\\{2,\\}" nil t)
+  (while (re-search-forward "\\(^>+ ?\n\\)\\{2,\\}" nil t)
 (replace-match "\\1"))
 
   ;; Remove citation leaders standing alone before a block of cited
   ;; text.
   (goto-char (point-min))
-  (while (re-search-forward "\\(\n\\|^[^>].*\\)\n\\(^>[> ]*\n\\)" nil t)
+  (while (re-search-forward "\\(\n\\|^[^>\n].*\\)\n\\(^>+ ?\n\\)" nil t)
 (replace-match "\\1\n"))
 
   ;; Remove citation trailers standing alone after a block of cited
   ;; text.
   (goto-char (point-min))
-  (while (re-search-forward "\\(^>[> ]*\n\\)\\(^$\\|^[^>].*\\)" nil t)
-(replace-match "\\2")))
+  (while (re-search-forward "\\(^>+ ?\n\\)\\(^$\\|^[^>].*\\)" nil t)
+(replace-match "\\2"))
+
+  ;; Ensure that a blank line separates a citation and any following
+  ;; text.
+  (goto-char (point-min))
+  (while (re-search-forward "\\(^>+ .*?\n\\)\\([^>\n]\\)" nil t)
+(replace-match "\\1\n\\2"))
+
+  ;; Ensure that a blank line separates any text and a following
+  ;; citation.
+  (goto-char (point-min))
+  (while (re-search-forward "\\(^[^>\n].+?\n\\)>" nil t)
+(replace-match "\\1\n>" ""))
+  )
 
 ;;
 
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 1/4] emacs: wash: Wrap long lines after tidying citations

2017-08-27 Thread David Edmondson 
Citation tidying might change the length of some lines, making it more
appropriate to perform line wrapping after citation tidying rather
than before.
---
 emacs/notmuch-show.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index cd901e47..ffd2f687 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -90,15 +90,15 @@ visible for any given message."
   :group 'notmuch-show
   :group 'notmuch-hooks)
 
-(defcustom notmuch-show-insert-text/plain-hook '(notmuch-wash-wrap-long-lines
-notmuch-wash-tidy-citations
+(defcustom notmuch-show-insert-text/plain-hook '(notmuch-wash-tidy-citations
+notmuch-wash-wrap-long-lines
 notmuch-wash-elide-blank-lines
 notmuch-wash-excerpt-citations)
   "Functions used to improve the display of text/plain parts."
   :type 'hook
   :options '(notmuch-wash-convert-inline-patch-to-part
-notmuch-wash-wrap-long-lines
 notmuch-wash-tidy-citations
+notmuch-wash-wrap-long-lines
 notmuch-wash-elide-blank-lines
 notmuch-wash-excerpt-citations)
   :group 'notmuch-show
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: check for subject consistency between search and show

2017-08-27 Thread David Bremner 
In [1] Mark showed that the the current code (d7a49e81) is not
consistent in it's handling of subjects of messages with duplicate
message-ids (or in notmuch-speak, of messages with multiple files).
notmuch-search uses indexing order and explicitedly preserves the
first. notmuch-show (apparently) uses alphabetical (or at least xapian
term order) of filenames. In a perfect world we would probably report
all subjects in the json output; at the very least we should be
consistent.

[1]: id:87378dny3d@qmul.ac.uk
---
 test/T670-duplicate-mid.sh | 62 +-
 1 file changed, 56 insertions(+), 6 deletions(-)

diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh
index decbc0a4..55ebde38 100755
--- a/test/T670-duplicate-mid.sh
+++ b/test/T670-duplicate-mid.sh
@@ -12,6 +12,29 @@ EOF
 notmuch search id:duplicate | notmuch_search_sanitize > OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest 'First subject preserved in notmuch-show (json)'
+output=$(notmuch show --body=false --format=json id:duplicate | 
notmuch_json_show_sanitize)
+expected='[[[{
+"id": "X",
+"match": true,
+"excluded": false,
+"filename": [
+"'"${MAIL_DIR}"/copy1'",
+"'"${MAIL_DIR}"/copy2'"
+],
+"timestamp": 42,
+"date_relative": "2001-01-05",
+"tags": ["inbox","unread"],
+"headers": {
+"Subject": "message 1",
+"From": "Notmuch Test Suite ",
+"To": "Notmuch Test Suite ",
+"Date": "GENERATED_DATE"
+}
+ },
+['
+test_expect_equal_json "$output" "$expected"
+
 test_begin_subtest 'Search for second subject'
 cat  OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
-rm ${MAIL_DIR}/copy1
+add_message '[id]="duplicate"' '[subject]="reorder file names"' 
'[filename]=00-copy4'
+test_begin_subtest 'First subject preserved in notmuch-show (json), file order'
+test_subtest_known_broken
+output=$(notmuch show --body=false --format=json id:duplicate | 
notmuch_json_show_sanitize)
+expected='[[[{
+"id": "X",
+"match": true,
+"excluded": false,
+"filename": [
+"'"${MAIL_DIR}"/00-copy4'",
+"'"${MAIL_DIR}"/copy1'",
+"'"${MAIL_DIR}"/copy2'"
+],
+"timestamp": 42,
+"date_relative": "2001-01-05",
+"tags": ["inbox","unread"],
+"headers": {
+"Subject": "message 1",
+"From": "Notmuch Test Suite ",
+"To": "Notmuch Test Suite ",
+"Date": "GENERATED_DATE"
+}
+ },
+['
+test_expect_equal_json "$output" "$expected"
+
+rm ${MAIL_DIR}/00-copy4
 test_begin_subtest 'Deleted first duplicate file does not stop notmuch show 
from working'
-output=$(notmuch show --body=false --format=json id:duplicate)
+output=$(notmuch show --body=false --format=json id:duplicate | 
notmuch_json_show_sanitize)
 expected='[[[{
-"id": "'duplicate'",
+"id": "X",
 "match": true,
 "excluded": false,
 "filename": [
+"'"${MAIL_DIR}"/00-copy4'",
 "'"${MAIL_DIR}"/copy1'",
 "'"${MAIL_DIR}"/copy2'"
 ],
-"timestamp": 978709435,
+"timestamp": 42,
 "date_relative": "2001-01-05",
 "tags": ["inbox","unread"],
 "headers": {
-"Subject": "message 2",
+"Subject": "message 1",
 "From": "Notmuch Test Suite ",
 "To": "Notmuch Test Suite ",
-"Date": "Fri, 05 Jan 2001 15:43:55 +"
+"Date": "GENERATED_DATE"
 }
  },
 ['
-- 
2.14.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 1/1] test: Perform T170 tests that don't require dtach before any that do.

2017-08-27 Thread David Edmondson 
This avoids the later tests seeing different versions of the database
depending on whether dtach is available.
---
 test/T170-sexp.sh | 11 +--
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/test/T170-sexp.sh b/test/T170-sexp.sh
index 40e5e21d..db142e49 100755
--- a/test/T170-sexp.sh
+++ b/test/T170-sexp.sh
@@ -26,6 +26,11 @@ add_message "[subject]=\"sexp-show-utf8-body-sübjéct\"" 
"[date]=\"Sat, 01 Jan
 output=$(notmuch show --format=sexp "jsön-show-méssage")
 test_expect_equal "$output" ":id \"${gen_msg_id}\" :match t :excluded nil 
:filename (\"${gen_msg_filename}\") :timestamp 946728000 :date_relative 
\"2000-01-01\" :tags (\"inbox\" \"unread\") :headers (:Subject 
\"sexp-show-utf8-body-sübjéct\" :From \"Notmuch Test Suite 
\" :To \"Notmuch Test Suite 
\" :Date \"Sat, 01 Jan 2000 12:00:00 +\") :body 
((:id 1 :content-type \"text/plain\" :content \"jsön-show-méssage\n\"))) ("
 
+test_begin_subtest "Search message: sexp, utf-8"
+add_message "[subject]=\"sexp-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 
Jan 2000 12:00:00 -\"" "[body]=\"jsön-search-méssage\""
+output=$(notmuch search --format=sexp "jsön-search-méssage" | 
notmuch_search_sanitize)
+test_expect_equal "$output" "((:thread \"0004\" :timestamp 
946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch 
Test Suite\" :subject \"sexp-search-utf8-body-sübjéct\" :query 
(\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))"
+
 test_begin_subtest "Show message: sexp, inline attachment filename"
 subject='sexp-show-inline-attachment-filename'
 id="sexp-show-inline-attachment-filen...@notmuchmail.org"
@@ -41,10 +46,4 @@ filename=$(notmuch search --output=files "id:$id")
 attachment_length=$(( $(base64 $TEST_DIRECTORY/README | wc -c) - 1 ))
 test_expect_equal "$output" ":id \"$id\" :match t :excluded nil :filename 
(\"$filename\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags 
(\"inbox\") :headers (:Subject \"sexp-show-inline-attachment-filename\" :From 
\"Notmuch Test Suite \" :To 
\"test_su...@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +\") :body 
((:id 1 :content-type \"multipart/mixed\" :content ((:id 2 :content-type 
\"text/plain\" :content \"This is a test message with inline attachment with a 
filename\") (:id 3 :content-type \"application/octet-stream\" 
:content-disposition \"inline\" :filename \"README\" :content-transfer-encoding 
\"base64\" :content-length $attachment_length) ("
 
-test_begin_subtest "Search message: sexp, utf-8"
-add_message "[subject]=\"sexp-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 
Jan 2000 12:00:00 -\"" "[body]=\"jsön-search-méssage\""
-output=$(notmuch search --format=sexp "jsön-search-méssage" | 
notmuch_search_sanitize)
-test_expect_equal "$output" "((:thread \"0005\" :timestamp 
946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch 
Test Suite\" :subject \"sexp-search-utf8-body-sübjéct\" :query 
(\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))"
-
-
 test_done
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Duplicate message ids

2017-08-27 Thread Mark Walters 

Hi

A concern for notmuch is some form of attack via someone sending a
message with a duplicate message id. I think I have seen it asserted
that it is not so much of a problem as the first message received by
notmuch will have priority.

However, I believe that this is not the case. The script below gives a
demonstration (on my system at least). I have written it as a "test" so
(I hope) it doesn't mess up the system. It should work if you put it in
the test directory and execute it.

It adds a message, runs notmuch new, adds a message with the same id,
runs notmuch new, and then runs notmuch search  and notmuch show
. The former shows the subject of the first message, and the latter
the subject of the second message.

Best wishes

Mark

#!/usr/bin/env bash
test_description='Do duplicate message ids get shown in arrival order'
. ./test-lib.sh || exit 1


find ${MAIL_DIR}
generate_message [id]=duplicate-id [subject]=first [body]=first
mkdir "${MAIL_DIR}"/b
mv "$gen_msg_filename" "${MAIL_DIR}"/b
notmuch new
generate_message [id]=duplicate-id [subject]=second [body]=second
mv "$gen_msg_filename" "${MAIL_DIR}"/a
notmuch new
find ${MAIL_DIR}

echo
echo SEARCH: observe \"first\" is the subject
notmuch search id:duplicate-id

echo
echo SHOW: observe \"second\" is the subject and body
notmuch show --format=json id:duplicate-id |json_pp
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/5] build: add target to run cppcheck

2017-08-27 Thread Jani Nikula 
On Sat, 26 Aug 2017, David Bremner  wrote:
> The advantage of having a target as opposed to running cppcheck by
> hand
>
>   - reuse list of source files
>   - output errors in a format parsable, e.g. by emacs
>   - returns exit code 1 on any error, for possibly use in other
> targets.
>
> Things not addressed here
>
>   - parallelism. Doing this correctly seems like a rabbit-hole.
>
>   - what target to invoke this from.  The 8s delay (on my machine) seems
> acceptable, but not necessarily the resulting fragility in
> autobuilders.

Perhaps something could be build around something like below. It
addresses the parallelism, but it's not without problems either. Only
the files that need to be built would get checked; OTOH that may be
beneficial while developing. Clearly we don't want to run cppcheck
unconditionally when it's available. Would it be a configure option that
would not be enabled by default? Or a make variable passed on the
command like, say 'make CPPCHECK=1'?

Food for thought; I'm not opposed to applying the patch at hand either.

BR,
Jani.



diff --git a/Makefile.local b/Makefile.local
index 6a5168f16427..fd59feafcd44 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -191,12 +191,20 @@ endif
 # Otherwise, print the full command line.
 quiet ?= $($(shell echo $1 | sed -e s'/ .*//'))
 
+CPPCHECK=cppcheck
+
 %.o: %.cc $(global_deps)
@mkdir -p $(patsubst %/.,%,.deps/$(@D))
+ifeq ($(HAVE_CPPCHECK),1)
+   $(call quiet,CPPCHECK) --template="{file}:{line}: {severity}: 
{message}" --quiet --error-exitcode=1 $<
+endif
$(call quiet,CXX $(CPPFLAGS) $(CXXFLAGS)) -c $(FINAL_CXXFLAGS) $< -o $@ 
-MD -MP -MF .deps/$*.d
 
 %.o: %.c $(global_deps)
@mkdir -p $(patsubst %/.,%,.deps/$(@D))
+ifeq ($(HAVE_CPPCHECK),1)
+   $(call quiet,CPPCHECK) --template="{file}:{line}: {severity}: 
{message}" --quiet --error-exitcode=1 $<
+endif
$(call quiet,CC $(CPPFLAGS) $(CFLAGS)) -c $(FINAL_CFLAGS) $< -o $@ -MD 
-MP -MF .deps/$*.d
 
 .PHONY : clean


> ---
>  Makefile.local | 10 ++
>  configure  | 11 +++
>  2 files changed, 21 insertions(+)
>
> diff --git a/Makefile.local b/Makefile.local
> index af12ca7f..6a5168f1 100644
> --- a/Makefile.local
> +++ b/Makefile.local
> @@ -283,6 +283,16 @@ CLEAN := $(CLEAN) version.stamp notmuch-*.tar.gz.tmp
>  
>  DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config
>  
> +CLEAN := $(CLEAN) cppcheck.stamp
> +cppcheck: cppcheck.stamp
> +cppcheck.stamp: $(SRCS)
> +ifeq ($(HAVE_CPPCHECK),1)
> + cppcheck --template="{file}:{line}: {severity}: {message}" --quiet 
> --error-exitcode=1 ${SRCS}
> + touch $@
> +else
> + @echo No cppcheck installed; skipping static checking
> +endif
> +
>  DEPS := $(SRCS:%.c=.deps/%.d)
>  DEPS := $(DEPS:%.cc=.deps/%.d)
>  -include $(DEPS)
> diff --git a/configure b/configure
> index c5e2ffed..364854f3 100755
> --- a/configure
> +++ b/configure
> @@ -646,6 +646,14 @@ if [ $WITH_DESKTOP = "1" ]; then
>  fi
>  fi
>  
> +printf "Checking for cppcheck... "
> +if command -v cppcheck > /dev/null; then
> +have_cppcheck=1
> +printf "Yes.\n"
> +else
> +printf "No.\n"
> +fi
> +
>  libdir_in_ldconfig=0
>  
>  printf "Checking which platform we are on... "
> @@ -1065,6 +1073,9 @@ zsh_completion_dir = 
> ${ZSHCOMLETIONDIR:=\$(prefix)/share/zsh/functions/Completio
>  # build its own version)
>  HAVE_CANONICALIZE_FILE_NAME = ${have_canonicalize_file_name}
>  
> +# Whether the cppcheck static checker is available
> +HAVE_CPPCHECK = ${have_cppcheck}
> +
>  # Whether the getline function is available (if not, then notmuch will
>  # build its own version)
>  HAVE_GETLINE = ${have_getline}
> -- 
> 2.14.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] emacs: Add notmuch-update-search-tags

2017-08-27 Thread Mark Walters 

Hi

OK I have now actually tested it, and I have read the patch more
carefully, but I am afraid I still have concerns.

The key problem is the patch assumes that the display of a thread in a
search buffer depends only on the thread. And this is not true as the
number of matching messages is displayed eg [10/19], and the author list
is split into matching authors | unmatching authors.

However, when the tags in a thread are changed the patch makes all
search buffers containing that thread update to look the same. Indeed,
if you change a tag on a single message I think the count will always
update to be [1/??] as there is only one message matching the tag-change
query.

I think you could get round this by modifying your code only slightly --
rather than calling notmuch-search-update-result in
notmuch-search-update-results, *just* update the tags, using something
like notmuch-search-set-tags. (I have just tried this and it seems to work.)

This is not perfect, as this will show tags of newly arrived messages in
the thread, but that might well be acceptable.

And this still keeps it to one call to the database, which avoids your
(completely correct) performance concerns.


> It may be possible to optimize this down to one batch search query per 
> notmuch-search buffer - however, this results in a large search query. 
> Not only would one take a while to execute, but the resulting query can 
> become too large to be passed as a command-line parameter, and "notmuch 
> search" does not seem to have a --batch switch to read queries from 
> standard input.

This points to my next concern -- this is already a problem in the
current patch. If you go into a moderately large search buffer, and do
something like * +foo (to tag all the messages with foo), the tag step
works because is uses the --batch switch to tag, but your search query
doesn't.

The options here are to:  add --batch to search handling, or  just
decide not to do display the tag updates for large queries.

Incidentally there also seems to a bug in the current that the first
thread in a search buffer doesn't seem to get updated. I think you are
using too low level functions -- things like
notmuch-search-foreach-result might do exactly what you need.

Finally, in the longer term, do you wanting to do this for tree and show
buffers too?

[An alternative approach, which I don't think I like but I mention in
case others do -- we could only propagate tag changes to parent
buffers. So updating a tag in a show buffer only updates the single
search buffer that called it. This might avoid the large query problem
as show buffers probably don't have large queries.]

Best wishes

Mark









___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch