[announce] Bower 1.0

2022-08-20 Thread Peter Wang
Hi,

Bower is a curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 1.0 (2022-08-21)
==
The first commit to this project was made 11 years ago. Let's call this v1.0.

* index: Use Alt+Enter to open a thread in "obscured" mode.
  Messages that did not match the search query are displayed differently and
  not interactable.
* thread/pager: Use 'M' key to toggle obscured mode.
* Allow omitting "+" prefix when entering tag additions.
* Support tag editing on compose screen.
* Inherit tags from message being replied to or forwarded.
* Also apply tags to sent message when using a custom post_sendmail action.
* Use domain of From address for right part of Message-ID.
* Expand ~ in config values.
* Expand ~ in interactively entered commands.
* config: Add ui.default_save_directory option.
* Remove lynx as the default HTML-to-text filter.
* Improve search string parser.
* Fix bug where "body:(foo bar)" was unparsed as "body: ( foo bar )".
* Restrict places in search string where "~" names a Bower search term alias
  to after whitespace or open parenthesis/brace.
* Accept mailto: argument on command line.
* Accept --help and --version options.


BTW, you may also be interested in my IMAP/Maildir synchronisation tool:
https://github.com/wangp/pushpull
It doesn't synchronise notmuch tags, though.

Peter
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[announce] Bower 0.13

2021-07-23 Thread Peter Wang
Hi,

Bower is a curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.13 (2021-07-24)
===
Please note there are a few key binding changes.

* Support composing messages with text/html alternative;
  see alt_html_filter config option. (Frank Seifferth)
* Add compose.signature_file config option.
* Support Alt+Key bindings for fast alias expansion in index. (Frank Seifferth)
* Use 'U' to toggle unread instead of 'N'. (Frank Seifferth)
* Add 'N' to search in opposite direction. (Frank Seifferth)
* Add 'w' as an alias to save parts. (Frank Seifferth)
* Add '$' to mark threads or messages as spam. (Sean E. Russell)
* compose: Use 'B' for "edit Bcc". (Frank Seifferth)
* Add thread_ordering config option. (Frank Seifferth)
* Increase thread pane size for long threads. (Frank Seifferth)
* Auto-refresh index view after a period of inactivity (off by default).
* Drop "Non-text part" and "Attachment" lines in reply templates.
* Minor improvements.

Peter
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[announce] Bower 0.12

2020-07-31 Thread Peter Wang
Hi,

Bower is a curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.12 (2020-08-01)
===
This release requires Mercury version 14.04 or above.

* Create reply templates in bower instead of using notmuch reply output.
* Include filtered non-text parts in reply/forward/edit-as-new templates.
* Make the visibility of parts in the pager determine which parts to include
  in message templates.
* Pass PART_CONTENT_TYPE and PART_CHARSET environment variables to filters.
* Set detected charset on outgoing text attachments.
* Use application/octet-stream for text attachments with non UTF-8 content.
* Improve parsing of notmuch output.
* Parse notmuch format v4 signature error flags.
* Clean up spaces and unprintable chars when displaying header values.
* Hide tags starting with "." (Niklas Haas)
* Change default cursor color.
* Handle screen resize in a few more places.
* Add make clean target.
* Minor improvements.

Bower 0.11 (2019-08-17)
===

* Add '|' key to pipe thread or message IDs to external command.
* index: Add 'z' key to toggle authors column. (Will Dietz)
* thread/pager: Show messages with exclude tags if the tags are explicitly
  searched for in the index view.
* compose: Allow non-UTF-8 "text" files to be attached.
* config: Add ui.wrap_width option to wrap message lines before terminal width.
* config: Add color.index.total option. (Will Dietz)
* Install signal handlers to improve behaviour with suspending and exiting.
* Write pgp-signature parts with name="signature.asc"
* Minor improvements.

Peter
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] completion: remove "setup" from the list of possible completions

2020-06-24 Thread Peter Wang
On Mon, 22 Jun 2020 12:22:50 +0200 Lukasz Stelmach  
wrote:
> It was <2020-06-20 sob 12:53>, when Reto wrote:
> > On Fri, Jun 19, 2020 at 12:40:49PM +0200, Łukasz Stelmach wrote:
> >> Having "setup" in the set requires entering three instad of two characters
> >> for "search". Since "setup" is rearly used it makes little sense to have
> >> it in the set and cripple UX for much more frequently used "search".
> >
> > I very much disagree with this patch.
> > The completions should contain all possible values, saving a single 
> > keystroke is
> > certainly not a valid reason to remove a valid option from the completions.
> >
> > Write an alias into your bashrc if that bothers you so much... Then you can 
> > save
> > much more keystrokes.
> 
> I already have several aliases covering most of my use cases, however, I
> still use "notmuch search" from time to time and I came to a conclusion
> expressed in this patch. Of course, as a random user, I can only suggest
> and by no means insist on applying it.

Another possibility may be to rename "notmuch setup" to "notmuch init",
treating "setup" as a deprecated synonym for "init". The completions
would include "init" but not "setup".

(Just an idea, I don't really care.)

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: timestamp overflows sprinter interface

2020-02-07 Thread Peter Wang
On Mon, 03 Feb 2020 09:40:49 -0400, David Bremner  wrote:
> Peter Wang  writes:
> 
> > Hi,
> >
> > On a system where time_t is 64-bit and 'int' is a signed 32-bit integer
> > type, timestamps beyond some time in 2038 will be serialised to a
> > negative value.
> 
> I admire your forward thinking!

Nah, someone reported an issue with my mail client when viewing a
message dated 09 Dec 2058.

> 
> >
> > The simplest solution appears to be to change the type in the sprinter
> > method to int64_t:
> >
> > void (*integer)(struct sprinter *, int64_t);
> >
> > Any other suggestions?
> 
> Since this is an internal API, I don't really see a big problem with
> doing this.

I've sent a patch.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2] sprinter: change integer method to use int64_t

2020-02-07 Thread Peter Wang
In particular, timestamps beyond 2038 could overflow the sprinter
interface on systems where time_t is 64-bit but 'int' is a signed 32-bit
integer type.
---
 sprinter-json.c   | 5 +++--
 sprinter-sexp.c   | 5 +++--
 sprinter-text.c   | 5 +++--
 sprinter.h| 2 +-
 test/T160-json.sh | 1 -
 5 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/sprinter-json.c b/sprinter-json.c
index c6ec8577..273bdeca 100644
--- a/sprinter-json.c
+++ b/sprinter-json.c
@@ -1,3 +1,4 @@
+#include 
 #include 
 #include 
 #include 
@@ -124,11 +125,11 @@ json_string (struct sprinter *sp, const char *val)
 }
 
 static void
-json_integer (struct sprinter *sp, int val)
+json_integer (struct sprinter *sp, int64_t val)
 {
 struct sprinter_json *spj = json_begin_value (sp);
 
-fprintf (spj->stream, "%d", val);
+fprintf (spj->stream, "%"PRId64, val);
 }
 
 static void
diff --git a/sprinter-sexp.c b/sprinter-sexp.c
index 6891ea42..35c007d5 100644
--- a/sprinter-sexp.c
+++ b/sprinter-sexp.c
@@ -18,6 +18,7 @@
  * Author: Peter Feigl 
  */
 
+#include 
 #include 
 #include 
 #include 
@@ -161,11 +162,11 @@ sexp_keyword (struct sprinter *sp, const char *val)
 }
 
 static void
-sexp_integer (struct sprinter *sp, int val)
+sexp_integer (struct sprinter *sp, int64_t val)
 {
 struct sprinter_sexp *sps = sexp_begin_value (sp);
 
-fprintf (sps->stream, "%d", val);
+fprintf (sps->stream, "%"PRId64, val);
 }
 
 static void
diff --git a/sprinter-text.c b/sprinter-text.c
index 648b54b1..7b68f98c 100644
--- a/sprinter-text.c
+++ b/sprinter-text.c
@@ -1,3 +1,4 @@
+#include 
 #include 
 #include 
 #include 
@@ -44,11 +45,11 @@ text_string (struct sprinter *sp, const char *val)
 }
 
 static void
-text_integer (struct sprinter *sp, int val)
+text_integer (struct sprinter *sp, int64_t val)
 {
 struct sprinter_text *sptxt = (struct sprinter_text *) sp;
 
-fprintf (sptxt->stream, "%d", val);
+fprintf (sptxt->stream, "%"PRId64, val);
 }
 
 static void
diff --git a/sprinter.h b/sprinter.h
index 182b1a8b..528d8a2d 100644
--- a/sprinter.h
+++ b/sprinter.h
@@ -33,7 +33,7 @@ typedef struct sprinter {
  */
 void (*string)(struct sprinter *, const char *);
 void (*string_len)(struct sprinter *, const char *, size_t);
-void (*integer)(struct sprinter *, int);
+void (*integer)(struct sprinter *, int64_t);
 void (*boolean)(struct sprinter *, bool);
 void (*null)(struct sprinter *);
 
diff --git a/test/T160-json.sh b/test/T160-json.sh
index ec1b5adb..d975efa7 100755
--- a/test/T160-json.sh
+++ b/test/T160-json.sh
@@ -65,7 +65,6 @@ test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
  \"unread\"]}]"
 
 test_begin_subtest "Search message: json, 64-bit timestamp"
-test_subtest_known_broken
 add_message "[subject]=\"json-search-64bit-timestamp-subject\"" "[date]=\"Tue, 
01 Jan 2999 12:00:00 -\"" "[body]=\"json-search-64bit-timestamp-message\""
 output=$(notmuch search --format=json "json-search-64bit-timestamp-message" | 
notmuch_search_sanitize)
 test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
-- 
2.25.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/2] test: add known broken test with timestamp beyond 2038

2020-02-07 Thread Peter Wang
---
 test/T160-json.sh | 15 +++
 1 file changed, 15 insertions(+)

diff --git a/test/T160-json.sh b/test/T160-json.sh
index 004adb4e..ec1b5adb 100755
--- a/test/T160-json.sh
+++ b/test/T160-json.sh
@@ -64,6 +64,21 @@ test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
  \"tags\": [\"inbox\",
  \"unread\"]}]"
 
+test_begin_subtest "Search message: json, 64-bit timestamp"
+test_subtest_known_broken
+add_message "[subject]=\"json-search-64bit-timestamp-subject\"" "[date]=\"Tue, 
01 Jan 2999 12:00:00 -\"" "[body]=\"json-search-64bit-timestamp-message\""
+output=$(notmuch search --format=json "json-search-64bit-timestamp-message" | 
notmuch_search_sanitize)
+test_expect_equal_json "$output" "[{\"thread\": \"XXX\",
+ \"timestamp\": 32472187200,
+ \"date_relative\": \"the future\",
+ \"matched\": 1,
+ \"total\": 1,
+ \"authors\": \"Notmuch Test Suite\",
+ \"subject\": \"json-search-64bit-timestamp-subject\",
+ \"query\": [\"id:$gen_msg_id\", null],
+ \"tags\": [\"inbox\",
+ \"unread\"]}]"
+
 test_begin_subtest "Format version: too low"
 test_expect_code 20 "notmuch search --format-version=0 \\*"
 
-- 
2.25.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


timestamp overflows sprinter interface

2020-01-30 Thread Peter Wang
Hi,

On a system where time_t is 64-bit and 'int' is a signed 32-bit integer
type, timestamps beyond some time in 2038 will be serialised to a
negative value.

The simplest solution appears to be to change the type in the sprinter
method to int64_t:

void (*integer)(struct sprinter *, int64_t);

Any other suggestions?

Peter

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] doc: fix references to search.exclude_tags

2019-01-26 Thread Peter Wang
The documentation incorrectly referred to a configuration item
"search.tag_exclude" in some places, instead of "search.exclude_tags".
---
 doc/man1/notmuch-address.rst | 2 +-
 doc/man1/notmuch-count.rst   | 2 +-
 doc/man1/notmuch-search.rst  | 2 +-
 doc/man1/notmuch-show.rst| 2 +-
 4 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index 12d86e89..2a7df6f0 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -92,7 +92,7 @@ Supported options for **address** include
 
 ``--exclude=(true|false)``
 A message is called "excluded" if it matches at least one tag in
-search.tag\_exclude that does not appear explicitly in the search
+search.exclude\_tags that does not appear explicitly in the search
 terms. This option specifies whether to omit excluded messages in
 the search process.
 
diff --git a/doc/man1/notmuch-count.rst b/doc/man1/notmuch-count.rst
index 9ca20dab..0eac5dbe 100644
--- a/doc/man1/notmuch-count.rst
+++ b/doc/man1/notmuch-count.rst
@@ -36,7 +36,7 @@ Supported options for **count** include
 same message-id).
 
 ``--exclude=(true|false)``
-Specify whether to omit messages matching search.tag\_exclude from
+Specify whether to omit messages matching search.exclude\_tags from
 the count (the default) or not.
 
 ``--batch``
diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst
index 654c5f2c..ed9ff4e5 100644
--- a/doc/man1/notmuch-search.rst
+++ b/doc/man1/notmuch-search.rst
@@ -100,7 +100,7 @@ Supported options for **search** include
 
 ``--exclude=(true|false|all|flag)``
 A message is called "excluded" if it matches at least one tag in
-search.tag\_exclude that does not appear explicitly in the search
+search.exclude\_tags that does not appear explicitly in the search
 terms. This option specifies whether to omit excluded messages in
 the search process.
 
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index a2708a04..becd3e79 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -161,7 +161,7 @@ Supported options for **show** include
 Default: ``auto``
 
 ``--exclude=(true|false)``
-Specify whether to omit threads only matching search.tag\_exclude
+Specify whether to omit threads only matching search.exclude\_tags
 from the search results (the default) or not. In either case the
 excluded message will be marked with the exclude flag (except when
 output=mbox when there is nowhere to put the flag).
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [announce] Bower 0.10

2019-01-26 Thread Peter Wang
On Sat, 26 Jan 2019 09:17:27 +, Ben Oliver  wrote:
> On 2019-01-26 14:10:48, Peter Wang wrote:
> >Hi,
> >
> >Bower is a curses frontend for the Notmuch email system.
> >I wrote it for me, but you might like it, too.
> >
> > https://github.com/wangp/bower
> 
> Is this not ripe for confusion with https://bower.io/ ?

They should have thought of that when they picked the name!

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[announce] Bower 0.10

2019-01-25 Thread Peter Wang
Hi,

Bower is a curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.10 (2019-01-26)
===

* thread/pager: Add 'W' key to forward message.
* Support BOWER_CONFIG environment variable.
* Document support for XDG Base Directory Specification.
* Support tab expansion for query:.
* Support ~NAME as a shorthand for query:NAME.
* Support Shift-Tab to cycle through completions in reverse.
* Make path completion work more smoothly.
* Minor improvements.

Bower 0.9 (2018-10-07)
==

* Support conversion of message parts to text using external filters.
* Add ',' key to go to next unread thread/message (synonym for Tab).
* thread/pager: Show pager progress as percentage of current message.
* thread/pager: Periodically poll for new messages.
* config: Replace index.poll_period_secs with ui.poll_period_secs.

Bower 0.8.1 (2018-04-01)


* Save/restore curses mode for any command that might invoke pinentry-curses.
* Add makefile target to produce man page (bower.1).

Bower 0.8 (2017-07-30)
==
This release requires notmuch 0.21 or above.

* Look up addresses using notmuch address.
* Move add to addressbook to '@' key.
* Add 'a' keys to apply '-inbox -unread' or else '+inbox'.
* Add 'A' key to leave thread view and apply '-inbox -unread' to the thread.
* Bug fixes and minor improvements.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] cli/insert: new message file can be world-readable (rely on umask)

2018-02-05 Thread Peter Wang
On Sun,  4 Feb 2018 23:37:03 -0500, Daniel Kahn Gillmor 
 wrote:
> There are legitimate cases (public archives) where a user might
> actually want their archive to be readable to the world.
> 
> "notmuch insert" historically used mode 0600 (unreadable by group or
> other), but that choice doesn't appear to have been specifically
> justified (perhaps an abundance of caution?).

I can't remember any specific reason for 0600 instead of 0644.
Probably just assumed that mail is supposed to be private.

> If the user wants "notmuch insert" to create files that are not
> readable by group or other, they can set their umask more
> restrictively.

By calling notmuch through a wrapper shell script, I suppose.

The mode for --create-folder should be reconsidered as well.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[announce] Bower 0.7

2016-03-27 Thread Peter Wang
Hi,

Bower is a curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.7 (2016-03-28)
==
This release requires notmuch 0.16 or above (notmuch 0.19+ is recommended)
and gpgme. Consider updating your bower.conf to make use of multiple accounts.

* Support for multiple accounts.
* PGP/MIME support.
* Use notmuch insert command instead of notmuch-deliver.
* Make use of notmuch search --exclude=all option.
* Make use of notmuch show --entire-thread=false option.
* Represent excluded messages (with replies) in thread view.
* Fix ambiguity between ~dDATE and search alias beginning with ~d.
* Do not hide the 'inbox' tag.
* Run poll_notify command when polling finds new messages.
* Accept mailto URI at To: prompt.
* Add 'E' edit current message as template for new message.
* Add '~' key to initiate limit prompt populated with "~".
* Wrap long header lines in pager.
* Link with libpanelw, not libpanel.
* Bug fixes and minor improvements.
* Compatibility with newer Mercury compilers.


Also, I have another project that may be of interest to notmuch users.
plugsink is a bidirectional IMAP/Maildir synchronisation tool.

https://github.com/wangp/plugsink

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 08/10] cli: add insert --must-index option

2014-07-29 Thread Peter Wang
On Wed, 09 Jul 2014 20:20:17 -0300, David Bremner  wrote:
> Peter Wang  writes:
> 
> > This option causes notmuch insert to fail (with exit code 3) on failure
> > to index the message, or failure to set the tags on the message, or if
> > closing (flushing) the database fails.  Failure to sync tags to flags
> > has no effect.
> 
> I don't really understand why it's OK to ignore failure to sync
> flags. Can you explain? Or point to a previous discussion if we already
> went through this?

It wasn't really discussed.  Only Mark expressed that he didn't really
mind in id:87hadtxfrr.fsf at qmul.ac.uk

It might be justified that, unlike a failure to index, the message will
still be found though the notmuch interface, with the tags intact.
Running notmuch new will not lose those tags either (I think).
It's only when you view the message through another interface that there
will be an inconsistency in the small subset of tags which can be mapped
to maildir flags.

If we were strict about failure to sync flags, then presumably the
newly-added message would need to be deleted from disk and removed from
the database.  I'm not sure it's worthwhile to avoid that (minor)
inconsistency.

Peter


Re: [PATCH v2 08/10] cli: add insert --must-index option

2014-07-28 Thread Peter Wang
On Wed, 09 Jul 2014 20:20:17 -0300, David Bremner  wrote:
> Peter Wang  writes:
> 
> > This option causes notmuch insert to fail (with exit code 3) on failure
> > to index the message, or failure to set the tags on the message, or if
> > closing (flushing) the database fails.  Failure to sync tags to flags
> > has no effect.
> 
> I don't really understand why it's OK to ignore failure to sync
> flags. Can you explain? Or point to a previous discussion if we already
> went through this?

It wasn't really discussed.  Only Mark expressed that he didn't really
mind in id:87hadtxfrr@qmul.ac.uk

It might be justified that, unlike a failure to index, the message will
still be found though the notmuch interface, with the tags intact.
Running notmuch new will not lose those tags either (I think).
It's only when you view the message through another interface that there
will be an inconsistency in the small subset of tags which can be mapped
to maildir flags.

If we were strict about failure to sync flags, then presumably the
newly-added message would need to be deleted from disk and removed from
the database.  I'm not sure it's worthwhile to avoid that (minor)
inconsistency.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 06/10] cli: refactor insert

2014-07-06 Thread Peter Wang
On Sat, 05 Jul 2014 10:18:05 -0300, David Bremner  wrote:
> Peter Wang  writes:
> 
> > -cleanup_path = tmppath;
> > -
> > -if (! copy_stdin (fdin, fdout))
> > -   goto FAIL;
> > +if (! copy_stdin (fdin, fdout)) {
> > +   close (fdout);
> > +   unlink (tmppath);
> > +   return FALSE;
> > +}
> 
> I'm not completely convinced by replacement of the "goto FAIL" with the
> multiple returns.  I'd lean to towards being consistent with the notmuch
> codebase unless the FAIL block is really horrendous

Eh, when I came back to the code I found it unnecessary convoluted.
However, you can squash in the attached patch if you like.
As an objective measure, the function with the FAIL block is longer.

> 
> Is there a good reason to use TRUE and FALSE for return values rather
> than EXIT_SUCCESS and EXIT_FAILURE? It seems like the latter would be
> overall slightly simpler in notmuch_insert_command.

Not sure what you have in mind.  I think CLI exit codes should be
confined to notmuch_insert_command.

Peter
-- next part --
A non-text attachment was scrubbed...
Name: 0001-goto-fail.patch
Type: text/x-diff
Size: 1806 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20140706/5cdb4025/attachment.patch>


Re: [PATCH v2 06/10] cli: refactor insert

2014-07-05 Thread Peter Wang
On Sat, 05 Jul 2014 10:18:05 -0300, David Bremner  wrote:
> Peter Wang  writes:
> 
> > -cleanup_path = tmppath;
> > -
> > -if (! copy_stdin (fdin, fdout))
> > -   goto FAIL;
> > +if (! copy_stdin (fdin, fdout)) {
> > +   close (fdout);
> > +   unlink (tmppath);
> > +   return FALSE;
> > +}
> 
> I'm not completely convinced by replacement of the "goto FAIL" with the
> multiple returns.  I'd lean to towards being consistent with the notmuch
> codebase unless the FAIL block is really horrendous

Eh, when I came back to the code I found it unnecessary convoluted.
However, you can squash in the attached patch if you like.
As an objective measure, the function with the FAIL block is longer.

> 
> Is there a good reason to use TRUE and FALSE for return values rather
> than EXIT_SUCCESS and EXIT_FAILURE? It seems like the latter would be
> overall slightly simpler in notmuch_insert_command.

Not sure what you have in mind.  I think CLI exit codes should be
confined to notmuch_insert_command.

Peter
>From be07c53d6d5f22ced723a44f996323d2a982113e Mon Sep 17 00:00:00 2001
From: Peter Wang 
Date: Sun, 6 Jul 2014 12:52:26 +1000
Subject: [PATCH] goto fail

---
 notmuch-insert.c | 32 ++--
 1 file changed, 18 insertions(+), 14 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 7db4f73..8dfc8bb 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -342,26 +342,25 @@ write_message (void *ctx, int fdin, const char *dir, char **newpath)
 {
 char *tmppath;
 char *newdir;
+char *cleanup_path;
 int fdout;
 
 fdout = maildir_open_tmp_file (ctx, dir, &tmppath, newpath, &newdir);
 if (fdout < 0)
 	return FALSE;
 
-if (! copy_stdin (fdin, fdout)) {
-	close (fdout);
-	unlink (tmppath);
-	return FALSE;
-}
+cleanup_path = tmppath;
+
+if (! copy_stdin (fdin, fdout))
+	goto FAIL;
 
 if (fsync (fdout) != 0) {
 	fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
-	close (fdout);
-	unlink (tmppath);
-	return FALSE;
+	goto FAIL;
 }
 
 close (fdout);
+fdout = -1;
 
 /* Atomically move the new message file from the Maildir 'tmp' directory
  * to the 'new' directory.  We follow the Dovecot recommendation to
@@ -370,16 +369,21 @@ write_message (void *ctx, int fdin, const char *dir, char **newpath)
  */
 if (rename (tmppath, *newpath) != 0) {
 	fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
-	unlink (tmppath);
-	return FALSE;
+	goto FAIL;
 }
 
-if (! sync_dir (newdir)) {
-	unlink (*newpath);
-	return FALSE;
-}
+cleanup_path = *newpath;
+
+if (! sync_dir (newdir))
+	goto FAIL;
 
 return TRUE;
+
+  FAIL:
+if (fdout >= 0)
+	close (fdout);
+unlink (cleanup_path);
+return FALSE;
 }
 
 int
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[announce] Bower 0.6

2014-05-01 Thread Peter Wang
Hi,

Bower is yet another curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.6 (2014-05-01)
==
This release requires notmuch 0.15 or above.  Users who access notmuch
via ssh must update their configuration (see bower.conf.sample).

* Use notmuch 0.15 date parser instead of GNU date.
* Use notmuch reply --reply-to=(sender|all).
* Support single ~d DATE syntax.
* Use the terminal's default background colour.
* Configurable colours.
* Use shell-style word splitting of all commands.
* Automatically perform extra round of shell-quoting for ssh commands.
* Run open part/URL command in background if suffixed by '&'.
* Leave curses when running open part/URL command in foreground.
* Configurable open URL and open part commands.
* Configurable polling period in index view.
* Configurable Drafts/Sent maildir folders in ~/.notmuch-config.
* Fold/unfold header lines when composing messages.
* Parse address lists per RFC 5322.
* Refuse to send message when a header contains an invalid address.
* Encode/decode headers per RFC 2047.
* Encode attachment filenames per RFC 2231.
* Allow non-ASCII characters in addressbook aliases.
* Hide long blocks of quoted text by default.
* Toggle inline display of any selected part with 'Z'.
* Allow inline display of unsupported part if it contains text.
* Improve handling of multipart/* parts.
* Support undo in text entry.
* Support text entry scrolling for lines wider than the screen.
* Support ^W to delete word backwards.
* Expand to common prefix initially when completing.
* Bug fixes and minor improvements.

Peter


[announce] Bower 0.6

2014-05-01 Thread Peter Wang
Hi,

Bower is yet another curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.6 (2014-05-01)
==
This release requires notmuch 0.15 or above.  Users who access notmuch
via ssh must update their configuration (see bower.conf.sample).

* Use notmuch 0.15 date parser instead of GNU date.
* Use notmuch reply --reply-to=(sender|all).
* Support single ~d DATE syntax.
* Use the terminal's default background colour.
* Configurable colours.
* Use shell-style word splitting of all commands.
* Automatically perform extra round of shell-quoting for ssh commands.
* Run open part/URL command in background if suffixed by '&'.
* Leave curses when running open part/URL command in foreground.
* Configurable open URL and open part commands.
* Configurable polling period in index view.
* Configurable Drafts/Sent maildir folders in ~/.notmuch-config.
* Fold/unfold header lines when composing messages.
* Parse address lists per RFC 5322.
* Refuse to send message when a header contains an invalid address.
* Encode/decode headers per RFC 2047.
* Encode attachment filenames per RFC 2231.
* Allow non-ASCII characters in addressbook aliases.
* Hide long blocks of quoted text by default.
* Toggle inline display of any selected part with 'Z'.
* Allow inline display of unsupported part if it contains text.
* Improve handling of multipart/* parts.
* Support undo in text entry.
* Support text entry scrolling for lines wider than the screen.
* Support ^W to delete word backwards.
* Expand to common prefix initially when completing.
* Bug fixes and minor improvements.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 10/10] man: update insert documentation

2014-04-16 Thread Peter Wang
Add documentation for the insert --must-index option
and failure exit codes.
---
 doc/man1/notmuch-insert.rst | 24 ++--
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index 2be1a7b..02c516b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -38,16 +38,28 @@ Supported options for **insert** include
 does not exist. Otherwise the folder must already exist for mail
 delivery to succeed.

+``--must-index``
+Succeed only if the message is written to disk, added to the
+notmuch database, and tagged (unless a duplicate message).
+Failure to synchronize tags to maildir flags has no effect.
+Without this option, **insert** succeeds as long as the message
+is written to disk.
+
 EXIT STATUS
 ===

-This command returns exit status 0 if the message was successfully added
-to the mail directory, even if the message could not be indexed and
-added to the notmuch database. In the latter case, a warning will be
-printed to standard error but the message file will be left on disk.
+This command returns exit status 0 on success. On failure, it returns a
+non-zero exit status:
+
+1
+General failure code.
+
+2
+Failed to write the message to disk.

-If the message could not be written to disk then a non-zero exit status
-is returned.
+3
+Failed to index or tag the message (unless a duplicate message),
+and ``--must-index`` was specified.

 SEE ALSO
 
-- 
1.8.4



[PATCH v2 09/10] test: test insert --must-index

2014-04-16 Thread Peter Wang
Test the insert --must-index option.
---
 test/T070-insert.sh | 28 +---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index c576efc..4e289c0 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -21,11 +21,20 @@ gen_insert_msg() {
 test_expect_code 2 "Insert zero-length file" \
 "notmuch insert < /dev/null"

-# This test is a proxy for other errors that may occur while trying to
-# add a message to the notmuch database, e.g. database locked.
-test_expect_code 0 "Insert non-message" \
+test_expect_code 3 "Insert non-message with --must-index on" \
+"echo bad_message | notmuch insert --must-index"
+
+test_begin_subtest "Non-message file should not exist"
+output=$(find "${MAIL_DIR}/cur" "${MAIL_DIR}/new" "${MAIL_DIR}/tmp" -type f 
-print | wc -l)
+test_expect_equal "$output" "0"
+
+test_expect_code 0 "Insert non-message with --must-index off" \
 "echo bad_message | notmuch insert"

+test_begin_subtest "Non-message file should exist"
+output=$(find "${MAIL_DIR}/cur" "${MAIL_DIR}/new" "${MAIL_DIR}/tmp" -type f 
-print | wc -l)
+test_expect_equal "$output" "1"
+
 test_begin_subtest "Database empty so far"
 test_expect_equal "0" "`notmuch count --output=messages '*'`"

@@ -77,6 +86,19 @@ notmuch insert +custom -unread < "$gen_msg_filename"
 output=$(notmuch search --output=messages tag:custom NOT tag:unread)
 test_expect_equal "$output" "id:$gen_msg_id"

+# overlongtag exceeds NOTMUCH_TAG_MAX
+ten=0123456789
+hundred=${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}
+overlongtag=x${hundred}${hundred}
+gen_insert_msg
+test_expect_code 3 "Tagging fails with --must-index on" \
+"notmuch insert --must-index +$overlongtag < $gen_msg_filename"
+
+test_begin_subtest "Tagging fails with --must-index off"
+notmuch insert +$overlongtag < "$gen_msg_filename"
+output=$(notmuch search --output=messages id:$gen_msg_id)
+test_expect_equal "$output" "id:$gen_msg_id"
+
 test_begin_subtest "Insert message with default tags stays in new/"
 gen_insert_msg
 notmuch insert < "$gen_msg_filename"
-- 
1.8.4



[PATCH v2 08/10] cli: add insert --must-index option

2014-04-16 Thread Peter Wang
This option causes notmuch insert to fail (with exit code 3) on failure
to index the message, or failure to set the tags on the message, or if
closing (flushing) the database fails.  Failure to sync tags to flags
has no effect.
---
 notmuch-insert.c | 57 +++-
 1 file changed, 40 insertions(+), 17 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 29d82c9..83257f4 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -31,7 +31,8 @@
 enum {
 INSERT_EXIT_SUCCESS = 0,
 INSERT_EXIT_FAILURE = 1,
-INSERT_EXIT_FAILED_WRITE = 2
+INSERT_EXIT_FAILED_WRITE = 2,
+INSERT_EXIT_FAILED_INDEX = 3
 };

 static volatile sig_atomic_t interrupted;
@@ -298,13 +299,15 @@ copy_stdin (int fdin, int fdout)
 }

 /* Add the specified message file to the notmuch database, applying tags.
- * The file is renamed to encode notmuch tags as maildir flags. */
-static void
+ * If synchronize_flags is set then file is renamed to encode notmuch tags as
+ * maildir flags. */
+static notmuch_bool_t
 add_file_to_database (notmuch_database_t *notmuch, const char *path,
  tag_op_list_t *tag_ops, notmuch_bool_t synchronize_flags)
 {
 notmuch_message_t *message;
 notmuch_status_t status;
+notmuch_status_t sync;

 status = notmuch_database_add_message (notmuch, path, &message);
 switch (status) {
@@ -324,23 +327,28 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 case NOTMUCH_STATUS_LAST_STATUS:
fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
 path, notmuch_status_to_string (status));
-   return;
+   return FALSE;
 }

 if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
/* Don't change tags of an existing message. */
-   if (synchronize_flags) {
-   status = notmuch_message_tags_to_maildir_flags (message);
-   if (status != NOTMUCH_STATUS_SUCCESS)
-   fprintf (stderr, "Error: failed to sync tags to maildir 
flags\n");
-   }
+   status = NOTMUCH_STATUS_SUCCESS;
 } else {
-   tag_op_flag_t flags = synchronize_flags ? TAG_FLAG_MAILDIR_SYNC : 0;
+   status = tag_op_list_apply (message, tag_ops, 0);
+}

-   tag_op_list_apply (message, tag_ops, flags);
+/* Call notmuch_message_tags_to_maildir_flags directly instead of doing it
+ * as part of tag_op_list_apply. For --must-index we want to succeed if
+ * tagging succeeds, but disregard whether synchronizing flags fails. */
+if (status == NOTMUCH_STATUS_SUCCESS && synchronize_flags) {
+   sync = notmuch_message_tags_to_maildir_flags (message);
+   if (sync != NOTMUCH_STATUS_SUCCESS)
+   fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
 }

 notmuch_message_destroy (message);
+
+return (status == NOTMUCH_STATUS_SUCCESS);
 }

 static notmuch_bool_t
@@ -400,15 +408,19 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 char *query_string = NULL;
 const char *folder = NULL;
 notmuch_bool_t create_folder = FALSE;
+notmuch_bool_t must_index = FALSE;
 notmuch_bool_t synchronize_flags;
 const char *maildir;
 char *newpath;
 int opt_index;
 unsigned int i;
+notmuch_bool_t indexed;
+notmuch_status_t status;

 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
{ NOTMUCH_OPT_BOOLEAN, &create_folder, "create-folder", 0, 0 },
+   { NOTMUCH_OPT_BOOLEAN, &must_index, "must-index", 0, 0 },
{ NOTMUCH_OPT_END, 0, 0, 0, 0 }
 };

@@ -485,12 +497,23 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
return INSERT_EXIT_FAILED_WRITE;
 }

-/* Add the message to the index.
- * Even if adding the message to the notmuch database fails,
- * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops,
+/* Add the message to the index. */
+indexed = add_file_to_database (notmuch, newpath, tag_ops,
synchronize_flags);

-notmuch_database_destroy (notmuch);
-return INSERT_EXIT_SUCCESS;
+/* If must_index is FALSE then succeed as the message is on disk.
+ * Otherwise message indexing and tagging must succeed, and the database
+ * must be flushed. Don't flush the database if there was an earlier
+ * error, so as to abandon the transaction (is there a better way?) */
+if (! must_index) {
+   notmuch_database_destroy (notmuch);
+   return INSERT_EXIT_SUCCESS;
+}
+if (indexed) {
+   status = notmuch_database_destroy (notmuch);
+   if (status == NOTMUCH_STATUS_SUCCESS)
+   return INSERT_EXIT_SUCCESS;
+}
+unlink (newpath);
+return INSERT_EXIT_FAILED_INDEX;
 }
-- 
1.8.4



[PATCH v2 07/10] cli: indicate insert failure mode in exit status

2014-04-16 Thread Peter Wang
Make insert return a different exit code, 2, for failure to write the
message file to disk, and exit code 1 for other errors.
---
 notmuch-insert.c| 30 ++
 test/T070-insert.sh |  4 ++--
 2 files changed, 20 insertions(+), 14 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 7db4f73..29d82c9 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -28,6 +28,12 @@
 #include 
 #include 

+enum {
+INSERT_EXIT_SUCCESS = 0,
+INSERT_EXIT_FAILURE = 1,
+INSERT_EXIT_FAILED_WRITE = 2
+};
+
 static volatile sig_atomic_t interrupted;

 static void
@@ -408,7 +414,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])

 opt_index = parse_arguments (argc, argv, options, 1);
 if (opt_index < 0)
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;

 db_path = notmuch_config_get_database_path (config);
 new_tags = notmuch_config_get_new_tags (config, &new_tags_length);
@@ -417,7 +423,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 tag_ops = tag_op_list_create (config);
 if (tag_ops == NULL) {
fprintf (stderr, "Out of memory.\n");
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }
 for (i = 0; i < new_tags_length; i++) {
const char *error_msg;
@@ -426,20 +432,20 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
if (error_msg) {
fprintf (stderr, "Error: tag '%s' in new.tags: %s\n",
 new_tags[i],  error_msg);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}

if (tag_op_list_append (tag_ops, new_tags[i], FALSE))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }

 if (parse_tag_command_line (config, argc - opt_index, argv + opt_index,
&query_string, tag_ops))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;

 if (*query_string != '\0') {
fprintf (stderr, "Error: unexpected query string: %s\n", query_string);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }

 if (folder == NULL) {
@@ -447,17 +453,17 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 } else {
if (! check_folder_name (folder)) {
fprintf (stderr, "Error: bad folder name: %s\n", folder);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
if (! maildir) {
fprintf (stderr, "Out of memory\n");
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
if (create_folder && ! maildir_create_folder (config, maildir)) {
fprintf (stderr, "Error: creating maildir %s: %s\n",
 maildir, strerror (errno));
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
 }

@@ -471,12 +477,12 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])

 if (notmuch_database_open (notmuch_config_get_database_path (config),
   NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;

 /* Write the message to the Maildir new directory. */
 if (! write_message (config, STDIN_FILENO, maildir, &newpath)) {
notmuch_database_destroy (notmuch);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILED_WRITE;
 }

 /* Add the message to the index.
@@ -486,5 +492,5 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
synchronize_flags);

 notmuch_database_destroy (notmuch);
-return EXIT_SUCCESS;
+return INSERT_EXIT_SUCCESS;
 }
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index ea9db07..c576efc 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -18,7 +18,7 @@ gen_insert_msg() {
"[body]=\"insert-message\""
 }

-test_expect_code 1 "Insert zero-length file" \
+test_expect_code 2 "Insert zero-length file" \
 "notmuch insert < /dev/null"

 # This test is a proxy for other errors that may occur while trying to
@@ -137,7 +137,7 @@ output=$(notmuch search --output=messages path:Drafts/cur 
tag:draft NOT tag:unre
 test_expect_equal "$output" "id:$gen_msg_id"

 gen_insert_msg
-test_expect_code 1 "Insert message into non-existent folder" \
+test_expect_code 2 "Insert message into non-existent folder" \
 "notmuch insert --folder=nonesuch < $gen_msg_filename"

 test_begin_subtest "Insert message, create folder"
-- 
1.8.4



[PATCH v2 06/10] cli: refactor insert

2014-04-16 Thread Peter Wang
Change insert_message into write_message and move its responsibilities
for indexing the message into the main function, to simplify the control
flow.
---
 notmuch-insert.c | 63 +++-
 1 file changed, 30 insertions(+), 33 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 6752fc8..7db4f73 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -338,59 +338,48 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 }

 static notmuch_bool_t
-insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
-   const char *dir, tag_op_list_t *tag_ops,
-   notmuch_bool_t synchronize_flags)
+write_message (void *ctx, int fdin, const char *dir, char **newpath)
 {
 char *tmppath;
-char *newpath;
 char *newdir;
 int fdout;
-char *cleanup_path;

-fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
+fdout = maildir_open_tmp_file (ctx, dir, &tmppath, newpath, &newdir);
 if (fdout < 0)
return FALSE;

-cleanup_path = tmppath;
-
-if (! copy_stdin (fdin, fdout))
-   goto FAIL;
+if (! copy_stdin (fdin, fdout)) {
+   close (fdout);
+   unlink (tmppath);
+   return FALSE;
+}

 if (fsync (fdout) != 0) {
fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
-   goto FAIL;
+   close (fdout);
+   unlink (tmppath);
+   return FALSE;
 }

 close (fdout);
-fdout = -1;

 /* Atomically move the new message file from the Maildir 'tmp' directory
  * to the 'new' directory.  We follow the Dovecot recommendation to
  * simply use rename() instead of link() and unlink().
  * See also: http://wiki.dovecot.org/MailboxFormat/Maildir#Mail_delivery
  */
-if (rename (tmppath, newpath) != 0) {
+if (rename (tmppath, *newpath) != 0) {
fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
-   goto FAIL;
+   unlink (tmppath);
+   return FALSE;
 }

-cleanup_path = newpath;
-
-if (! sync_dir (newdir))
-   goto FAIL;
-
-/* Even if adding the message to the notmuch database fails,
- * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops, synchronize_flags);
+if (! sync_dir (newdir)) {
+   unlink (*newpath);
+   return FALSE;
+}

 return TRUE;
-
-  FAIL:
-if (fdout >= 0)
-   close (fdout);
-unlink (cleanup_path);
-return FALSE;
 }

 int
@@ -407,9 +396,9 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 notmuch_bool_t create_folder = FALSE;
 notmuch_bool_t synchronize_flags;
 const char *maildir;
+char *newpath;
 int opt_index;
 unsigned int i;
-notmuch_bool_t ret;

 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
@@ -484,10 +473,18 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
   NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
return EXIT_FAILURE;

-ret = insert_message (config, notmuch, STDIN_FILENO, maildir, tag_ops,
- synchronize_flags);
+/* Write the message to the Maildir new directory. */
+if (! write_message (config, STDIN_FILENO, maildir, &newpath)) {
+   notmuch_database_destroy (notmuch);
+   return EXIT_FAILURE;
+}

-notmuch_database_destroy (notmuch);
+/* Add the message to the index.
+ * Even if adding the message to the notmuch database fails,
+ * the message is on disk and we consider the delivery completed. */
+add_file_to_database (notmuch, newpath, tag_ops,
+   synchronize_flags);

-return ret ? EXIT_SUCCESS : EXIT_FAILURE;
+notmuch_database_destroy (notmuch);
+return EXIT_SUCCESS;
 }
-- 
1.8.4



[PATCH v2 05/10] ruby: handle return status of database close

2014-04-16 Thread Peter Wang
Throw an exception if notmuch_database_destroy fails.
---
 bindings/ruby/database.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/bindings/ruby/database.c b/bindings/ruby/database.c
index e84f726..c03d701 100644
--- a/bindings/ruby/database.c
+++ b/bindings/ruby/database.c
@@ -113,11 +113,13 @@ notmuch_rb_database_open (int argc, VALUE *argv, VALUE 
klass)
 VALUE
 notmuch_rb_database_close (VALUE self)
 {
+notmuch_status_t ret;
 notmuch_database_t *db;

 Data_Get_Notmuch_Database (self, db);
-notmuch_database_destroy (db);
+ret = notmuch_database_destroy (db);
 DATA_PTR (self) = NULL;
+notmuch_rb_status_raise (ret);

 return Qnil;
 }
-- 
1.8.4



[PATCH v2 04/10] go: add return status to database close method

2014-04-16 Thread Peter Wang
Add return status to the Database.Close() method that calls
notmuch_database_destroy.
---
 bindings/go/src/notmuch/notmuch.go | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bindings/go/src/notmuch/notmuch.go 
b/bindings/go/src/notmuch/notmuch.go
index 00bd53a..b9230ad 100644
--- a/bindings/go/src/notmuch/notmuch.go
+++ b/bindings/go/src/notmuch/notmuch.go
@@ -144,8 +144,8 @@ func OpenDatabase(path string, mode DatabaseMode) 
(*Database, Status) {

 /* Close the given notmuch database, freeing all associated
  * resources. See notmuch_database_open. */
-func (self *Database) Close() {
-   C.notmuch_database_destroy(self.db)
+func (self *Database) Close() Status {
+   return Status(C.notmuch_database_destroy(self.db))
 }

 /* Return the database path of the given database.
-- 
1.8.4



[PATCH v2 03/10] python: handle return status of database close and destroy

2014-04-16 Thread Peter Wang
Throw an exception if notmuch_database_close or notmuch_database_destroy
fail.
---
 bindings/python/notmuch/database.py | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 7ddf5cf..5b58e09 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -157,11 +157,13 @@ class Database(object):

 _destroy = nmlib.notmuch_database_destroy
 _destroy.argtypes = [NotmuchDatabaseP]
-_destroy.restype = None
+_destroy.restype = c_uint

 def __del__(self):
 if self._db:
-self._destroy(self._db)
+status = self._destroy(self._db)
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)

 def _assert_db_is_initialized(self):
 """Raises :exc:`NotInitializedError` if self._db is `None`"""
@@ -217,7 +219,7 @@ class Database(object):

 _close = nmlib.notmuch_database_close
 _close.argtypes = [NotmuchDatabaseP]
-_close.restype = None
+_close.restype = c_uint

 def close(self):
 '''
@@ -231,7 +233,9 @@ class Database(object):
 NotmuchError.
 '''
 if self._db:
-self._close(self._db)
+status = self._close(self._db)
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)

 def __enter__(self):
 '''
-- 
1.8.4



[PATCH v2 02/10] lib: bump soname

2014-04-16 Thread Peter Wang
Adding return values to notmuch_database_close and
notmuch_database_destroy may require bumping the soname.
---
 lib/Makefile.local | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/Makefile.local b/lib/Makefile.local
index c56cba9..4120390 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -5,7 +5,7 @@
 # the library interface, (such as the deletion of an API or a major
 # semantic change that breaks formerly functioning code).
 #
-LIBNOTMUCH_VERSION_MAJOR = 3
+LIBNOTMUCH_VERSION_MAJOR = 4

 # The minor version of the library interface. This should be incremented at
 # the time of release for any additions to the library interface,
-- 
1.8.4



[PATCH v2 01/10] lib: add return status to database close and destroy

2014-04-16 Thread Peter Wang
From: Jani Nikula 

notmuch_database_close may fail in Xapian ->flush() or ->close(), so
report the status. Similarly for notmuch_database_destroy which calls
close.

This is required for notmuch insert to report error status if message
indexing failed.
---
 lib/database.cc | 30 --
 lib/notmuch.h   | 15 +--
 2 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 1efb14d..ef7005b 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -774,14 +774,17 @@ notmuch_database_open (const char *path,
 return status;
 }

-void
+notmuch_status_t
 notmuch_database_close (notmuch_database_t *notmuch)
 {
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+
 try {
if (notmuch->xapian_db != NULL &&
notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE)
(static_cast  
(notmuch->xapian_db))->flush ();
 } catch (const Xapian::Error &error) {
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
if (! notmuch->exception_reported) {
fprintf (stderr, "Error: A Xapian exception occurred flushing 
database: %s\n",
 error.get_msg().c_str());
@@ -795,7 +798,9 @@ notmuch_database_close (notmuch_database_t *notmuch)
try {
notmuch->xapian_db->close();
} catch (const Xapian::Error &error) {
-   /* do nothing */
+   /* don't clobber previous error status */
+   if (status == NOTMUCH_STATUS_SUCCESS)
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
}
 }

@@ -809,6 +814,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch->value_range_processor = NULL;
 delete notmuch->date_range_processor;
 notmuch->date_range_processor = NULL;
+
+return status;
 }

 #if HAVE_XAPIAN_COMPACT
@@ -972,8 +979,15 @@ notmuch_database_compact (const char *path,
 }

   DONE:
-if (notmuch)
-   notmuch_database_destroy (notmuch);
+if (notmuch) {
+   notmuch_status_t ret2;
+
+   ret2 = notmuch_database_destroy (notmuch);
+
+   /* don't clobber previous error status */
+   if (ret == NOTMUCH_STATUS_SUCCESS && ret2 != NOTMUCH_STATUS_SUCCESS)
+   ret = ret2;
+}

 talloc_free (local);

@@ -991,11 +1005,15 @@ notmuch_database_compact (unused (const char *path),
 }
 #endif

-void
+notmuch_status_t
 notmuch_database_destroy (notmuch_database_t *notmuch)
 {
-notmuch_database_close (notmuch);
+notmuch_status_t status;
+
+status = notmuch_database_close (notmuch);
 talloc_free (notmuch);
+
+return status;
 }

 const char *
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 350bed8..3c5ec98 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -287,8 +287,16 @@ notmuch_database_open (const char *path,
  *
  * notmuch_database_close can be called multiple times.  Later calls
  * have no effect.
+ *
+ * Return value:
+ *
+ * NOTMUCH_STATUS_SUCCESS: Successfully closed the database.
+ *
+ * NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred; the
+ * database has been closed but there are no guarantees the
+ * changes to the database, if any, have been flushed to disk.
  */
-void
+notmuch_status_t
 notmuch_database_close (notmuch_database_t *database);

 /**
@@ -317,8 +325,11 @@ notmuch_database_compact (const char* path,
 /**
  * Destroy the notmuch database, closing it if necessary and freeing
  * all associated resources.
+ *
+ * Return value as in notmuch_database_close if the database was open;
+ * notmuch_database_destroy itself has no failure modes.
  */
-void
+notmuch_status_t
 notmuch_database_destroy (notmuch_database_t *database);

 /**
-- 
1.8.4



[PATCH v2 00/10] add insert --must-index option

2014-04-16 Thread Peter Wang
Follow up to id:1374365254-13227-1-git-send-email-novalazy at gmail.com
The main changes are to take into account failures during
tagging and flushing of the database.

I took Jani's patch id:1390152046-6509-1-git-send-email-jani at nikula.org
without modification.

The soname bump is included in case it is required.

The python/go/ruby changes are untested.


Jani Nikula (1):
  lib: add return status to database close and destroy

Peter Wang (9):
  lib: bump soname
  python: handle return status of database close and destroy
  go: add return status to database close method
  ruby: handle return status of database close
  cli: refactor insert
  cli: indicate insert failure mode in exit status
  cli: add insert --must-index option
  test: test insert --must-index
  man: update insert documentation

 bindings/go/src/notmuch/notmuch.go  |   4 +-
 bindings/python/notmuch/database.py |  12 ++--
 bindings/ruby/database.c|   4 +-
 doc/man1/notmuch-insert.rst |  24 +--
 lib/Makefile.local  |   2 +-
 lib/database.cc |  30 ++--
 lib/notmuch.h   |  15 +++-
 notmuch-insert.c| 134 +---
 test/T070-insert.sh |  32 +++--
 9 files changed, 176 insertions(+), 81 deletions(-)

-- 
1.8.4



[PATCH v2 09/10] test: test insert --must-index

2014-04-16 Thread Peter Wang
Test the insert --must-index option.
---
 test/T070-insert.sh | 28 +---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index c576efc..4e289c0 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -21,11 +21,20 @@ gen_insert_msg() {
 test_expect_code 2 "Insert zero-length file" \
 "notmuch insert < /dev/null"
 
-# This test is a proxy for other errors that may occur while trying to
-# add a message to the notmuch database, e.g. database locked.
-test_expect_code 0 "Insert non-message" \
+test_expect_code 3 "Insert non-message with --must-index on" \
+"echo bad_message | notmuch insert --must-index"
+
+test_begin_subtest "Non-message file should not exist"
+output=$(find "${MAIL_DIR}/cur" "${MAIL_DIR}/new" "${MAIL_DIR}/tmp" -type f 
-print | wc -l)
+test_expect_equal "$output" "0"
+
+test_expect_code 0 "Insert non-message with --must-index off" \
 "echo bad_message | notmuch insert"
 
+test_begin_subtest "Non-message file should exist"
+output=$(find "${MAIL_DIR}/cur" "${MAIL_DIR}/new" "${MAIL_DIR}/tmp" -type f 
-print | wc -l)
+test_expect_equal "$output" "1"
+
 test_begin_subtest "Database empty so far"
 test_expect_equal "0" "`notmuch count --output=messages '*'`"
 
@@ -77,6 +86,19 @@ notmuch insert +custom -unread < "$gen_msg_filename"
 output=$(notmuch search --output=messages tag:custom NOT tag:unread)
 test_expect_equal "$output" "id:$gen_msg_id"
 
+# overlongtag exceeds NOTMUCH_TAG_MAX
+ten=0123456789
+hundred=${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}${ten}
+overlongtag=x${hundred}${hundred}
+gen_insert_msg
+test_expect_code 3 "Tagging fails with --must-index on" \
+"notmuch insert --must-index +$overlongtag < $gen_msg_filename"
+
+test_begin_subtest "Tagging fails with --must-index off"
+notmuch insert +$overlongtag < "$gen_msg_filename"
+output=$(notmuch search --output=messages id:$gen_msg_id)
+test_expect_equal "$output" "id:$gen_msg_id"
+
 test_begin_subtest "Insert message with default tags stays in new/"
 gen_insert_msg
 notmuch insert < "$gen_msg_filename"
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 10/10] man: update insert documentation

2014-04-16 Thread Peter Wang
Add documentation for the insert --must-index option
and failure exit codes.
---
 doc/man1/notmuch-insert.rst | 24 ++--
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index 2be1a7b..02c516b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -38,16 +38,28 @@ Supported options for **insert** include
 does not exist. Otherwise the folder must already exist for mail
 delivery to succeed.
 
+``--must-index``
+Succeed only if the message is written to disk, added to the
+notmuch database, and tagged (unless a duplicate message).
+Failure to synchronize tags to maildir flags has no effect.
+Without this option, **insert** succeeds as long as the message
+is written to disk.
+
 EXIT STATUS
 ===
 
-This command returns exit status 0 if the message was successfully added
-to the mail directory, even if the message could not be indexed and
-added to the notmuch database. In the latter case, a warning will be
-printed to standard error but the message file will be left on disk.
+This command returns exit status 0 on success. On failure, it returns a
+non-zero exit status:
+
+1
+General failure code.
+
+2
+Failed to write the message to disk.
 
-If the message could not be written to disk then a non-zero exit status
-is returned.
+3
+Failed to index or tag the message (unless a duplicate message),
+and ``--must-index`` was specified.
 
 SEE ALSO
 
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 08/10] cli: add insert --must-index option

2014-04-16 Thread Peter Wang
This option causes notmuch insert to fail (with exit code 3) on failure
to index the message, or failure to set the tags on the message, or if
closing (flushing) the database fails.  Failure to sync tags to flags
has no effect.
---
 notmuch-insert.c | 57 +++-
 1 file changed, 40 insertions(+), 17 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 29d82c9..83257f4 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -31,7 +31,8 @@
 enum {
 INSERT_EXIT_SUCCESS = 0,
 INSERT_EXIT_FAILURE = 1,
-INSERT_EXIT_FAILED_WRITE = 2
+INSERT_EXIT_FAILED_WRITE = 2,
+INSERT_EXIT_FAILED_INDEX = 3
 };
 
 static volatile sig_atomic_t interrupted;
@@ -298,13 +299,15 @@ copy_stdin (int fdin, int fdout)
 }
 
 /* Add the specified message file to the notmuch database, applying tags.
- * The file is renamed to encode notmuch tags as maildir flags. */
-static void
+ * If synchronize_flags is set then file is renamed to encode notmuch tags as
+ * maildir flags. */
+static notmuch_bool_t
 add_file_to_database (notmuch_database_t *notmuch, const char *path,
  tag_op_list_t *tag_ops, notmuch_bool_t synchronize_flags)
 {
 notmuch_message_t *message;
 notmuch_status_t status;
+notmuch_status_t sync;
 
 status = notmuch_database_add_message (notmuch, path, &message);
 switch (status) {
@@ -324,23 +327,28 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 case NOTMUCH_STATUS_LAST_STATUS:
fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
 path, notmuch_status_to_string (status));
-   return;
+   return FALSE;
 }
 
 if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
/* Don't change tags of an existing message. */
-   if (synchronize_flags) {
-   status = notmuch_message_tags_to_maildir_flags (message);
-   if (status != NOTMUCH_STATUS_SUCCESS)
-   fprintf (stderr, "Error: failed to sync tags to maildir 
flags\n");
-   }
+   status = NOTMUCH_STATUS_SUCCESS;
 } else {
-   tag_op_flag_t flags = synchronize_flags ? TAG_FLAG_MAILDIR_SYNC : 0;
+   status = tag_op_list_apply (message, tag_ops, 0);
+}
 
-   tag_op_list_apply (message, tag_ops, flags);
+/* Call notmuch_message_tags_to_maildir_flags directly instead of doing it
+ * as part of tag_op_list_apply. For --must-index we want to succeed if
+ * tagging succeeds, but disregard whether synchronizing flags fails. */
+if (status == NOTMUCH_STATUS_SUCCESS && synchronize_flags) {
+   sync = notmuch_message_tags_to_maildir_flags (message);
+   if (sync != NOTMUCH_STATUS_SUCCESS)
+   fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
 }
 
 notmuch_message_destroy (message);
+
+return (status == NOTMUCH_STATUS_SUCCESS);
 }
 
 static notmuch_bool_t
@@ -400,15 +408,19 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 char *query_string = NULL;
 const char *folder = NULL;
 notmuch_bool_t create_folder = FALSE;
+notmuch_bool_t must_index = FALSE;
 notmuch_bool_t synchronize_flags;
 const char *maildir;
 char *newpath;
 int opt_index;
 unsigned int i;
+notmuch_bool_t indexed;
+notmuch_status_t status;
 
 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
{ NOTMUCH_OPT_BOOLEAN, &create_folder, "create-folder", 0, 0 },
+   { NOTMUCH_OPT_BOOLEAN, &must_index, "must-index", 0, 0 },
{ NOTMUCH_OPT_END, 0, 0, 0, 0 }
 };
 
@@ -485,12 +497,23 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
return INSERT_EXIT_FAILED_WRITE;
 }
 
-/* Add the message to the index.
- * Even if adding the message to the notmuch database fails,
- * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops,
+/* Add the message to the index. */
+indexed = add_file_to_database (notmuch, newpath, tag_ops,
synchronize_flags);
 
-notmuch_database_destroy (notmuch);
-return INSERT_EXIT_SUCCESS;
+/* If must_index is FALSE then succeed as the message is on disk.
+ * Otherwise message indexing and tagging must succeed, and the database
+ * must be flushed. Don't flush the database if there was an earlier
+ * error, so as to abandon the transaction (is there a better way?) */
+if (! must_index) {
+   notmuch_database_destroy (notmuch);
+   return INSERT_EXIT_SUCCESS;
+}
+if (indexed) {
+   status = notmuch_database_destroy (notmuch);
+   if (status == NOTMUCH_STATUS_SUCCESS)
+   return INSERT_EXIT_SUCCESS;
+}
+unlink (newpath);
+return INSERT_EXIT_FAILED_INDEX;
 }
-- 
1.8.4

___
notmuch mailing lis

[PATCH v2 06/10] cli: refactor insert

2014-04-16 Thread Peter Wang
Change insert_message into write_message and move its responsibilities
for indexing the message into the main function, to simplify the control
flow.
---
 notmuch-insert.c | 63 +++-
 1 file changed, 30 insertions(+), 33 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 6752fc8..7db4f73 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -338,59 +338,48 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 }
 
 static notmuch_bool_t
-insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
-   const char *dir, tag_op_list_t *tag_ops,
-   notmuch_bool_t synchronize_flags)
+write_message (void *ctx, int fdin, const char *dir, char **newpath)
 {
 char *tmppath;
-char *newpath;
 char *newdir;
 int fdout;
-char *cleanup_path;
 
-fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
+fdout = maildir_open_tmp_file (ctx, dir, &tmppath, newpath, &newdir);
 if (fdout < 0)
return FALSE;
 
-cleanup_path = tmppath;
-
-if (! copy_stdin (fdin, fdout))
-   goto FAIL;
+if (! copy_stdin (fdin, fdout)) {
+   close (fdout);
+   unlink (tmppath);
+   return FALSE;
+}
 
 if (fsync (fdout) != 0) {
fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
-   goto FAIL;
+   close (fdout);
+   unlink (tmppath);
+   return FALSE;
 }
 
 close (fdout);
-fdout = -1;
 
 /* Atomically move the new message file from the Maildir 'tmp' directory
  * to the 'new' directory.  We follow the Dovecot recommendation to
  * simply use rename() instead of link() and unlink().
  * See also: http://wiki.dovecot.org/MailboxFormat/Maildir#Mail_delivery
  */
-if (rename (tmppath, newpath) != 0) {
+if (rename (tmppath, *newpath) != 0) {
fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
-   goto FAIL;
+   unlink (tmppath);
+   return FALSE;
 }
 
-cleanup_path = newpath;
-
-if (! sync_dir (newdir))
-   goto FAIL;
-
-/* Even if adding the message to the notmuch database fails,
- * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops, synchronize_flags);
+if (! sync_dir (newdir)) {
+   unlink (*newpath);
+   return FALSE;
+}
 
 return TRUE;
-
-  FAIL:
-if (fdout >= 0)
-   close (fdout);
-unlink (cleanup_path);
-return FALSE;
 }
 
 int
@@ -407,9 +396,9 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 notmuch_bool_t create_folder = FALSE;
 notmuch_bool_t synchronize_flags;
 const char *maildir;
+char *newpath;
 int opt_index;
 unsigned int i;
-notmuch_bool_t ret;
 
 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
@@ -484,10 +473,18 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
   NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
return EXIT_FAILURE;
 
-ret = insert_message (config, notmuch, STDIN_FILENO, maildir, tag_ops,
- synchronize_flags);
+/* Write the message to the Maildir new directory. */
+if (! write_message (config, STDIN_FILENO, maildir, &newpath)) {
+   notmuch_database_destroy (notmuch);
+   return EXIT_FAILURE;
+}
 
-notmuch_database_destroy (notmuch);
+/* Add the message to the index.
+ * Even if adding the message to the notmuch database fails,
+ * the message is on disk and we consider the delivery completed. */
+add_file_to_database (notmuch, newpath, tag_ops,
+   synchronize_flags);
 
-return ret ? EXIT_SUCCESS : EXIT_FAILURE;
+notmuch_database_destroy (notmuch);
+return EXIT_SUCCESS;
 }
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 04/10] go: add return status to database close method

2014-04-16 Thread Peter Wang
Add return status to the Database.Close() method that calls
notmuch_database_destroy.
---
 bindings/go/src/notmuch/notmuch.go | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bindings/go/src/notmuch/notmuch.go 
b/bindings/go/src/notmuch/notmuch.go
index 00bd53a..b9230ad 100644
--- a/bindings/go/src/notmuch/notmuch.go
+++ b/bindings/go/src/notmuch/notmuch.go
@@ -144,8 +144,8 @@ func OpenDatabase(path string, mode DatabaseMode) 
(*Database, Status) {
 
 /* Close the given notmuch database, freeing all associated
  * resources. See notmuch_database_open. */
-func (self *Database) Close() {
-   C.notmuch_database_destroy(self.db)
+func (self *Database) Close() Status {
+   return Status(C.notmuch_database_destroy(self.db))
 }
 
 /* Return the database path of the given database.
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 05/10] ruby: handle return status of database close

2014-04-16 Thread Peter Wang
Throw an exception if notmuch_database_destroy fails.
---
 bindings/ruby/database.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/bindings/ruby/database.c b/bindings/ruby/database.c
index e84f726..c03d701 100644
--- a/bindings/ruby/database.c
+++ b/bindings/ruby/database.c
@@ -113,11 +113,13 @@ notmuch_rb_database_open (int argc, VALUE *argv, VALUE 
klass)
 VALUE
 notmuch_rb_database_close (VALUE self)
 {
+notmuch_status_t ret;
 notmuch_database_t *db;
 
 Data_Get_Notmuch_Database (self, db);
-notmuch_database_destroy (db);
+ret = notmuch_database_destroy (db);
 DATA_PTR (self) = NULL;
+notmuch_rb_status_raise (ret);
 
 return Qnil;
 }
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 07/10] cli: indicate insert failure mode in exit status

2014-04-16 Thread Peter Wang
Make insert return a different exit code, 2, for failure to write the
message file to disk, and exit code 1 for other errors.
---
 notmuch-insert.c| 30 ++
 test/T070-insert.sh |  4 ++--
 2 files changed, 20 insertions(+), 14 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 7db4f73..29d82c9 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -28,6 +28,12 @@
 #include 
 #include 
 
+enum {
+INSERT_EXIT_SUCCESS = 0,
+INSERT_EXIT_FAILURE = 1,
+INSERT_EXIT_FAILED_WRITE = 2
+};
+
 static volatile sig_atomic_t interrupted;
 
 static void
@@ -408,7 +414,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 
 opt_index = parse_arguments (argc, argv, options, 1);
 if (opt_index < 0)
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 
 db_path = notmuch_config_get_database_path (config);
 new_tags = notmuch_config_get_new_tags (config, &new_tags_length);
@@ -417,7 +423,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 tag_ops = tag_op_list_create (config);
 if (tag_ops == NULL) {
fprintf (stderr, "Out of memory.\n");
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }
 for (i = 0; i < new_tags_length; i++) {
const char *error_msg;
@@ -426,20 +432,20 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
if (error_msg) {
fprintf (stderr, "Error: tag '%s' in new.tags: %s\n",
 new_tags[i],  error_msg);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
 
if (tag_op_list_append (tag_ops, new_tags[i], FALSE))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }
 
 if (parse_tag_command_line (config, argc - opt_index, argv + opt_index,
&query_string, tag_ops))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 
 if (*query_string != '\0') {
fprintf (stderr, "Error: unexpected query string: %s\n", query_string);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 }
 
 if (folder == NULL) {
@@ -447,17 +453,17 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 } else {
if (! check_folder_name (folder)) {
fprintf (stderr, "Error: bad folder name: %s\n", folder);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
if (! maildir) {
fprintf (stderr, "Out of memory\n");
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
if (create_folder && ! maildir_create_folder (config, maildir)) {
fprintf (stderr, "Error: creating maildir %s: %s\n",
 maildir, strerror (errno));
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
}
 }
 
@@ -471,12 +477,12 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 if (notmuch_database_open (notmuch_config_get_database_path (config),
   NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much))
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILURE;
 
 /* Write the message to the Maildir new directory. */
 if (! write_message (config, STDIN_FILENO, maildir, &newpath)) {
notmuch_database_destroy (notmuch);
-   return EXIT_FAILURE;
+   return INSERT_EXIT_FAILED_WRITE;
 }
 
 /* Add the message to the index.
@@ -486,5 +492,5 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
synchronize_flags);
 
 notmuch_database_destroy (notmuch);
-return EXIT_SUCCESS;
+return INSERT_EXIT_SUCCESS;
 }
diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index ea9db07..c576efc 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -18,7 +18,7 @@ gen_insert_msg() {
"[body]=\"insert-message\""
 }
 
-test_expect_code 1 "Insert zero-length file" \
+test_expect_code 2 "Insert zero-length file" \
 "notmuch insert < /dev/null"
 
 # This test is a proxy for other errors that may occur while trying to
@@ -137,7 +137,7 @@ output=$(notmuch search --output=messages path:Drafts/cur 
tag:draft NOT tag:unre
 test_expect_equal "$output" "id:$gen_msg_id"
 
 gen_insert_msg
-test_expect_code 1 "Insert message into non-existent folder" \
+test_expect_code 2 "Insert message into non-existent folder" \
 "notmuch insert --folder=nonesuch < $gen_msg_filename"
 
 test_begin_subtest "Insert message, create folder"
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 03/10] python: handle return status of database close and destroy

2014-04-16 Thread Peter Wang
Throw an exception if notmuch_database_close or notmuch_database_destroy
fail.
---
 bindings/python/notmuch/database.py | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 7ddf5cf..5b58e09 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -157,11 +157,13 @@ class Database(object):
 
 _destroy = nmlib.notmuch_database_destroy
 _destroy.argtypes = [NotmuchDatabaseP]
-_destroy.restype = None
+_destroy.restype = c_uint
 
 def __del__(self):
 if self._db:
-self._destroy(self._db)
+status = self._destroy(self._db)
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)
 
 def _assert_db_is_initialized(self):
 """Raises :exc:`NotInitializedError` if self._db is `None`"""
@@ -217,7 +219,7 @@ class Database(object):
 
 _close = nmlib.notmuch_database_close
 _close.argtypes = [NotmuchDatabaseP]
-_close.restype = None
+_close.restype = c_uint
 
 def close(self):
 '''
@@ -231,7 +233,9 @@ class Database(object):
 NotmuchError.
 '''
 if self._db:
-self._close(self._db)
+status = self._close(self._db)
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)
 
 def __enter__(self):
 '''
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 01/10] lib: add return status to database close and destroy

2014-04-16 Thread Peter Wang
From: Jani Nikula 

notmuch_database_close may fail in Xapian ->flush() or ->close(), so
report the status. Similarly for notmuch_database_destroy which calls
close.

This is required for notmuch insert to report error status if message
indexing failed.
---
 lib/database.cc | 30 --
 lib/notmuch.h   | 15 +--
 2 files changed, 37 insertions(+), 8 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 1efb14d..ef7005b 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -774,14 +774,17 @@ notmuch_database_open (const char *path,
 return status;
 }
 
-void
+notmuch_status_t
 notmuch_database_close (notmuch_database_t *notmuch)
 {
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+
 try {
if (notmuch->xapian_db != NULL &&
notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE)
(static_cast  
(notmuch->xapian_db))->flush ();
 } catch (const Xapian::Error &error) {
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
if (! notmuch->exception_reported) {
fprintf (stderr, "Error: A Xapian exception occurred flushing 
database: %s\n",
 error.get_msg().c_str());
@@ -795,7 +798,9 @@ notmuch_database_close (notmuch_database_t *notmuch)
try {
notmuch->xapian_db->close();
} catch (const Xapian::Error &error) {
-   /* do nothing */
+   /* don't clobber previous error status */
+   if (status == NOTMUCH_STATUS_SUCCESS)
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
}
 }
 
@@ -809,6 +814,8 @@ notmuch_database_close (notmuch_database_t *notmuch)
 notmuch->value_range_processor = NULL;
 delete notmuch->date_range_processor;
 notmuch->date_range_processor = NULL;
+
+return status;
 }
 
 #if HAVE_XAPIAN_COMPACT
@@ -972,8 +979,15 @@ notmuch_database_compact (const char *path,
 }
 
   DONE:
-if (notmuch)
-   notmuch_database_destroy (notmuch);
+if (notmuch) {
+   notmuch_status_t ret2;
+
+   ret2 = notmuch_database_destroy (notmuch);
+
+   /* don't clobber previous error status */
+   if (ret == NOTMUCH_STATUS_SUCCESS && ret2 != NOTMUCH_STATUS_SUCCESS)
+   ret = ret2;
+}
 
 talloc_free (local);
 
@@ -991,11 +1005,15 @@ notmuch_database_compact (unused (const char *path),
 }
 #endif
 
-void
+notmuch_status_t
 notmuch_database_destroy (notmuch_database_t *notmuch)
 {
-notmuch_database_close (notmuch);
+notmuch_status_t status;
+
+status = notmuch_database_close (notmuch);
 talloc_free (notmuch);
+
+return status;
 }
 
 const char *
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 350bed8..3c5ec98 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -287,8 +287,16 @@ notmuch_database_open (const char *path,
  *
  * notmuch_database_close can be called multiple times.  Later calls
  * have no effect.
+ *
+ * Return value:
+ *
+ * NOTMUCH_STATUS_SUCCESS: Successfully closed the database.
+ *
+ * NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred; the
+ * database has been closed but there are no guarantees the
+ * changes to the database, if any, have been flushed to disk.
  */
-void
+notmuch_status_t
 notmuch_database_close (notmuch_database_t *database);
 
 /**
@@ -317,8 +325,11 @@ notmuch_database_compact (const char* path,
 /**
  * Destroy the notmuch database, closing it if necessary and freeing
  * all associated resources.
+ *
+ * Return value as in notmuch_database_close if the database was open;
+ * notmuch_database_destroy itself has no failure modes.
  */
-void
+notmuch_status_t
 notmuch_database_destroy (notmuch_database_t *database);
 
 /**
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 02/10] lib: bump soname

2014-04-16 Thread Peter Wang
Adding return values to notmuch_database_close and
notmuch_database_destroy may require bumping the soname.
---
 lib/Makefile.local | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/Makefile.local b/lib/Makefile.local
index c56cba9..4120390 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -5,7 +5,7 @@
 # the library interface, (such as the deletion of an API or a major
 # semantic change that breaks formerly functioning code).
 #
-LIBNOTMUCH_VERSION_MAJOR = 3
+LIBNOTMUCH_VERSION_MAJOR = 4
 
 # The minor version of the library interface. This should be incremented at
 # the time of release for any additions to the library interface,
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 00/10] add insert --must-index option

2014-04-16 Thread Peter Wang
Follow up to id:1374365254-13227-1-git-send-email-noval...@gmail.com
The main changes are to take into account failures during
tagging and flushing of the database.

I took Jani's patch id:1390152046-6509-1-git-send-email-j...@nikula.org
without modification.

The soname bump is included in case it is required.

The python/go/ruby changes are untested.


Jani Nikula (1):
  lib: add return status to database close and destroy

Peter Wang (9):
  lib: bump soname
  python: handle return status of database close and destroy
  go: add return status to database close method
  ruby: handle return status of database close
  cli: refactor insert
  cli: indicate insert failure mode in exit status
  cli: add insert --must-index option
  test: test insert --must-index
  man: update insert documentation

 bindings/go/src/notmuch/notmuch.go  |   4 +-
 bindings/python/notmuch/database.py |  12 ++--
 bindings/ruby/database.c|   4 +-
 doc/man1/notmuch-insert.rst |  24 +--
 lib/Makefile.local  |   2 +-
 lib/database.cc |  30 ++--
 lib/notmuch.h   |  15 +++-
 notmuch-insert.c| 134 +---
 test/T070-insert.sh |  32 +++--
 9 files changed, 176 insertions(+), 81 deletions(-)

-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] show: add In-reply-to, References fields to structured formats

2014-01-13 Thread Peter Wang
On Sun, 12 Jan 2014 17:31:32 +0200, Jani Nikula  wrote:
> On Sun, 12 Jan 2014, Peter Wang  wrote:
> > This is useful when 'show' is used to retrieve a draft message
> > which is in reply to another message.
> 
> I'd like to know more about *how* this is useful. Indeed the whole big
> picture about supporting draft or postponed messages is foggy. I would
> like to have some clarity about that first.
> 
> Apparently the idea is to index draft messages. How do you save them?
> What guarantees are there that they look enough like real messages that
> they get indexed? Does this patch mean that the idea is to resume draft
> messages using the structured formats instead of opening the raw file?
> Why?  What do you plan to do with the saved draft? And so on...

I didn't realise storing drafts in your Maildir was unusual.

A draft message (including its attachments) may be added to a Maildir
folder with notmuch insert or notmuch-deliver, and then tagged.
The message must look enough like a real message for indexing but it is
not hard for an email client to arrange -- the message is ostensibly to
be sent anyway.  Unlike saving in a local file, keeping the draft in a
central mail store means it will be accessible anywhere that you can
access the rest of your mail.

Draft messages may be retrieved for previewing or resumption with
notmuch show, like other messages.  The raw file may be on another
machine so it is not always possible to read it directly.  The advantage
of using a structured output format instead of the raw output format is
the same as for any other message -- notmuch has already parsed it for
you.  The disadvantage is that notmuch show's structured output only
presents a subset of headers, so other headers will be lost.
(This suggests an alternative change if notmuch maintainers are
receptive.)

Indexed draft messages will, by default, show up in normal display
so they will need to be hidden with notmuch search exclusions.
Drafts may tagged with 'delete' for eventual removal.

Peter


[PATCH] show: add In-reply-to, References fields to structured formats

2014-01-12 Thread Peter Wang
This is useful when 'show' is used to retrieve a draft message
which is in reply to another message.
---
 devel/schemata  |  9 -
 notmuch-show.c  | 16 
 test/thread-replies |  7 +++
 3 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/devel/schemata b/devel/schemata
index 41dc4a6..dd41217 100644
--- a/devel/schemata
+++ b/devel/schemata
@@ -14,7 +14,7 @@ are interleaved. Keys are printed as keywords (symbols 
preceded by a
 colon), e.g. (:id "123" :time 54321 :from "foobar"). Null is printed as
 nil, true as t and false as nil.

-This is version 2 of the structured output format.
+This is version 3 of the structured output format.

 Version history
 ---
@@ -26,6 +26,9 @@ v1
 v2
 - Added the thread_summary.query field.

+v3
+- Added headers.in-reply-to and headers.references fields.
+
 Common non-terminals
 

@@ -105,6 +108,10 @@ headers = {
 Cc?:string,
 Bcc?:   string,
 Reply-To?:  string,
+# Added in schema version 3.
+In-reply-to?:   string,
+# Added in schema version 3.
+References?:string,
 Date:   string
 }

diff --git a/notmuch-show.c b/notmuch-show.c
index c07f887..774ba44 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -222,6 +222,8 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
*message,
 InternetAddressList *recipients;
 const char *recipients_string;
 const char *reply_to_string;
+const char *in_reply_to_string;
+const char *references_string;

 sp->begin_map (sp);

@@ -258,13 +260,19 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
*message,
sp->string (sp, reply_to_string);
 }

-if (reply) {
+in_reply_to_string = g_mime_object_get_header (GMIME_OBJECT (message), 
"In-reply-to");
+if (in_reply_to_string || reply) {
sp->map_key (sp, "In-reply-to");
-   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"In-reply-to"));
+   sp->string (sp, in_reply_to_string);
+}

+references_string = g_mime_object_get_header (GMIME_OBJECT (message), 
"References");
+if (references_string || reply) {
sp->map_key (sp, "References");
-   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"References"));
-} else {
+   sp->string (sp, references_string);
+}
+
+if (! reply) {
sp->map_key (sp, "Date");
sp->string (sp, g_mime_message_get_date_as_string (message));
 }
diff --git a/test/thread-replies b/test/thread-replies
index eeb70d0..9d4b379 100755
--- a/test/thread-replies
+++ b/test/thread-replies
@@ -39,6 +39,8 @@ expected='[[[{"id": "foo at one.com",
  "tags": ["inbox", "unread"], "headers": {"Subject": "Re: one",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "mumble",
+ "References": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"},
  "body": [{"id": 1, "content-type": "text/plain",
  "content": "This is just a test message (#2)\n"}]}, []]'
@@ -68,6 +70,8 @@ expected='[[[{"id": "foo at two.com",
  "headers": {"Subject": "Re: two",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
+ "References": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"},
  "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#4)\n"}]},
@@ -95,6 +99,7 @@ expected='[[[{"id": "foo at three.com", "match": true, 
"excluded": false,
  "headers": {"Subject": "Re: three",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#6)\n"}]},
  []]'
@@ -124,6 +129,8 @@ expected='[[[{"id": "foo at four.com", "match": true, 
"excluded": false,
  "headers": {"Subject": "neither",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
+ "References": " ",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#9)\n"}]},
  [], [[{"id": "bar at four.com", "match": true, "excluded": false,
-- 
1.8.4



Re: [PATCH] show: add In-reply-to, References fields to structured formats

2014-01-12 Thread Peter Wang
On Sun, 12 Jan 2014 17:31:32 +0200, Jani Nikula  wrote:
> On Sun, 12 Jan 2014, Peter Wang  wrote:
> > This is useful when 'show' is used to retrieve a draft message
> > which is in reply to another message.
> 
> I'd like to know more about *how* this is useful. Indeed the whole big
> picture about supporting draft or postponed messages is foggy. I would
> like to have some clarity about that first.
> 
> Apparently the idea is to index draft messages. How do you save them?
> What guarantees are there that they look enough like real messages that
> they get indexed? Does this patch mean that the idea is to resume draft
> messages using the structured formats instead of opening the raw file?
> Why?  What do you plan to do with the saved draft? And so on...

I didn't realise storing drafts in your Maildir was unusual.

A draft message (including its attachments) may be added to a Maildir
folder with notmuch insert or notmuch-deliver, and then tagged.
The message must look enough like a real message for indexing but it is
not hard for an email client to arrange -- the message is ostensibly to
be sent anyway.  Unlike saving in a local file, keeping the draft in a
central mail store means it will be accessible anywhere that you can
access the rest of your mail.

Draft messages may be retrieved for previewing or resumption with
notmuch show, like other messages.  The raw file may be on another
machine so it is not always possible to read it directly.  The advantage
of using a structured output format instead of the raw output format is
the same as for any other message -- notmuch has already parsed it for
you.  The disadvantage is that notmuch show's structured output only
presents a subset of headers, so other headers will be lost.
(This suggests an alternative change if notmuch maintainers are
receptive.)

Indexed draft messages will, by default, show up in normal display
so they will need to be hidden with notmuch search exclusions.
Drafts may tagged with 'delete' for eventual removal.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] show: add In-reply-to, References fields to structured formats

2014-01-11 Thread Peter Wang
This is useful when 'show' is used to retrieve a draft message
which is in reply to another message.
---
 devel/schemata  |  9 -
 notmuch-show.c  | 16 
 test/thread-replies |  7 +++
 3 files changed, 27 insertions(+), 5 deletions(-)

diff --git a/devel/schemata b/devel/schemata
index 41dc4a6..dd41217 100644
--- a/devel/schemata
+++ b/devel/schemata
@@ -14,7 +14,7 @@ are interleaved. Keys are printed as keywords (symbols 
preceded by a
 colon), e.g. (:id "123" :time 54321 :from "foobar"). Null is printed as
 nil, true as t and false as nil.
 
-This is version 2 of the structured output format.
+This is version 3 of the structured output format.
 
 Version history
 ---
@@ -26,6 +26,9 @@ v1
 v2
 - Added the thread_summary.query field.
 
+v3
+- Added headers.in-reply-to and headers.references fields.
+
 Common non-terminals
 
 
@@ -105,6 +108,10 @@ headers = {
 Cc?:string,
 Bcc?:   string,
 Reply-To?:  string,
+# Added in schema version 3.
+In-reply-to?:   string,
+# Added in schema version 3.
+References?:string,
 Date:   string
 }
 
diff --git a/notmuch-show.c b/notmuch-show.c
index c07f887..774ba44 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -222,6 +222,8 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
*message,
 InternetAddressList *recipients;
 const char *recipients_string;
 const char *reply_to_string;
+const char *in_reply_to_string;
+const char *references_string;
 
 sp->begin_map (sp);
 
@@ -258,13 +260,19 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
*message,
sp->string (sp, reply_to_string);
 }
 
-if (reply) {
+in_reply_to_string = g_mime_object_get_header (GMIME_OBJECT (message), 
"In-reply-to");
+if (in_reply_to_string || reply) {
sp->map_key (sp, "In-reply-to");
-   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"In-reply-to"));
+   sp->string (sp, in_reply_to_string);
+}
 
+references_string = g_mime_object_get_header (GMIME_OBJECT (message), 
"References");
+if (references_string || reply) {
sp->map_key (sp, "References");
-   sp->string (sp, g_mime_object_get_header (GMIME_OBJECT (message), 
"References"));
-} else {
+   sp->string (sp, references_string);
+}
+
+if (! reply) {
sp->map_key (sp, "Date");
sp->string (sp, g_mime_message_get_date_as_string (message));
 }
diff --git a/test/thread-replies b/test/thread-replies
index eeb70d0..9d4b379 100755
--- a/test/thread-replies
+++ b/test/thread-replies
@@ -39,6 +39,8 @@ expected='[[[{"id": "f...@one.com",
  "tags": ["inbox", "unread"], "headers": {"Subject": "Re: one",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "mumble",
+ "References": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"},
  "body": [{"id": 1, "content-type": "text/plain",
  "content": "This is just a test message (#2)\n"}]}, []]'
@@ -68,6 +70,8 @@ expected='[[[{"id": "f...@two.com",
  "headers": {"Subject": "Re: two",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
+ "References": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"},
  "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#4)\n"}]},
@@ -95,6 +99,7 @@ expected='[[[{"id": "f...@three.com", "match": true, 
"excluded": false,
  "headers": {"Subject": "Re: three",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#6)\n"}]},
  []]'
@@ -124,6 +129,8 @@ expected='[[[{"id": "f...@four.com", "match": true, 
"excluded": false,
  "headers": {"Subject": "neither",
  "From": "Notmuch Test Suite ",
  "To": "Notmuch Test Suite ",
+ "In-reply-to": "",
+ "References": " ",
  "Date": "Fri, 05 Jan 2001 15:43:57 +"}, "body": [{"id": 1,
  "content-type": "text/plain", "content": "This is just a test message 
(#9)\n"}]},
  [], [[{"id": "b...@four.com", "match": true, "excluded": false,
-- 
1.8.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/3] cli: add insert --must-index option

2013-09-12 Thread Peter Wang
On Tue, 10 Sep 2013 09:06:00 +0100, Mark Walters  
wrote:
> 
> Alternatively maybe add notmuch_database_destroy_with_flush or something
> which does give a return value. notmuch_database_close is only called 3
> times and notmuch_database_destroy lots of times so changing close is
> much less intrusive than changing destroy. But I don't know whether we
> would break any  bindings or external programs etc.
> 
> What do you think?

I think notmuch_database_close and notmuch_database_destroy should
return the status, and we update the three language bindings
and bump the soname.

Peter


Re: [PATCH 1/3] cli: add insert --must-index option

2013-09-11 Thread Peter Wang
On Tue, 10 Sep 2013 09:06:00 +0100, Mark Walters  
wrote:
> 
> Alternatively maybe add notmuch_database_destroy_with_flush or something
> which does give a return value. notmuch_database_close is only called 3
> times and notmuch_database_destroy lots of times so changing close is
> much less intrusive than changing destroy. But I don't know whether we
> would break any  bindings or external programs etc.
> 
> What do you think?

I think notmuch_database_close and notmuch_database_destroy should
return the status, and we update the three language bindings
and bump the soname.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/3] cli: add insert --must-index option

2013-07-27 Thread Peter Wang
On Sun, 21 Jul 2013 09:31:28 +0100, Mark Walters  
wrote:
> 
> Do you have a particular use case where indexing is required but tagging
> is not? For my uses I would prefer failing if either indexing or tagging
> failed. (My use being postponing messages; If they don't get the
> postponed tag they could be hard to find)

You're right.

What about a failure to sync tags to maildir flags?

I now noticed that database modifications aren't flushed until the
notmuch_database_destroy call (right?), which has no return value and
failure of which is silently ignored.  That's acceptable in the default
mode, but with --must-index the failure should be reported (and the
file deleted).

Peter


[PATCH 3/3] test: test insert --must-index

2013-07-21 Thread Peter Wang
Test the new insert --must-index option.
---
 test/insert | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/insert b/test/insert
index 021edb6..938753e 100755
--- a/test/insert
+++ b/test/insert
@@ -26,6 +26,9 @@ test_expect_code 1 "Insert zero-length file" \
 test_expect_code 0 "Insert non-message" \
 "echo bad_message | notmuch insert"

+test_expect_code 2 "Insert non-message, --must-index" \
+"echo bad_message | notmuch insert --must-index"
+
 test_begin_subtest "Database empty so far"
 test_expect_equal "0" "`notmuch count --output=messages '*'`"

-- 
1.7.12.1



[PATCH 2/3] man: document insert --must-index

2013-07-21 Thread Peter Wang
Add documentation for the new option.
---
 man/man1/notmuch-insert.1 | 39 +--
 1 file changed, 33 insertions(+), 6 deletions(-)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index d5202ac..44c18d8 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -55,15 +55,42 @@ Otherwise the folder must already exist for mail
 delivery to succeed.

 .RE
+
+.RS 4
+.TP 4
+.B "--must-index"
+
+The default behaviour of
+.B notmuch insert
+is to succeed if the message is added to the mail directory, even if
+the message could not be indexed and added to the notmuch database.
+In the latter case, a warning will be printed to standard error but
+the message file will be left on disk.
+
+The
+.B "--must-index"
+option causes
+.B notmuch insert
+to fail if the message could not be added to the notmuch database,
+and the newly-created message file deleted.
+
+Failure to set tags does not consitute a failure of the overall
+command.
+
+.RE
 .SH EXIT STATUS

-This command returns exit status 0 if the message was successfully
-added to the mail directory, even if the message could not be indexed
-and added to the notmuch database.  In the latter case, a warning will
-be printed to standard error but the message file will be left on disk.
+This command returns exit status 0 on success.
+On failure, it returns a non-zero exit status.

-If the message could not be written to disk then a non-zero exit
-status is returned.
+.TP
+.B 1
+Failed to write the message to disk.
+.TP
+.B 2
+Failed to index the message, and
+.B "--must-index"
+was specified.

 .RE
 .SH SEE ALSO
-- 
1.7.12.1



[PATCH 1/3] cli: add insert --must-index option

2013-07-21 Thread Peter Wang
This option causes notmuch insert to fail as a whole if the message
failed to be added to the notmuch database.  The new message file
will be deleted from disk, and a distinct status code (2) returned.
---
 notmuch-insert.c | 76 ++--
 1 file changed, 46 insertions(+), 30 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 2207b1e..505b647 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -28,6 +28,10 @@
 #include 
 #include 

+#define INSERT_EXIT_SUCCESS0
+#define INSERT_EXIT_FAIL_WRITE 1
+#define INSERT_EXIT_FAIL_INDEX 2
+
 static volatile sig_atomic_t interrupted;

 static void
@@ -293,12 +297,13 @@ copy_stdin (int fdin, int fdout)

 /* Add the specified message file to the notmuch database, applying tags.
  * The file is renamed to encode notmuch tags as maildir flags. */
-static void
+static notmuch_status_t
 add_file_to_database (notmuch_database_t *notmuch, const char *path,
  tag_op_list_t *tag_ops)
 {
 notmuch_message_t *message;
 notmuch_status_t status;
+notmuch_status_t sync;

 status = notmuch_database_add_message (notmuch, path, &message);
 switch (status) {
@@ -318,47 +323,52 @@ add_file_to_database (notmuch_database_t *notmuch, const 
char *path,
 case NOTMUCH_STATUS_LAST_STATUS:
fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
 path, notmuch_status_to_string (status));
-   return;
+   return status;
 }

 if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
/* Don't change tags of an existing message. */
-   status = notmuch_message_tags_to_maildir_flags (message);
-   if (status != NOTMUCH_STATUS_SUCCESS)
+   sync = notmuch_message_tags_to_maildir_flags (message);
+   if (sync != NOTMUCH_STATUS_SUCCESS)
fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
 } else {
tag_op_list_apply (message, tag_ops, TAG_FLAG_MAILDIR_SYNC);
 }

 notmuch_message_destroy (message);
+
+return status;
 }

-static notmuch_bool_t
+static int
 insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
-   const char *dir, tag_op_list_t *tag_ops)
+   const char *dir, tag_op_list_t *tag_ops,
+   notmuch_bool_t must_index)
 {
 char *tmppath;
 char *newpath;
 char *newdir;
 int fdout;
-char *cleanup_path;
+notmuch_status_t status;

 fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
 if (fdout < 0)
-   return FALSE;
+   return INSERT_EXIT_FAIL_WRITE;

-cleanup_path = tmppath;
-
-if (! copy_stdin (fdin, fdout))
-   goto FAIL;
+if (! copy_stdin (fdin, fdout)) {
+   close (fdout);
+   unlink (tmppath);
+   return INSERT_EXIT_FAIL_WRITE;
+}

 if (fsync (fdout) != 0) {
fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
-   goto FAIL;
+   close (fdout);
+   unlink (tmppath);
+   return INSERT_EXIT_FAIL_WRITE;
 }

 close (fdout);
-fdout = -1;

 /* Atomically move the new message file from the Maildir 'tmp' directory
  * to the 'new' directory.  We follow the Dovecot recommendation to
@@ -367,25 +377,28 @@ insert_message (void *ctx, notmuch_database_t *notmuch, 
int fdin,
  */
 if (rename (tmppath, newpath) != 0) {
fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
-   goto FAIL;
+   unlink (tmppath);
+   return INSERT_EXIT_FAIL_WRITE;
 }

-cleanup_path = newpath;
-
-if (! sync_dir (newdir))
-   goto FAIL;
+if (! sync_dir (newdir)) {
+   unlink (newpath);
+   return INSERT_EXIT_FAIL_WRITE;
+}

-/* Even if adding the message to the notmuch database fails,
- * the message is on disk and we consider the delivery completed. */
-add_file_to_database (notmuch, newpath, tag_ops);
+status = add_file_to_database (notmuch, newpath, tag_ops);

-return TRUE;
+/* If must_index is TRUE, then indexing must succeed.  Otherwise, we
+ * consider the delivery completed as long as the message is on disk. */
+if (must_index &&
+   status != NOTMUCH_STATUS_SUCCESS &&
+   status != NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID)
+{
+   unlink (newpath);
+   return INSERT_EXIT_FAIL_INDEX;
+}

-  FAIL:
-if (fdout >= 0)
-   close (fdout);
-unlink (cleanup_path);
-return FALSE;
+return INSERT_EXIT_SUCCESS;
 }

 int
@@ -400,6 +413,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 char *query_string = NULL;
 const char *folder = NULL;
 notmuch_bool_t create_folder = FALSE;
+notmuch_bool_t must_index = FALSE;
 const char *maildir;
 int opt_index;
 unsigned int i;
@@ -408,6 +422,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 notmuch_opt_desc_t options[] = {
{ NOTMU

[PATCH] NEWS: announce insert command

2013-07-06 Thread Peter Wang
---
 NEWS | 5 +
 1 file changed, 5 insertions(+)

diff --git a/NEWS b/NEWS
index 811e5dd..de5bb83 100644
--- a/NEWS
+++ b/NEWS
@@ -49,6 +49,11 @@ Bash command-line completion
   `notmuch config`. The new completion support depends on the
   bash-completion package.

+Support for delivering messages to Maildir
+
+  There is a new command `insert` that adds a message to a Maildir
+  folder and notmuch index.
+
 Vim Front-End
 -

-- 
1.7.12.1



bower (git) to show Inbox (folder)

2013-07-01 Thread Peter Wang
On Sun, 30 Jun 2013 15:52:33 +0200, Marius  wrote:
> Hi!
> 
> Today I ditched mutt(-kz) in favour of a notmuch/isync setup. I
> compiled bower, because it's able to use a remote notmuch instance.
> My issue is a follows: I start bower, and I see an unsorted selection
> of mails (300 newest).
> How can I navigate by folders/tags?
> 
> - A search like folder:Inbox shows 0 mails. Also I disable the limit
> of 300 mails... with no effect on the search. Could someone enlighten
> me on the usability here a little?

Hi,

folder:Inbox would search for messages in a sub-directory 'Inbox' under
your Maildir directory.  You could search for 'tag:inbox' if new
messages are tagged as such by the new.tags configuration option.
However, bower doesn't have a quick way to remove the 'inbox' tag,
though it could be added easily enough.

I normally leave bower open on the last few days' mail, and only change
the search query if I'm looking for something in particular.  I use
folders and tags sparingly.

Peter


[PATCH v7b] cli: add insert command

2013-06-23 Thread Peter Wang
The notmuch insert command reads a message from standard input,
writes it to a Maildir folder, and then incorporates the message into
the notmuch database.  Essentially it moves the functionality of
notmuch-deliver into notmuch.

Though it could be used as an alternative to notmuch new, the reason
I want this is to allow my notmuch frontend to add postponed or sent
messages to the mail store and notmuch database, without resorting to
another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
---
 Makefile.local   |   1 +
 notmuch-client.h |   3 +
 notmuch-insert.c | 337 +++
 notmuch.c|   2 +
 4 files changed, 343 insertions(+)
 create mode 100644 notmuch-insert.c

diff --git a/Makefile.local b/Makefile.local
index 644623f..84043fe 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -261,6 +261,7 @@ notmuch_client_srcs =   \
notmuch-config.c\
notmuch-count.c \
notmuch-dump.c  \
+   notmuch-insert.c\
notmuch-new.c   \
notmuch-reply.c \
notmuch-restore.c   \
diff --git a/notmuch-client.h b/notmuch-client.h
index 4a3c7ac..dfe81e6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -181,6 +181,9 @@ int
 notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]);

 int
+notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]);
+
+int
 notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);

 int
diff --git a/notmuch-insert.c b/notmuch-insert.c
new file mode 100644
index 000..1228afa
--- /dev/null
+++ b/notmuch-insert.c
@@ -0,0 +1,337 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ? 2013 Peter Wang
+ *
+ * Based in part on notmuch-deliver
+ * Copyright ? 2010 Ali Polatel
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Peter Wang 
+ */
+
+#include "notmuch-client.h"
+#include "tag-util.h"
+
+#include 
+#include 
+#include 
+
+static volatile sig_atomic_t interrupted;
+
+static void
+handle_sigint (unused (int sig))
+{
+static char msg[] = "Stopping... \n";
+
+/* This write is "opportunistic", so it's okay to ignore the
+ * result.  It is not required for correctness, and if it does
+ * fail or produce a short write, we want to get out of the signal
+ * handler as quickly as possible, not retry it. */
+IGNORE_RESULT (write (2, msg, sizeof (msg) - 1));
+interrupted = 1;
+}
+
+/* Like gethostname but guarantees that a null-terminated hostname is
+ * returned, even if it has to make one up. Invalid characters are
+ * substituted such that the hostname can be used within a filename.
+ */
+static void
+safe_gethostname (char *hostname, size_t len)
+{
+char *p;
+
+if (gethostname (hostname, len) == -1) {
+   strncpy (hostname, "unknown", len);
+}
+hostname[len - 1] = '\0';
+
+for (p = hostname; *p != '\0'; p++) {
+   if (*p == '/' || *p == ':')
+   *p = '_';
+}
+}
+
+/* Call fsync() on a directory path. */
+static notmuch_bool_t
+sync_dir (const char *dir)
+{
+notmuch_bool_t ret;
+int fd;
+
+fd = open (dir, O_RDONLY);
+if (fd == -1) {
+   fprintf (stderr, "Error: open() dir failed: %s\n", strerror (errno));
+   return FALSE;
+}
+ret = (fsync (fd) == 0);
+if (! ret) {
+   fprintf (stderr, "Error: fsync() dir failed: %s\n", strerror (errno));
+}
+close (fd);
+return ret;
+}
+
+/* Open a unique file in the 'tmp' sub-directory of dir.
+ * Returns the file descriptor on success, or -1 on failure.
+ * On success, file paths for the message in the 'tmp' and 'new'
+ * directories are returned via tmppath and newpath,
+ * and the path of the 'new' directory itself in newdir. */
+static int
+maildir_open_tmp_file (void *ctx, const char *dir,
+  char **tmppath, char **newpath, char **newdir)
+{
+pid_t pid;
+char hostname[256];
+struct timeval tv;
+char *filename;
+int fd = -1;
+
+/* We follow the Dovecot file name generation algorithm. */
+pid = getpid ();
+safe_gethostname (hostname, siz

[PATCH v7 03/12] cli: add insert command

2013-06-23 Thread Peter Wang
On Sun, 23 Jun 2013 07:42:49 +0100, Mark Walters  
wrote:
> 
> This is a +1 modulo one small bug (I think) I found below. I am happy to
> delay the fail-on-index-fail option, especially as that will need some
> bikeshedding on its name.
> 
> Also when posting new versions please include a diff from the previous
> version (this is more difficult if there was significant rebasing). This
> makew the v6 to v7 change obvious (the one comment change and the
> bugfix).
> 
> Moreover doing the diff with v4 (which I happen to have locally) I
> found the bug below.
> 
> Best wishes
> 
> Mark
> 
> 
> 
...
> > +static notmuch_bool_t
> > +insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
> > +   const char *dir, tag_op_list_t *tag_ops)
> > +{
> > +char *tmppath;
> > +char *newpath;
> > +char *newdir;
> > +int fdout;
> > +char *cleanup_path;
> > +
> > +fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
> > +if (fdout < 0)
> > +   return FALSE;
> > +
> > +cleanup_path = tmppath;
> > +
> > +if (! copy_stdin (fdin, fdout))
> > +   goto FAIL;
> > +
> > +if (fsync (fdout) != 0) {
> > +   fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
> > +   goto FAIL;
> > +}
> > +
> > +close (fdout);
> > +fdout = -1;
> > +
> > +/* Atomically move the new message file from the Maildir 'tmp' 
> > directory
> > + * to the 'new' directory.  We follow the Dovecot recommendation to
> > + * simply use rename() instead of link() and unlink().
> > + * See also: 
> > http://wiki.dovecot.org/MailboxFormat/Maildir#Mail_delivery
> > + */
> > +if (rename (tmppath, newpath) != 0) {
> > +   fprintf (stderr, "Error: rename() failed: %s\n", strerror (errno));
> > +   goto FAIL;
> > +}
> > +
> > +if (! sync_dir (newdir))
> > +   goto FAIL;
> 
> I think cleanup_path needs to be updated before the sync_dir is test as
> newpath should be unlinked rather than oldpath. (v4 explicitly unlinked 
> newpath)
> 

Thanks for the continued close review.
I'll post a followup to this specific patch.

Peter


[PATCH v7 12/12] test: test insert --create-folder option

2013-06-23 Thread Peter Wang
Add tests for notmuch insert --create-folder option.
---
 test/insert | 24 
 1 file changed, 24 insertions(+)

diff --git a/test/insert b/test/insert
index f573c76..021edb6 100755
--- a/test/insert
+++ b/test/insert
@@ -94,4 +94,28 @@ gen_insert_msg
 test_expect_code 1 "Insert message into non-existent folder" \
 "notmuch insert --folder=nonesuch < $gen_msg_filename"

+test_begin_subtest "Insert message, create folder"
+gen_insert_msg
+notmuch insert --folder=F --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" "$MAIL_DIR/F/cur/${basename}"
+
+test_begin_subtest "Insert message, create subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F/G/H/I/J tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" 
"${MAIL_DIR}/F/G/H/I/J/cur/${basename}"
+
+test_begin_subtest "Insert message, create existing subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch count folder:F/G/H/I/J tag:folder)
+test_expect_equal "$output" "2"
+
+gen_insert_msg
+test_expect_code 1 "Insert message, create invalid subfolder" \
+"notmuch insert --folder=../G --create-folder $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v7 11/12] man: document insert --create-folder

2013-06-23 Thread Peter Wang
Add documentation for notmuch insert --create-folder option.
---
 man/man1/notmuch-insert.1 | 12 
 1 file changed, 12 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index e85fef8..d5202ac 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -43,6 +43,18 @@ relative to the top-level directory given by the value of
 The default is to deliver to the top-level directory.

 .RE
+
+.RS 4
+.TP 4
+.B "--create-folder"
+
+Try to create the folder named by the
+.B "--folder"
+option, if it does not exist.
+Otherwise the folder must already exist for mail
+delivery to succeed.
+
+.RE
 .SH EXIT STATUS

 This command returns exit status 0 if the message was successfully
-- 
1.7.12.1



[PATCH v7 10/12] insert: add --create-folder option

2013-06-23 Thread Peter Wang
Allow the insert command to create the maildir folder
into which the new message should be delivered.
---
 notmuch-insert.c | 100 +++
 1 file changed, 100 insertions(+)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 5a4b3ea..8d0235d 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -100,6 +100,99 @@ check_folder_name (const char *folder)
 }
 }

+/* Make the given directory, succeeding if it already exists. */
+static notmuch_bool_t
+make_directory (char *path, int mode)
+{
+notmuch_bool_t ret;
+char *slash;
+
+if (mkdir (path, mode) != 0)
+   return (errno == EEXIST);
+
+/* Sync the parent directory for durability. */
+ret = TRUE;
+slash = strrchr (path, '/');
+if (slash) {
+   *slash = '\0';
+   ret = sync_dir (path);
+   *slash = '/';
+}
+return ret;
+}
+
+/* Make the given directory including its parent directories as necessary.
+ * Return TRUE on success, FALSE on error. */
+static notmuch_bool_t
+make_directory_and_parents (char *path, int mode)
+{
+struct stat st;
+char *start;
+char *end;
+notmuch_bool_t ret;
+
+/* First check the common case: directory already exists. */
+if (stat (path, &st) == 0)
+   return S_ISDIR (st.st_mode) ? TRUE : FALSE;
+
+for (start = path; *start != '\0'; start = end + 1) {
+   /* start points to the first unprocessed character.
+* Find the next slash from start onwards. */
+   end = strchr (start, '/');
+
+   /* If there are no more slashes then all the parent directories
+* have been made.  Now attempt to make the whole path. */
+   if (end == NULL)
+   return make_directory (path, mode);
+
+   /* Make the path up to the next slash, unless the current
+* directory component is actually empty. */
+   if (end > start) {
+   *end = '\0';
+   ret = make_directory (path, mode);
+   *end = '/';
+   if (! ret)
+   return FALSE;
+   }
+}
+
+return TRUE;
+}
+
+/* Create the given maildir folder, i.e. dir and its subdirectories
+ * 'cur', 'new', 'tmp'. */
+static notmuch_bool_t
+maildir_create_folder (void *ctx, const char *dir)
+{
+const int mode = 0700;
+char *subdir;
+char *tail;
+
+/* Create 'cur' directory, including parent directories. */
+subdir = talloc_asprintf (ctx, "%s/cur", dir);
+if (! subdir) {
+   fprintf (stderr, "Out of memory.\n");
+   return FALSE;
+}
+if (! make_directory_and_parents (subdir, mode))
+   return FALSE;
+
+tail = subdir + strlen (subdir) - 3;
+
+/* Create 'new' directory. */
+strcpy (tail, "new");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+/* Create 'tmp' directory. */
+strcpy (tail, "tmp");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+talloc_free (subdir);
+return TRUE;
+}
+
 /* Open a unique file in the 'tmp' sub-directory of dir.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -304,6 +397,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
 const char *folder = NULL;
+notmuch_bool_t create_folder = FALSE;
 const char *maildir;
 int opt_index;
 unsigned int i;
@@ -311,6 +405,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])

 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_BOOLEAN, &create_folder, "create-folder", 0, 0 },
{ NOTMUCH_OPT_END, 0, 0, 0, 0 }
 };

@@ -355,6 +450,11 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
fprintf (stderr, "Out of memory\n");
return 1;
}
+   if (create_folder && ! maildir_create_folder (config, maildir)) {
+   fprintf (stderr, "Error: creating maildir %s: %s\n",
+maildir, strerror (errno));
+   return 1;
+   }
 }

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
-- 
1.7.12.1



[PATCH v7 09/12] test: test insert --folder option

2013-06-23 Thread Peter Wang
Add tests for notmuch insert --folder option.
---
 test/insert | 17 +
 1 file changed, 17 insertions(+)

diff --git a/test/insert b/test/insert
index b432a74..f573c76 100755
--- a/test/insert
+++ b/test/insert
@@ -77,4 +77,21 @@ notmuch insert +custom -unread < "$gen_msg_filename"
 output=$(notmuch count tag:custom NOT tag:unread)
 test_expect_equal "$output" "1"

+test_begin_subtest "Insert message into folder"
+gen_insert_msg
+notmuch insert --folder=Drafts < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:Drafts)
+dirname=$(dirname "$output")
+test_expect_equal "$dirname" "$MAIL_DIR/Drafts/cur"
+
+test_begin_subtest "Insert message into folder, add/remove tags"
+gen_insert_msg
+notmuch insert --folder=Drafts +draft -unread < "$gen_msg_filename"
+output=$(notmuch count folder:Drafts tag:draft NOT tag:unread)
+test_expect_equal "$output" "1"
+
+gen_insert_msg
+test_expect_code 1 "Insert message into non-existent folder" \
+"notmuch insert --folder=nonesuch < $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v7 08/12] man: document insert --folder option

2013-06-23 Thread Peter Wang
Add documentation for notmuch insert --folder option.
---
 man/man1/notmuch-insert.1 | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index fbf83e0..e85fef8 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -4,6 +4,7 @@ notmuch-insert \- add a message to the maildir and notmuch 
database
 .SH SYNOPSIS

 .B notmuch insert
+.RI "[" options "]"
 .RI "[ +<" tag> "|\-<" tag "> ... ]"

 .SH DESCRIPTION
@@ -28,6 +29,19 @@ If the new message is a duplicate of an existing message in 
the database
 (it has same Message-ID), it will be added to the maildir folder and
 notmuch database, but the tags will not be changed.

+Option arguments must appear before any tag operation arguments.
+Supported options for
+.B insert
+include
+.RS 4
+.TP 4
+.BI "--folder=<" folder ">"
+
+Deliver the message to the specified folder,
+relative to the top-level directory given by the value of
+\fBdatabase.path\fR.
+The default is to deliver to the top-level directory.
+
 .RE
 .SH EXIT STATUS

-- 
1.7.12.1



[PATCH v7 07/12] insert: add --folder option

2013-06-23 Thread Peter Wang
Allow the new message to be inserted into a folder within the Maildir
hierarchy instead of the top-level folder.
---
 notmuch-insert.c | 46 --
 1 file changed, 44 insertions(+), 2 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 591189f..5a4b3ea 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -83,6 +83,23 @@ sync_dir (const char *dir)
 return ret;
 }

+/* Check the specified folder name does not contain a directory
+ * component ".." to prevent writes outside of the Maildir hierarchy. */
+static notmuch_bool_t
+check_folder_name (const char *folder)
+{
+const char *p = folder;
+
+for (;;) {
+   if ((p[0] == '.') && (p[1] == '.') && (p[2] == '\0' || p[2] == '/'))
+   return FALSE;
+   p = strchr (p, '/');
+   if (!p)
+   return TRUE;
+   p++;
+}
+}
+
 /* Open a unique file in the 'tmp' sub-directory of dir.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -286,11 +303,24 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 size_t new_tags_length;
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
+const char *folder = NULL;
 const char *maildir;
-int opt_index = 1;
+int opt_index;
 unsigned int i;
 notmuch_bool_t ret;

+notmuch_opt_desc_t options[] = {
+   { NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_END, 0, 0, 0, 0 }
+};
+
+opt_index = parse_arguments (argc, argv, options, 1);
+
+if (opt_index < 0) {
+   /* diagnostics already printed */
+   return 1;
+}
+
 db_path = notmuch_config_get_database_path (config);
 new_tags = notmuch_config_get_new_tags (config, &new_tags_length);

@@ -313,7 +343,19 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
return 1;
 }

-maildir = db_path;
+if (folder == NULL) {
+   maildir = db_path;
+} else {
+   if (! check_folder_name (folder)) {
+   fprintf (stderr, "Error: bad folder name: %s\n", folder);
+   return 1;
+   }
+   maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
+   if (! maildir) {
+   fprintf (stderr, "Out of memory\n");
+   return 1;
+   }
+}

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
  * from standard input may be interrupted. */
-- 
1.7.12.1



[PATCH v7 06/12] test: add tests for insert

2013-06-23 Thread Peter Wang
Add tests for new 'insert' command.
---
 test/insert   | 80 +++
 test/notmuch-test |  1 +
 2 files changed, 81 insertions(+)
 create mode 100755 test/insert

diff --git a/test/insert b/test/insert
new file mode 100755
index 000..b432a74
--- /dev/null
+++ b/test/insert
@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+test_description='"notmuch insert"'
+. ./test-lib.sh
+
+# Create directories and database before inserting.
+mkdir -p "$MAIL_DIR"/{cur,new,tmp}
+mkdir -p "$MAIL_DIR"/Drafts/{cur,new,tmp}
+notmuch new > /dev/null
+
+# We use generate_message to create the temporary message files.
+# They happen to be in the mail directory already but that is okay
+# since we do not call notmuch new hereafter.
+
+gen_insert_msg() {
+generate_message \
+   "[subject]=\"insert-subject\"" \
+   "[date]=\"Sat, 01 Jan 2000 12:00:00 -\"" \
+   "[body]=\"insert-message\""
+}
+
+test_expect_code 1 "Insert zero-length file" \
+"notmuch insert < /dev/null"
+
+# This test is a proxy for other errors that may occur while trying to
+# add a message to the notmuch database, e.g. database locked.
+test_expect_code 0 "Insert non-message" \
+"echo bad_message | notmuch insert"
+
+test_begin_subtest "Database empty so far"
+test_expect_equal "0" "`notmuch count --output=messages '*'`"
+
+test_begin_subtest "Insert message"
+gen_insert_msg
+notmuch insert < "$gen_msg_filename"
+cur_msg_filename=$(notmuch search --output=files "subject:insert-subject")
+test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
+
+test_begin_subtest "Insert message adds default tags"
+output=$(notmuch show --format=json "subject:insert-subject")
+expected='[[[{
+ "id": "'"${gen_msg_id}"'",
+ "match": true,
+ "excluded": false,
+ "filename": "'"${cur_msg_filename}"'",
+ "timestamp": 946728000,
+ "date_relative": "2000-01-01",
+ "tags": ["inbox","unread"],
+ "headers": {
+  "Subject": "insert-subject",
+  "From": "Notmuch Test Suite ",
+  "To": "Notmuch Test Suite ",
+  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
+ "body": [{"id": 1,
+  "content-type": "text/plain",
+  "content": "insert-message\n"}]},
+ ['
+test_expect_equal_json "$output" "$expected"
+
+test_begin_subtest "Insert duplicate message"
+notmuch insert +duptag -unread < "$gen_msg_filename"
+output=$(notmuch search --output=files "subject:insert-subject" | wc -l)
+test_expect_equal "$output" 2
+
+test_begin_subtest "Duplicate message does not change tags"
+output=$(notmuch search --format=json --output=tags "subject:insert-subject")
+test_expect_equal_json "$output" '["inbox", "unread"]'
+
+test_begin_subtest "Insert message, add tag"
+gen_insert_msg
+notmuch insert +custom < "$gen_msg_filename"
+output=$(notmuch count tag:custom)
+test_expect_equal "$output" "1"
+
+test_begin_subtest "Insert message, add/remove tags"
+gen_insert_msg
+notmuch insert +custom -unread < "$gen_msg_filename"
+output=$(notmuch count tag:custom NOT tag:unread)
+test_expect_equal "$output" "1"
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index a0c47d4..6db7979 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -23,6 +23,7 @@ TESTS="
   setup
   new
   count
+  insert
   search
   search-output
   search-by-folder
-- 
1.7.12.1



[PATCH v7 05/12] man: reference notmuch-insert.1

2013-06-23 Thread Peter Wang
Add references to notmuch-insert.1 from other man pages.
---
 man/man1/notmuch-config.1   | 4 ++--
 man/man1/notmuch-count.1| 4 ++--
 man/man1/notmuch-dump.1 | 4 ++--
 man/man1/notmuch-new.1  | 4 ++--
 man/man1/notmuch-reply.1| 3 ++-
 man/man1/notmuch-restore.1  | 3 ++-
 man/man1/notmuch-search.1   | 3 ++-
 man/man1/notmuch-show.1 | 3 ++-
 man/man1/notmuch-tag.1  | 3 ++-
 man/man1/notmuch.1  | 3 ++-
 man/man5/notmuch-hooks.5| 4 ++--
 man/man7/notmuch-search-terms.7 | 3 ++-
 12 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/man/man1/notmuch-config.1 b/man/man1/notmuch-config.1
index db31166..1635951 100644
--- a/man/man1/notmuch-config.1
+++ b/man/man1/notmuch-config.1
@@ -152,7 +152,7 @@ use ${HOME}/.notmuch\-config if this variable is not set.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-count\fR(1), \fBnotmuch-dump\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-count.1 b/man/man1/notmuch-count.1
index 7fc4378..c172a22 100644
--- a/man/man1/notmuch-count.1
+++ b/man/man1/notmuch-count.1
@@ -72,7 +72,7 @@ Read input from given file, instead of from stdin. Implies
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-dump\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
index 3fa51bd..a758a52 100644
--- a/man/man1/notmuch-dump.1
+++ b/man/man1/notmuch-dump.1
@@ -92,7 +92,7 @@ for details of the supported syntax for .
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-new.1 b/man/man1/notmuch-new.1
index 02f7954..3fcfc29 100644
--- a/man/man1/notmuch-new.1
+++ b/man/man1/notmuch-new.1
@@ -64,7 +64,7 @@ Prevents hooks from being run.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-reply.1 b/man/man1/notmuch-reply.1
index bf2021f..f0394b2 100644
--- a/man/man1/notmuch-reply.1
+++ b/man/man1/notmuch-reply.1
@@ -126,7 +126,8 @@ The requested format version is too new.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
index 4ec4c80..686c0ac 100644
--- a/man/man1/notmuch-restore.1
+++ b/man/man1/notmuch-restore.1
@@ -84,7 +84,8 @@ should be accurate.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-reply\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-search.1 b/man/man1/notmuch-search.1
index 1c1e049..1206770 100644
--- a/man/man1/notmuch-search.1
+++ b/man/man1/notmuch-search.1
@@ -172,7 +172,8 @@ The requested format version is too new.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff -

[PATCH v7 04/12] man: document 'insert' command

2013-06-23 Thread Peter Wang
Add initial documentation for notmuch insert command.
---
 man/Makefile.local|  1 +
 man/man1/notmuch-insert.1 | 49 +++
 2 files changed, 50 insertions(+)
 create mode 100644 man/man1/notmuch-insert.1

diff --git a/man/Makefile.local b/man/Makefile.local
index 72e2a18..216aaa0 100644
--- a/man/Makefile.local
+++ b/man/Makefile.local
@@ -12,6 +12,7 @@ MAN1 := \
$(dir)/man1/notmuch-count.1 \
$(dir)/man1/notmuch-dump.1 \
$(dir)/man1/notmuch-restore.1 \
+   $(dir)/man1/notmuch-insert.1 \
$(dir)/man1/notmuch-new.1 \
$(dir)/man1/notmuch-reply.1 \
$(dir)/man1/notmuch-search.1 \
diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
new file mode 100644
index 000..fbf83e0
--- /dev/null
+++ b/man/man1/notmuch-insert.1
@@ -0,0 +1,49 @@
+.TH NOTMUCH-INSERT 1 2013-xx-xx "Notmuch 0.xx"
+.SH NAME
+notmuch-insert \- add a message to the maildir and notmuch database
+.SH SYNOPSIS
+
+.B notmuch insert
+.RI "[ +<" tag> "|\-<" tag "> ... ]"
+
+.SH DESCRIPTION
+
+.B notmuch insert
+reads a message from standard input
+and delivers it into the maildir directory given by configuration option
+.BR database.path ,
+then incorporates the message into the notmuch database.
+It is an alternative to using a separate tool to deliver
+the message then running
+.B notmuch new
+afterwards.
+
+The new message will be tagged with the tags specified by the
+.B new.tags
+configuration option, then by operations specified on the command-line:
+tags prefixed by '+' are added while
+those prefixed by '\-' are removed.
+
+If the new message is a duplicate of an existing message in the database
+(it has same Message-ID), it will be added to the maildir folder and
+notmuch database, but the tags will not be changed.
+
+.RE
+.SH EXIT STATUS
+
+This command returns exit status 0 if the message was successfully
+added to the mail directory, even if the message could not be indexed
+and added to the notmuch database.  In the latter case, a warning will
+be printed to standard error but the message file will be left on disk.
+
+If the message could not be written to disk then a non-zero exit
+status is returned.
+
+.RE
+.SH SEE ALSO
+
+\fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-reply\fR(1),
+\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
+\fBnotmuch-tag\fR(1)
-- 
1.7.12.1



[PATCH v7 03/12] cli: add insert command

2013-06-23 Thread Peter Wang
The notmuch insert command reads a message from standard input,
writes it to a Maildir folder, and then incorporates the message into
the notmuch database.  Essentially it moves the functionality of
notmuch-deliver into notmuch.

Though it could be used as an alternative to notmuch new, the reason
I want this is to allow my notmuch frontend to add postponed or sent
messages to the mail store and notmuch database, without resorting to
another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
---
 Makefile.local   |   1 +
 notmuch-client.h |   3 +
 notmuch-insert.c | 335 +++
 notmuch.c|   2 +
 4 files changed, 341 insertions(+)
 create mode 100644 notmuch-insert.c

diff --git a/Makefile.local b/Makefile.local
index 644623f..84043fe 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -261,6 +261,7 @@ notmuch_client_srcs =   \
notmuch-config.c\
notmuch-count.c \
notmuch-dump.c  \
+   notmuch-insert.c\
notmuch-new.c   \
notmuch-reply.c \
notmuch-restore.c   \
diff --git a/notmuch-client.h b/notmuch-client.h
index 4a3c7ac..dfe81e6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -181,6 +181,9 @@ int
 notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]);

 int
+notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]);
+
+int
 notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);

 int
diff --git a/notmuch-insert.c b/notmuch-insert.c
new file mode 100644
index 000..591189f
--- /dev/null
+++ b/notmuch-insert.c
@@ -0,0 +1,335 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ? 2013 Peter Wang
+ *
+ * Based in part on notmuch-deliver
+ * Copyright ? 2010 Ali Polatel
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Peter Wang 
+ */
+
+#include "notmuch-client.h"
+#include "tag-util.h"
+
+#include 
+#include 
+#include 
+
+static volatile sig_atomic_t interrupted;
+
+static void
+handle_sigint (unused (int sig))
+{
+static char msg[] = "Stopping... \n";
+
+/* This write is "opportunistic", so it's okay to ignore the
+ * result.  It is not required for correctness, and if it does
+ * fail or produce a short write, we want to get out of the signal
+ * handler as quickly as possible, not retry it. */
+IGNORE_RESULT (write (2, msg, sizeof (msg) - 1));
+interrupted = 1;
+}
+
+/* Like gethostname but guarantees that a null-terminated hostname is
+ * returned, even if it has to make one up. Invalid characters are
+ * substituted such that the hostname can be used within a filename.
+ */
+static void
+safe_gethostname (char *hostname, size_t len)
+{
+char *p;
+
+if (gethostname (hostname, len) == -1) {
+   strncpy (hostname, "unknown", len);
+}
+hostname[len - 1] = '\0';
+
+for (p = hostname; *p != '\0'; p++) {
+   if (*p == '/' || *p == ':')
+   *p = '_';
+}
+}
+
+/* Call fsync() on a directory path. */
+static notmuch_bool_t
+sync_dir (const char *dir)
+{
+notmuch_bool_t ret;
+int fd;
+
+fd = open (dir, O_RDONLY);
+if (fd == -1) {
+   fprintf (stderr, "Error: open() dir failed: %s\n", strerror (errno));
+   return FALSE;
+}
+ret = (fsync (fd) == 0);
+if (! ret) {
+   fprintf (stderr, "Error: fsync() dir failed: %s\n", strerror (errno));
+}
+close (fd);
+return ret;
+}
+
+/* Open a unique file in the 'tmp' sub-directory of dir.
+ * Returns the file descriptor on success, or -1 on failure.
+ * On success, file paths for the message in the 'tmp' and 'new'
+ * directories are returned via tmppath and newpath,
+ * and the path of the 'new' directory itself in newdir. */
+static int
+maildir_open_tmp_file (void *ctx, const char *dir,
+  char **tmppath, char **newpath, char **newdir)
+{
+pid_t pid;
+char hostname[256];
+struct timeval tv;
+char *filename;
+int fd = -1;
+
+/* We follow the Dovecot file name generation algorithm. */
+pid = getpid ();
+safe_gethostname (hostname, siz

[PATCH v7 02/12] tag-util: do not reset list in parse_tag_command_line

2013-06-23 Thread Peter Wang
The 'insert' command will be better served if parse_tag_command_line
modifies a pre-populated list (of new.tags) instead of clobbering the
list outright.  The sole existing caller, notmuch_tag_command, is
unaffected by this change.
---
 tag-util.c | 2 --
 tag-util.h | 2 ++
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tag-util.c b/tag-util.c
index 92e08a1..3bde409 100644
--- a/tag-util.c
+++ b/tag-util.c
@@ -165,8 +165,6 @@ parse_tag_command_line (void *ctx, int argc, char **argv,

 int i;

-tag_op_list_reset (tag_ops);
-
 for (i = 0; i < argc; i++) {
if (strcmp (argv[i], "--") == 0) {
i++;
diff --git a/tag-util.h b/tag-util.h
index 246de85..4628f16 100644
--- a/tag-util.h
+++ b/tag-util.h
@@ -81,6 +81,8 @@ parse_tag_line (void *ctx, char *line,
  * Output Parameters:
  * ops contains a list of tag operations
  * query_str the search terms.
+ *
+ * The ops argument is not cleared.
  */

 tag_parse_status_t
-- 
1.7.12.1



[PATCH v7 01/12] tag-util: move out 'tag' command-line check

2013-06-23 Thread Peter Wang
Move an error condition specific to the 'tag' command out of
parse_tag_command_line so that parse_tag_command_line can be used for
the forthcoming 'insert' command.
---
 notmuch-tag.c | 5 +
 tag-util.c| 6 +++---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/notmuch-tag.c b/notmuch-tag.c
index 9a5d3e7..3b09df9 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -247,6 +247,11 @@ notmuch_tag_command (notmuch_config_t *config, int argc, 
char *argv[])
fprintf (stderr, "Error: 'notmuch tag' requires at least one tag to 
add or remove.\n");
return 1;
}
+
+   if (*query_string == '\0') {
+   fprintf (stderr, "Error: notmuch tag requires at least one search 
term.\n");
+   return 1;
+   }
 }

 if (notmuch_database_open (notmuch_config_get_database_path (config),
diff --git a/tag-util.c b/tag-util.c
index c5f5859..92e08a1 100644
--- a/tag-util.c
+++ b/tag-util.c
@@ -190,9 +190,9 @@ parse_tag_command_line (void *ctx, int argc, char **argv,

 *query_str = query_string_from_args (ctx, argc - i, &argv[i]);

-if (*query_str == NULL || **query_str == '\0') {
-   fprintf (stderr, "Error: notmuch tag requires at least one search 
term.\n");
-   return TAG_PARSE_INVALID;
+if (*query_str == NULL) {
+   fprintf (stderr, "Out of memory.\n");
+   return TAG_PARSE_OUT_OF_MEMORY;
 }

 return TAG_PARSE_SUCCESS;
-- 
1.7.12.1



[PATCH v7 00/12] insert command

2013-06-23 Thread Peter Wang
- address Mark's review comments

Peter Wang (12):
  tag-util: move out 'tag' command-line check
  tag-util: do not reset list in parse_tag_command_line
  cli: add insert command
  man: document 'insert' command
  man: reference notmuch-insert.1
  test: add tests for insert
  insert: add --folder option
  man: document insert --folder option
  test: test insert --folder option
  insert: add --create-folder option
  man: document insert --create-folder
  test: test insert --create-folder option

 Makefile.local  |   1 +
 man/Makefile.local  |   1 +
 man/man1/notmuch-config.1   |   4 +-
 man/man1/notmuch-count.1|   4 +-
 man/man1/notmuch-dump.1 |   4 +-
 man/man1/notmuch-insert.1   |  75 +++
 man/man1/notmuch-new.1  |   4 +-
 man/man1/notmuch-reply.1|   3 +-
 man/man1/notmuch-restore.1  |   3 +-
 man/man1/notmuch-search.1   |   3 +-
 man/man1/notmuch-show.1 |   3 +-
 man/man1/notmuch-tag.1  |   3 +-
 man/man1/notmuch.1  |   3 +-
 man/man5/notmuch-hooks.5|   4 +-
 man/man7/notmuch-search-terms.7 |   3 +-
 notmuch-client.h|   3 +
 notmuch-insert.c| 477 
 notmuch-tag.c   |   5 +
 notmuch.c   |   2 +
 tag-util.c  |   8 +-
 tag-util.h  |   2 +
 test/insert | 121 ++
 test/notmuch-test   |   1 +
 23 files changed, 715 insertions(+), 22 deletions(-)
 create mode 100644 man/man1/notmuch-insert.1
 create mode 100644 notmuch-insert.c
 create mode 100755 test/insert

-- 
1.7.12.1



[PATCH v6 03/12] cli: add insert command

2013-06-23 Thread Peter Wang
On Sat, 22 Jun 2013 08:48:56 +0100, Mark Walters  
wrote:
> 
> I think I would like the option to have the command fail if it doesn't
> get added to the database. If, for example, the command is used to
> postpone a message it may be very difficult/impossible to find it
> without the postponed tag that I asked for.
> 
> Could we have a --fail-on-index-fail cli option which says which
> behaviour we want? It is a little ugly but I think different users will
> want different behaviour and there is a risk of dataloss in either case.

I would like that option, too (not sure about the name).  It can be
added after at least the first few patches in the series are pushed.

Peter


[PATCH v6 12/12] test: test insert --create-folder option

2013-06-22 Thread Peter Wang
Add tests for notmuch insert --create-folder option.
---
 test/insert | 24 
 1 file changed, 24 insertions(+)

diff --git a/test/insert b/test/insert
index f573c76..021edb6 100755
--- a/test/insert
+++ b/test/insert
@@ -94,4 +94,28 @@ gen_insert_msg
 test_expect_code 1 "Insert message into non-existent folder" \
 "notmuch insert --folder=nonesuch < $gen_msg_filename"

+test_begin_subtest "Insert message, create folder"
+gen_insert_msg
+notmuch insert --folder=F --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" "$MAIL_DIR/F/cur/${basename}"
+
+test_begin_subtest "Insert message, create subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F/G/H/I/J tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" 
"${MAIL_DIR}/F/G/H/I/J/cur/${basename}"
+
+test_begin_subtest "Insert message, create existing subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch count folder:F/G/H/I/J tag:folder)
+test_expect_equal "$output" "2"
+
+gen_insert_msg
+test_expect_code 1 "Insert message, create invalid subfolder" \
+"notmuch insert --folder=../G --create-folder $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v6 11/12] man: document insert --create-folder

2013-06-22 Thread Peter Wang
Add documentation for notmuch insert --create-folder option.
---
 man/man1/notmuch-insert.1 | 12 
 1 file changed, 12 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index e85fef8..d5202ac 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -43,6 +43,18 @@ relative to the top-level directory given by the value of
 The default is to deliver to the top-level directory.

 .RE
+
+.RS 4
+.TP 4
+.B "--create-folder"
+
+Try to create the folder named by the
+.B "--folder"
+option, if it does not exist.
+Otherwise the folder must already exist for mail
+delivery to succeed.
+
+.RE
 .SH EXIT STATUS

 This command returns exit status 0 if the message was successfully
-- 
1.7.12.1



[PATCH v6 10/12] insert: add --create-folder option

2013-06-22 Thread Peter Wang
Allow the insert command to create the maildir folder
into which the new message should be delivered.
---
 notmuch-insert.c | 100 +++
 1 file changed, 100 insertions(+)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index a6134a8..25df3be 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -100,6 +100,99 @@ check_folder_name (const char *folder)
 }
 }

+/* Make the given directory, succeeding if it already exists. */
+static notmuch_bool_t
+make_directory (char *path, int mode)
+{
+notmuch_bool_t ret;
+char *slash;
+
+if (mkdir (path, mode) != 0)
+   return (errno == EEXIST);
+
+/* Sync the parent directory for durability. */
+ret = TRUE;
+slash = strrchr (path, '/');
+if (slash) {
+   *slash = '\0';
+   ret = sync_dir (path);
+   *slash = '/';
+}
+return ret;
+}
+
+/* Make the given directory including its parent directories as necessary.
+ * Return TRUE on success, FALSE on error. */
+static notmuch_bool_t
+make_directory_and_parents (char *path, int mode)
+{
+struct stat st;
+char *start;
+char *end;
+notmuch_bool_t ret;
+
+/* First check the common case: directory already exists. */
+if (stat (path, &st) == 0)
+   return S_ISDIR (st.st_mode) ? TRUE : FALSE;
+
+for (start = path; *start != '\0'; start = end + 1) {
+   /* start points to the first unprocessed character.
+* Find the next slash from start onwards. */
+   end = strchr (start, '/');
+
+   /* If there are no more slashes then all the parent directories
+* have been made.  Now attempt to make the whole path. */
+   if (end == NULL)
+   return make_directory (path, mode);
+
+   /* Make the path up to the next slash, unless the current
+* directory component is actually empty. */
+   if (end > start) {
+   *end = '\0';
+   ret = make_directory (path, mode);
+   *end = '/';
+   if (! ret)
+   return FALSE;
+   }
+}
+
+return TRUE;
+}
+
+/* Create the given maildir folder, i.e. dir and its subdirectories
+ * 'cur', 'new', 'tmp'. */
+static notmuch_bool_t
+maildir_create_folder (void *ctx, const char *dir)
+{
+const int mode = 0700;
+char *subdir;
+char *tail;
+
+/* Create 'cur' directory, including parent directories. */
+subdir = talloc_asprintf (ctx, "%s/cur", dir);
+if (! subdir) {
+   fprintf (stderr, "Out of memory.\n");
+   return FALSE;
+}
+if (! make_directory_and_parents (subdir, mode))
+   return FALSE;
+
+tail = subdir + strlen (subdir) - 3;
+
+/* Create 'new' directory. */
+strcpy (tail, "new");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+/* Create 'tmp' directory. */
+strcpy (tail, "tmp");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+talloc_free (subdir);
+return TRUE;
+}
+
 /* Open a unique file in the Maildir 'tmp' directory.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -304,6 +397,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
 const char *folder = NULL;
+notmuch_bool_t create_folder = FALSE;
 const char *maildir;
 int opt_index;
 unsigned int i;
@@ -311,6 +405,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])

 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_BOOLEAN, &create_folder, "create-folder", 0, 0 },
{ NOTMUCH_OPT_END, 0, 0, 0, 0 }
 };

@@ -355,6 +450,11 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
fprintf (stderr, "Out of memory\n");
return 1;
}
+   if (create_folder && ! maildir_create_folder (config, maildir)) {
+   fprintf (stderr, "Error: creating maildir %s: %s\n",
+maildir, strerror (errno));
+   return 1;
+   }
 }

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
-- 
1.7.12.1



[PATCH v6 09/12] test: test insert --folder option

2013-06-22 Thread Peter Wang
Add tests for notmuch insert --folder option.
---
 test/insert | 17 +
 1 file changed, 17 insertions(+)

diff --git a/test/insert b/test/insert
index b432a74..f573c76 100755
--- a/test/insert
+++ b/test/insert
@@ -77,4 +77,21 @@ notmuch insert +custom -unread < "$gen_msg_filename"
 output=$(notmuch count tag:custom NOT tag:unread)
 test_expect_equal "$output" "1"

+test_begin_subtest "Insert message into folder"
+gen_insert_msg
+notmuch insert --folder=Drafts < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:Drafts)
+dirname=$(dirname "$output")
+test_expect_equal "$dirname" "$MAIL_DIR/Drafts/cur"
+
+test_begin_subtest "Insert message into folder, add/remove tags"
+gen_insert_msg
+notmuch insert --folder=Drafts +draft -unread < "$gen_msg_filename"
+output=$(notmuch count folder:Drafts tag:draft NOT tag:unread)
+test_expect_equal "$output" "1"
+
+gen_insert_msg
+test_expect_code 1 "Insert message into non-existent folder" \
+"notmuch insert --folder=nonesuch < $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v6 08/12] man: document insert --folder option

2013-06-22 Thread Peter Wang
Add documentation for notmuch insert --folder option.
---
 man/man1/notmuch-insert.1 | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index fbf83e0..e85fef8 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -4,6 +4,7 @@ notmuch-insert \- add a message to the maildir and notmuch 
database
 .SH SYNOPSIS

 .B notmuch insert
+.RI "[" options "]"
 .RI "[ +<" tag> "|\-<" tag "> ... ]"

 .SH DESCRIPTION
@@ -28,6 +29,19 @@ If the new message is a duplicate of an existing message in 
the database
 (it has same Message-ID), it will be added to the maildir folder and
 notmuch database, but the tags will not be changed.

+Option arguments must appear before any tag operation arguments.
+Supported options for
+.B insert
+include
+.RS 4
+.TP 4
+.BI "--folder=<" folder ">"
+
+Deliver the message to the specified folder,
+relative to the top-level directory given by the value of
+\fBdatabase.path\fR.
+The default is to deliver to the top-level directory.
+
 .RE
 .SH EXIT STATUS

-- 
1.7.12.1



[PATCH v6 07/12] insert: add --folder option

2013-06-22 Thread Peter Wang
Allow the new message to be inserted into a folder within the Maildir
hierarchy instead of the top-level folder.
---
 notmuch-insert.c | 46 --
 1 file changed, 44 insertions(+), 2 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 1495234..a6134a8 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -83,6 +83,23 @@ sync_dir (const char *dir)
 return ret;
 }

+/* Check the specified folder name does not contain a directory
+ * component ".." to prevent writes outside of the Maildir hierarchy. */
+static notmuch_bool_t
+check_folder_name (const char *folder)
+{
+const char *p = folder;
+
+for (;;) {
+   if ((p[0] == '.') && (p[1] == '.') && (p[2] == '\0' || p[2] == '/'))
+   return FALSE;
+   p = strchr (p, '/');
+   if (!p)
+   return TRUE;
+   p++;
+}
+}
+
 /* Open a unique file in the Maildir 'tmp' directory.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -286,11 +303,24 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 size_t new_tags_length;
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
+const char *folder = NULL;
 const char *maildir;
-int opt_index = 1;
+int opt_index;
 unsigned int i;
 notmuch_bool_t ret;

+notmuch_opt_desc_t options[] = {
+   { NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_END, 0, 0, 0, 0 }
+};
+
+opt_index = parse_arguments (argc, argv, options, 1);
+
+if (opt_index < 0) {
+   /* diagnostics already printed */
+   return 1;
+}
+
 db_path = notmuch_config_get_database_path (config);
 new_tags = notmuch_config_get_new_tags (config, &new_tags_length);

@@ -313,7 +343,19 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
return 1;
 }

-maildir = db_path;
+if (folder == NULL) {
+   maildir = db_path;
+} else {
+   if (! check_folder_name (folder)) {
+   fprintf (stderr, "Error: bad folder name: %s\n", folder);
+   return 1;
+   }
+   maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
+   if (! maildir) {
+   fprintf (stderr, "Out of memory\n");
+   return 1;
+   }
+}

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
  * from standard input may be interrupted. */
-- 
1.7.12.1



[PATCH v6 06/12] test: add tests for insert

2013-06-22 Thread Peter Wang
Add tests for new 'insert' command.
---
 test/insert   | 80 +++
 test/notmuch-test |  1 +
 2 files changed, 81 insertions(+)
 create mode 100755 test/insert

diff --git a/test/insert b/test/insert
new file mode 100755
index 000..b432a74
--- /dev/null
+++ b/test/insert
@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+test_description='"notmuch insert"'
+. ./test-lib.sh
+
+# Create directories and database before inserting.
+mkdir -p "$MAIL_DIR"/{cur,new,tmp}
+mkdir -p "$MAIL_DIR"/Drafts/{cur,new,tmp}
+notmuch new > /dev/null
+
+# We use generate_message to create the temporary message files.
+# They happen to be in the mail directory already but that is okay
+# since we do not call notmuch new hereafter.
+
+gen_insert_msg() {
+generate_message \
+   "[subject]=\"insert-subject\"" \
+   "[date]=\"Sat, 01 Jan 2000 12:00:00 -\"" \
+   "[body]=\"insert-message\""
+}
+
+test_expect_code 1 "Insert zero-length file" \
+"notmuch insert < /dev/null"
+
+# This test is a proxy for other errors that may occur while trying to
+# add a message to the notmuch database, e.g. database locked.
+test_expect_code 0 "Insert non-message" \
+"echo bad_message | notmuch insert"
+
+test_begin_subtest "Database empty so far"
+test_expect_equal "0" "`notmuch count --output=messages '*'`"
+
+test_begin_subtest "Insert message"
+gen_insert_msg
+notmuch insert < "$gen_msg_filename"
+cur_msg_filename=$(notmuch search --output=files "subject:insert-subject")
+test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
+
+test_begin_subtest "Insert message adds default tags"
+output=$(notmuch show --format=json "subject:insert-subject")
+expected='[[[{
+ "id": "'"${gen_msg_id}"'",
+ "match": true,
+ "excluded": false,
+ "filename": "'"${cur_msg_filename}"'",
+ "timestamp": 946728000,
+ "date_relative": "2000-01-01",
+ "tags": ["inbox","unread"],
+ "headers": {
+  "Subject": "insert-subject",
+  "From": "Notmuch Test Suite ",
+  "To": "Notmuch Test Suite ",
+  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
+ "body": [{"id": 1,
+  "content-type": "text/plain",
+  "content": "insert-message\n"}]},
+ ['
+test_expect_equal_json "$output" "$expected"
+
+test_begin_subtest "Insert duplicate message"
+notmuch insert +duptag -unread < "$gen_msg_filename"
+output=$(notmuch search --output=files "subject:insert-subject" | wc -l)
+test_expect_equal "$output" 2
+
+test_begin_subtest "Duplicate message does not change tags"
+output=$(notmuch search --format=json --output=tags "subject:insert-subject")
+test_expect_equal_json "$output" '["inbox", "unread"]'
+
+test_begin_subtest "Insert message, add tag"
+gen_insert_msg
+notmuch insert +custom < "$gen_msg_filename"
+output=$(notmuch count tag:custom)
+test_expect_equal "$output" "1"
+
+test_begin_subtest "Insert message, add/remove tags"
+gen_insert_msg
+notmuch insert +custom -unread < "$gen_msg_filename"
+output=$(notmuch count tag:custom NOT tag:unread)
+test_expect_equal "$output" "1"
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index a0c47d4..6db7979 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -23,6 +23,7 @@ TESTS="
   setup
   new
   count
+  insert
   search
   search-output
   search-by-folder
-- 
1.7.12.1



[PATCH v6 05/12] man: reference notmuch-insert.1

2013-06-22 Thread Peter Wang
Add references to notmuch-insert.1 from other man pages.
---
 man/man1/notmuch-config.1   | 4 ++--
 man/man1/notmuch-count.1| 4 ++--
 man/man1/notmuch-dump.1 | 4 ++--
 man/man1/notmuch-new.1  | 4 ++--
 man/man1/notmuch-reply.1| 3 ++-
 man/man1/notmuch-restore.1  | 3 ++-
 man/man1/notmuch-search.1   | 3 ++-
 man/man1/notmuch-show.1 | 3 ++-
 man/man1/notmuch-tag.1  | 3 ++-
 man/man1/notmuch.1  | 3 ++-
 man/man5/notmuch-hooks.5| 4 ++--
 man/man7/notmuch-search-terms.7 | 3 ++-
 12 files changed, 24 insertions(+), 17 deletions(-)

diff --git a/man/man1/notmuch-config.1 b/man/man1/notmuch-config.1
index db31166..1635951 100644
--- a/man/man1/notmuch-config.1
+++ b/man/man1/notmuch-config.1
@@ -152,7 +152,7 @@ use ${HOME}/.notmuch\-config if this variable is not set.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-count\fR(1), \fBnotmuch-dump\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-count.1 b/man/man1/notmuch-count.1
index 7fc4378..c172a22 100644
--- a/man/man1/notmuch-count.1
+++ b/man/man1/notmuch-count.1
@@ -72,7 +72,7 @@ Read input from given file, instead of from stdin. Implies
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-dump\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
index 3fa51bd..a758a52 100644
--- a/man/man1/notmuch-dump.1
+++ b/man/man1/notmuch-dump.1
@@ -92,7 +92,7 @@ for details of the supported syntax for .
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-new.1 b/man/man1/notmuch-new.1
index 02f7954..3fcfc29 100644
--- a/man/man1/notmuch-new.1
+++ b/man/man1/notmuch-new.1
@@ -64,7 +64,7 @@ Prevents hooks from being run.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-reply\fR(1),
-\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-insert\fR(1),
+\fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-reply.1 b/man/man1/notmuch-reply.1
index bf2021f..f0394b2 100644
--- a/man/man1/notmuch-reply.1
+++ b/man/man1/notmuch-reply.1
@@ -126,7 +126,8 @@ The requested format version is too new.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
index 4ec4c80..686c0ac 100644
--- a/man/man1/notmuch-restore.1
+++ b/man/man1/notmuch-restore.1
@@ -84,7 +84,8 @@ should be accurate.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-reply\fR(1), \fBnotmuch-search\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff --git a/man/man1/notmuch-search.1 b/man/man1/notmuch-search.1
index 1c1e049..1206770 100644
--- a/man/man1/notmuch-search.1
+++ b/man/man1/notmuch-search.1
@@ -172,7 +172,8 @@ The requested format version is too new.
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-new\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5),
+\fBnotmuch-insert\fR(1), \fBnotmuch-new\fR(1),
 \fBnotmuch-reply\fR(1), \fBnotmuch-restore\fR(1),
 \fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
 \fBnotmuch-tag\fR(1)
diff -

[PATCH v6 04/12] man: document 'insert' command

2013-06-22 Thread Peter Wang
Add initial documentation for notmuch insert command.
---
 man/Makefile.local|  1 +
 man/man1/notmuch-insert.1 | 49 +++
 2 files changed, 50 insertions(+)
 create mode 100644 man/man1/notmuch-insert.1

diff --git a/man/Makefile.local b/man/Makefile.local
index 72e2a18..216aaa0 100644
--- a/man/Makefile.local
+++ b/man/Makefile.local
@@ -12,6 +12,7 @@ MAN1 := \
$(dir)/man1/notmuch-count.1 \
$(dir)/man1/notmuch-dump.1 \
$(dir)/man1/notmuch-restore.1 \
+   $(dir)/man1/notmuch-insert.1 \
$(dir)/man1/notmuch-new.1 \
$(dir)/man1/notmuch-reply.1 \
$(dir)/man1/notmuch-search.1 \
diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
new file mode 100644
index 000..fbf83e0
--- /dev/null
+++ b/man/man1/notmuch-insert.1
@@ -0,0 +1,49 @@
+.TH NOTMUCH-INSERT 1 2013-xx-xx "Notmuch 0.xx"
+.SH NAME
+notmuch-insert \- add a message to the maildir and notmuch database
+.SH SYNOPSIS
+
+.B notmuch insert
+.RI "[ +<" tag> "|\-<" tag "> ... ]"
+
+.SH DESCRIPTION
+
+.B notmuch insert
+reads a message from standard input
+and delivers it into the maildir directory given by configuration option
+.BR database.path ,
+then incorporates the message into the notmuch database.
+It is an alternative to using a separate tool to deliver
+the message then running
+.B notmuch new
+afterwards.
+
+The new message will be tagged with the tags specified by the
+.B new.tags
+configuration option, then by operations specified on the command-line:
+tags prefixed by '+' are added while
+those prefixed by '\-' are removed.
+
+If the new message is a duplicate of an existing message in the database
+(it has same Message-ID), it will be added to the maildir folder and
+notmuch database, but the tags will not be changed.
+
+.RE
+.SH EXIT STATUS
+
+This command returns exit status 0 if the message was successfully
+added to the mail directory, even if the message could not be indexed
+and added to the notmuch database.  In the latter case, a warning will
+be printed to standard error but the message file will be left on disk.
+
+If the message could not be written to disk then a non-zero exit
+status is returned.
+
+.RE
+.SH SEE ALSO
+
+\fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
+\fBnotmuch-dump\fR(1), \fBnotmuch-hooks\fR(5), \fBnotmuch-reply\fR(1),
+\fBnotmuch-restore\fR(1), \fBnotmuch-search\fR(1),
+\fBnotmuch-search-terms\fR(7), \fBnotmuch-show\fR(1),
+\fBnotmuch-tag\fR(1)
-- 
1.7.12.1



[PATCH v6 03/12] cli: add insert command

2013-06-22 Thread Peter Wang
The notmuch insert command reads a message from standard input,
writes it to a Maildir folder, and then incorporates the message into
the notmuch database.  Essentially it moves the functionality of
notmuch-deliver into notmuch.

Though it could be used as an alternative to notmuch new, the reason
I want this is to allow my notmuch frontend to add postponed or sent
messages to the mail store and notmuch database, without resorting to
another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
---
 Makefile.local   |   1 +
 notmuch-client.h |   3 +
 notmuch-insert.c | 335 +++
 notmuch.c|   3 +
 4 files changed, 342 insertions(+)
 create mode 100644 notmuch-insert.c

diff --git a/Makefile.local b/Makefile.local
index 644623f..84043fe 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -261,6 +261,7 @@ notmuch_client_srcs =   \
notmuch-config.c\
notmuch-count.c \
notmuch-dump.c  \
+   notmuch-insert.c\
notmuch-new.c   \
notmuch-reply.c \
notmuch-restore.c   \
diff --git a/notmuch-client.h b/notmuch-client.h
index 4a3c7ac..dfe81e6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -181,6 +181,9 @@ int
 notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]);

 int
+notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]);
+
+int
 notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]);

 int
diff --git a/notmuch-insert.c b/notmuch-insert.c
new file mode 100644
index 000..1495234
--- /dev/null
+++ b/notmuch-insert.c
@@ -0,0 +1,335 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ? 2013 Peter Wang
+ *
+ * Based in part on notmuch-deliver
+ * Copyright ? 2010 Ali Polatel
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Peter Wang 
+ */
+
+#include "notmuch-client.h"
+#include "tag-util.h"
+
+#include 
+#include 
+#include 
+
+static volatile sig_atomic_t interrupted;
+
+static void
+handle_sigint (unused (int sig))
+{
+static char msg[] = "Stopping... \n";
+
+/* This write is "opportunistic", so it's okay to ignore the
+ * result.  It is not required for correctness, and if it does
+ * fail or produce a short write, we want to get out of the signal
+ * handler as quickly as possible, not retry it. */
+IGNORE_RESULT (write (2, msg, sizeof (msg) - 1));
+interrupted = 1;
+}
+
+/* Like gethostname but guarantees that a null-terminated hostname is
+ * returned, even if it has to make one up. Invalid characters are
+ * substituted such that the hostname can be used within a filename.
+ */
+static void
+safe_gethostname (char *hostname, size_t len)
+{
+char *p;
+
+if (gethostname (hostname, len) == -1) {
+   strncpy (hostname, "unknown", len);
+}
+hostname[len - 1] = '\0';
+
+for (p = hostname; *p != '\0'; p++) {
+   if (*p == '/' || *p == ':')
+   *p = '_';
+}
+}
+
+/* Call fsync() on a directory path. */
+static notmuch_bool_t
+sync_dir (const char *dir)
+{
+notmuch_bool_t ret;
+int fd;
+
+fd = open (dir, O_RDONLY);
+if (fd == -1) {
+   fprintf (stderr, "Error: open() dir failed: %s\n", strerror (errno));
+   return FALSE;
+}
+ret = (fsync (fd) == 0);
+if (! ret) {
+   fprintf (stderr, "Error: fsync() dir failed: %s\n", strerror (errno));
+}
+close (fd);
+return ret;
+}
+
+/* Open a unique file in the Maildir 'tmp' directory.
+ * Returns the file descriptor on success, or -1 on failure.
+ * On success, file paths for the message in the 'tmp' and 'new'
+ * directories are returned via tmppath and newpath,
+ * and the path of the 'new' directory itself in newdir. */
+static int
+maildir_open_tmp_file (void *ctx, const char *dir,
+  char **tmppath, char **newpath, char **newdir)
+{
+pid_t pid;
+char hostname[256];
+struct timeval tv;
+char *filename;
+int fd = -1;
+
+/* We follow the Dovecot file name generation algorithm. */
+pid = getpid ();
+safe_gethostname (hostname, siz

[PATCH v6 02/12] tag-util: do not reset list in parse_tag_command_line

2013-06-22 Thread Peter Wang
No current callers of parse_tag_command_line require that it clear its
tag list argument.  The notmuch 'insert' command will be better served
if the function modifies a pre-populated list (of new.tags) instead of
clobbering it outright.
---
 tag-util.c | 2 --
 tag-util.h | 2 ++
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tag-util.c b/tag-util.c
index 92e08a1..3bde409 100644
--- a/tag-util.c
+++ b/tag-util.c
@@ -165,8 +165,6 @@ parse_tag_command_line (void *ctx, int argc, char **argv,

 int i;

-tag_op_list_reset (tag_ops);
-
 for (i = 0; i < argc; i++) {
if (strcmp (argv[i], "--") == 0) {
i++;
diff --git a/tag-util.h b/tag-util.h
index 246de85..4628f16 100644
--- a/tag-util.h
+++ b/tag-util.h
@@ -81,6 +81,8 @@ parse_tag_line (void *ctx, char *line,
  * Output Parameters:
  * ops contains a list of tag operations
  * query_str the search terms.
+ *
+ * The ops argument is not cleared.
  */

 tag_parse_status_t
-- 
1.7.12.1



[PATCH v6 01/12] tag-util: move out 'tag' command-line checks

2013-06-22 Thread Peter Wang
parse_tag_command_line checked for two error conditions which are
specific to the 'tag' command.  It can be reused for the forthcoming
notmuch 'insert' command if we move the checks out, into notmuch-tag.c.
---
 notmuch-tag.c | 5 +
 tag-util.c| 6 +++---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/notmuch-tag.c b/notmuch-tag.c
index 9a5d3e7..3b09df9 100644
--- a/notmuch-tag.c
+++ b/notmuch-tag.c
@@ -247,6 +247,11 @@ notmuch_tag_command (notmuch_config_t *config, int argc, 
char *argv[])
fprintf (stderr, "Error: 'notmuch tag' requires at least one tag to 
add or remove.\n");
return 1;
}
+
+   if (*query_string == '\0') {
+   fprintf (stderr, "Error: notmuch tag requires at least one search 
term.\n");
+   return 1;
+   }
 }

 if (notmuch_database_open (notmuch_config_get_database_path (config),
diff --git a/tag-util.c b/tag-util.c
index c5f5859..92e08a1 100644
--- a/tag-util.c
+++ b/tag-util.c
@@ -190,9 +190,9 @@ parse_tag_command_line (void *ctx, int argc, char **argv,

 *query_str = query_string_from_args (ctx, argc - i, &argv[i]);

-if (*query_str == NULL || **query_str == '\0') {
-   fprintf (stderr, "Error: notmuch tag requires at least one search 
term.\n");
-   return TAG_PARSE_INVALID;
+if (*query_str == NULL) {
+   fprintf (stderr, "Out of memory.\n");
+   return TAG_PARSE_OUT_OF_MEMORY;
 }

 return TAG_PARSE_SUCCESS;
-- 
1.7.12.1



[PATCH v6 00/12] insert command

2013-06-22 Thread Peter Wang
Since v5:

- return success if the message was successfully added to the maildir
  even if we weren't able to add it to the notmuch database
- document exit status
- add a couple of tests
- don't print anything after parse_arguments returns failure

Peter Wang (12):
  tag-util: move out 'tag' command-line checks
  tag-util: do not reset list in parse_tag_command_line
  cli: add insert command
  man: document 'insert' command
  man: reference notmuch-insert.1
  test: add tests for insert
  insert: add --folder option
  man: document insert --folder option
  test: test insert --folder option
  insert: add --create-folder option
  man: document insert --create-folder
  test: test insert --create-folder option

 Makefile.local  |   1 +
 man/Makefile.local  |   1 +
 man/man1/notmuch-config.1   |   4 +-
 man/man1/notmuch-count.1|   4 +-
 man/man1/notmuch-dump.1 |   4 +-
 man/man1/notmuch-insert.1   |  75 +++
 man/man1/notmuch-new.1  |   4 +-
 man/man1/notmuch-reply.1|   3 +-
 man/man1/notmuch-restore.1  |   3 +-
 man/man1/notmuch-search.1   |   3 +-
 man/man1/notmuch-show.1 |   3 +-
 man/man1/notmuch-tag.1  |   3 +-
 man/man1/notmuch.1  |   3 +-
 man/man5/notmuch-hooks.5|   4 +-
 man/man7/notmuch-search-terms.7 |   3 +-
 notmuch-client.h|   3 +
 notmuch-insert.c| 477 
 notmuch-tag.c   |   5 +
 notmuch.c   |   3 +
 tag-util.c  |   8 +-
 tag-util.h  |   2 +
 test/insert | 121 ++
 test/notmuch-test   |   1 +
 23 files changed, 716 insertions(+), 22 deletions(-)
 create mode 100644 man/man1/notmuch-insert.1
 create mode 100644 notmuch-insert.c
 create mode 100755 test/insert

-- 
1.7.12.1



[PATCH v3 7/8] lib: add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t

2013-06-22 Thread Peter Wang
On Fri, 21 Jun 2013 13:59:57 +0300, Tomi Ollila  wrote:
> On Thu, Jun 20 2013, Mark Walters  wrote:
> 
> > I think we should decide before 0.16 whether to go with this patch and
> > patch 8/8 or for Peter's suggestion at
> > id:1368454815-1854-1-git-send-email-novalazy at gmail.com
> >
> > Now we have an enum for the NOTMUCH_EXCLUDE possibilities (rather than a
> > bool) I think it makes sense to implement NOTMUCH_EXCLUDE_FALSE properly
> > but I am happy with either choice.
> 
> So, the choice here is to choose between
> 
> id:"1368454815-1854-1-git-send-email-novalazy at gmail.com"
> and
> id:"87bo8edif8.fsf at qmul.ac.uk" 
> 
> (might be hard to catch from this thread -- was easier to take from
> http://nmbug.tethera.net/status/ )
> 
> Both apply cleanly to current master ( d0bd88f )
> 
> While Peter's version surely looks simpler (and may work, didn't test atm)
> the comments Mark presents makes sense (especially the "subtlety" thing ;)
> 
> IMHO Mark's version makes future development "safer" and therefore my
> vote (or million of those) goes to id:"87bo8edif8.fsf at qmul.ac.uk"

Sure, I don't care.

Peter


[PATCH v5 03/12] cli: add insert command

2013-05-28 Thread Peter Wang
On Sun, 28 Apr 2013 00:24:28 +0300, Jani Nikula  wrote:
> On Wed, 03 Apr 2013, Peter Wang  wrote:
> > The notmuch insert command reads a message from standard input,
> > writes it to a Maildir folder, and then incorporates the message into
> > the notmuch database.  Essentially it moves the functionality of
> > notmuch-deliver into notmuch.
> >
> > Though it could be used as an alternative to notmuch new, the reason
> > I want this is to allow my notmuch frontend to add postponed or sent
> > messages to the mail store and notmuch database, without resorting to
> > another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
> > ---
> >  Makefile.local   |   1 +
> >  notmuch-client.h |   3 +
> >  notmuch-insert.c | 336 
> > +++
> >  notmuch.c|   3 +
> >  4 files changed, 343 insertions(+)
> >  create mode 100644 notmuch-insert.c
> >
...
> > +/* Add the specified message file to the notmuch database, applying tags.
> > + * The file is renamed to encode notmuch tags as maildir flags. */
> > +static notmuch_bool_t
> > +add_file_to_database (notmuch_database_t *notmuch, const char *path,
> > + tag_op_list_t *tag_ops)
> > +{
> > +notmuch_message_t *message;
> > +notmuch_status_t status;
> > +
> > +status = notmuch_database_add_message (notmuch, path, &message);
> > +switch (status) {
> > +case NOTMUCH_STATUS_SUCCESS:
> > +case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
> > +   break;
> > +default:
> > +case NOTMUCH_STATUS_FILE_NOT_EMAIL:
> 
> If such a message really arrives, the mail system will keep trying if
> failure is returned. Maybe deliver the file without indexing, and return
> success?
> 

Rethinking it, if notmuch insert is going to used as a general mail
delivery tool (not my own use case) then its primary job should be to
get the file to disk.  As long as that is done, we should return success.

Indexing the message would be considered a bonus, and failure there
or in syncing tags to flags should not cause the file to be deleted and
an error code returned.  (A warning can be written to standard error.)

> > +case NOTMUCH_STATUS_READ_ONLY_DATABASE:
> > +case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
> > +case NOTMUCH_STATUS_OUT_OF_MEMORY:
> > +case NOTMUCH_STATUS_FILE_ERROR:
> > +case NOTMUCH_STATUS_NULL_POINTER:
> > +case NOTMUCH_STATUS_TAG_TOO_LONG:
> > +case NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW:
> > +case NOTMUCH_STATUS_UNBALANCED_ATOMIC:
> > +case NOTMUCH_STATUS_LAST_STATUS:
> > +   fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
> > +path, notmuch_status_to_string (status));
> > +   return FALSE;
> > +}
> > +
> > +if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
> > +   /* Don't change tags of an existing message. */
> > +   status = notmuch_message_tags_to_maildir_flags (message);
> > +   if (status != NOTMUCH_STATUS_SUCCESS)
> > +   fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
> > +} else {
> > +   status = tag_op_list_apply (message, tag_ops, TAG_FLAG_MAILDIR_SYNC);
> 
> Syncing tags to maildir flags is more interesting here than above. And
> it should be done because notmuch insert allows arbitrary tags on the
> command line. Having, for example, -unread or +flagged on the command
> line makes the flags go out of sync. (notmuch new should do the syncing
> too, but it's less important because it only adds new.tags.)
> 
> However, calling notmuch_message_tags_to_maildir_flags() may rename the
> file from new to cur, which blows up the directory syncing and file
> unlinking on the error path in insert_message() below.

We would sidestep these problems.

> > +static notmuch_bool_t
> > +insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
> > +   const char *dir, tag_op_list_t *tag_ops)
> > +{
> > +char *tmppath;
> > +char *newpath;
> > +char *newdir;
> > +int fdout;
> > +char *cleanup_path;
> > +
> > +fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
> > +if (fdout < 0)
> > +   return FALSE;
> > +
> > +cleanup_path = tmppath;
> > +
> > +if (! copy_stdin (fdin, fdout))
> > +   goto FAIL;
> > +
> > +if (fsync (fdout) != 0) {
> > +   fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
> > +   goto FAIL;
> > + 

Re: [PATCH v5 03/12] cli: add insert command

2013-05-28 Thread Peter Wang
On Sun, 28 Apr 2013 00:24:28 +0300, Jani Nikula  wrote:
> On Wed, 03 Apr 2013, Peter Wang  wrote:
> > The notmuch insert command reads a message from standard input,
> > writes it to a Maildir folder, and then incorporates the message into
> > the notmuch database.  Essentially it moves the functionality of
> > notmuch-deliver into notmuch.
> >
> > Though it could be used as an alternative to notmuch new, the reason
> > I want this is to allow my notmuch frontend to add postponed or sent
> > messages to the mail store and notmuch database, without resorting to
> > another tool (e.g. notmuch-deliver) nor directly modifying the maildir.
> > ---
> >  Makefile.local   |   1 +
> >  notmuch-client.h |   3 +
> >  notmuch-insert.c | 336 
> > +++
> >  notmuch.c|   3 +
> >  4 files changed, 343 insertions(+)
> >  create mode 100644 notmuch-insert.c
> >
...
> > +/* Add the specified message file to the notmuch database, applying tags.
> > + * The file is renamed to encode notmuch tags as maildir flags. */
> > +static notmuch_bool_t
> > +add_file_to_database (notmuch_database_t *notmuch, const char *path,
> > + tag_op_list_t *tag_ops)
> > +{
> > +notmuch_message_t *message;
> > +notmuch_status_t status;
> > +
> > +status = notmuch_database_add_message (notmuch, path, &message);
> > +switch (status) {
> > +case NOTMUCH_STATUS_SUCCESS:
> > +case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
> > +   break;
> > +default:
> > +case NOTMUCH_STATUS_FILE_NOT_EMAIL:
> 
> If such a message really arrives, the mail system will keep trying if
> failure is returned. Maybe deliver the file without indexing, and return
> success?
> 

Rethinking it, if notmuch insert is going to used as a general mail
delivery tool (not my own use case) then its primary job should be to
get the file to disk.  As long as that is done, we should return success.

Indexing the message would be considered a bonus, and failure there
or in syncing tags to flags should not cause the file to be deleted and
an error code returned.  (A warning can be written to standard error.)

> > +case NOTMUCH_STATUS_READ_ONLY_DATABASE:
> > +case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
> > +case NOTMUCH_STATUS_OUT_OF_MEMORY:
> > +case NOTMUCH_STATUS_FILE_ERROR:
> > +case NOTMUCH_STATUS_NULL_POINTER:
> > +case NOTMUCH_STATUS_TAG_TOO_LONG:
> > +case NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW:
> > +case NOTMUCH_STATUS_UNBALANCED_ATOMIC:
> > +case NOTMUCH_STATUS_LAST_STATUS:
> > +   fprintf (stderr, "Error: failed to add `%s' to notmuch database: %s\n",
> > +path, notmuch_status_to_string (status));
> > +   return FALSE;
> > +}
> > +
> > +if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
> > +   /* Don't change tags of an existing message. */
> > +   status = notmuch_message_tags_to_maildir_flags (message);
> > +   if (status != NOTMUCH_STATUS_SUCCESS)
> > +   fprintf (stderr, "Error: failed to sync tags to maildir flags\n");
> > +} else {
> > +   status = tag_op_list_apply (message, tag_ops, TAG_FLAG_MAILDIR_SYNC);
> 
> Syncing tags to maildir flags is more interesting here than above. And
> it should be done because notmuch insert allows arbitrary tags on the
> command line. Having, for example, -unread or +flagged on the command
> line makes the flags go out of sync. (notmuch new should do the syncing
> too, but it's less important because it only adds new.tags.)
> 
> However, calling notmuch_message_tags_to_maildir_flags() may rename the
> file from new to cur, which blows up the directory syncing and file
> unlinking on the error path in insert_message() below.

We would sidestep these problems.

> > +static notmuch_bool_t
> > +insert_message (void *ctx, notmuch_database_t *notmuch, int fdin,
> > +   const char *dir, tag_op_list_t *tag_ops)
> > +{
> > +char *tmppath;
> > +char *newpath;
> > +char *newdir;
> > +int fdout;
> > +char *cleanup_path;
> > +
> > +fdout = maildir_open_tmp_file (ctx, dir, &tmppath, &newpath, &newdir);
> > +if (fdout < 0)
> > +   return FALSE;
> > +
> > +cleanup_path = tmppath;
> > +
> > +if (! copy_stdin (fdin, fdout))
> > +   goto FAIL;
> > +
> > +if (fsync (fdout) != 0) {
> > +   fprintf (stderr, "Error: fsync failed: %s\n", strerror (errno));
> > +   goto FAIL;
> > + 

[PATCH v2] cli: clarify correspondence of --exclude to omit_excluded in search

2013-05-14 Thread Peter Wang
Make it obvious how the --exclude command-line option affects the
omit_excluded field in notmuch_query_t objects, with an explicit and
exhaustive switch.  Do not expect the reader to know the default value
of omit_excluded.
---
This can be inserted after patch 2.

 notmuch-search.c | 23 +++
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 4323201..893df10 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -290,6 +290,24 @@ enum {
 EXCLUDE_ALL
 };

+static int
+exclude_option_to_omit_excluded (int exclude)
+{
+switch (exclude) {
+case EXCLUDE_TRUE:
+   return NOTMUCH_EXCLUDE_TRUE;
+case EXCLUDE_FALSE:
+   return NOTMUCH_EXCLUDE_FALSE;
+case EXCLUDE_FLAG:
+   return NOTMUCH_EXCLUDE_FALSE;
+case EXCLUDE_ALL:
+   return NOTMUCH_EXCLUDE_ALL;
+default:
+   INTERNAL_ERROR ("unhandled exclude option %d", exclude);
+   /*UNREACHED*/
+}
+}
+
 int
 notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
 {
@@ -410,11 +428,8 @@ notmuch_search_command (notmuch_config_t *config, int 
argc, char *argv[])
(config, &search_exclude_tags_length);
for (i = 0; i < search_exclude_tags_length; i++)
notmuch_query_add_tag_exclude (query, search_exclude_tags[i]);
-   if (exclude == EXCLUDE_FLAG)
-   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FALSE);
-   if (exclude == EXCLUDE_ALL)
-   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_ALL);
 }
+notmuch_query_set_omit_excluded (query, exclude_option_to_omit_excluded 
(exclude));

 switch (output) {
 default:
-- 
1.7.12.1



[PATCH v2 7/8] lib: add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t

2013-05-13 Thread Peter Wang
On Sun, 12 May 2013 08:20:04 -0300, David Bremner  wrote:
> Mark Walters  writes:
> 
> > -   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FALSE);
> > +   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FLAG);
> 
> Shouldn't the documentation be updated to cover NOTMUCH_EXCLUDE_FLAG?  I
> realize it is not _wrong_ as written, but it is a bit confusing not to
> cover all possible values of the enum.

Actually, please drop patch 7 and patch 8.  It was wrong to combine the
CLI-level EXCLUDE_* constants with the lib-level NOTMUCH_EXCLUDE_* constants.

Peter


[PATCH v2] cli: clarify correspondence of --exclude to omit_excluded in search

2013-05-13 Thread Peter Wang
Make it obvious how the --exclude command-line option affects the
omit_excluded field in notmuch_query_t objects, with an explicit and
exhaustive switch.  Do not expect the reader to know the default value
of omit_excluded.
---
This can be inserted after patch 2.

 notmuch-search.c | 23 +++
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 4323201..893df10 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -290,6 +290,24 @@ enum {
 EXCLUDE_ALL
 };
 
+static int
+exclude_option_to_omit_excluded (int exclude)
+{
+switch (exclude) {
+case EXCLUDE_TRUE:
+   return NOTMUCH_EXCLUDE_TRUE;
+case EXCLUDE_FALSE:
+   return NOTMUCH_EXCLUDE_FALSE;
+case EXCLUDE_FLAG:
+   return NOTMUCH_EXCLUDE_FALSE;
+case EXCLUDE_ALL:
+   return NOTMUCH_EXCLUDE_ALL;
+default:
+   INTERNAL_ERROR ("unhandled exclude option %d", exclude);
+   /*UNREACHED*/
+}
+}
+
 int
 notmuch_search_command (notmuch_config_t *config, int argc, char *argv[])
 {
@@ -410,11 +428,8 @@ notmuch_search_command (notmuch_config_t *config, int 
argc, char *argv[])
(config, &search_exclude_tags_length);
for (i = 0; i < search_exclude_tags_length; i++)
notmuch_query_add_tag_exclude (query, search_exclude_tags[i]);
-   if (exclude == EXCLUDE_FLAG)
-   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FALSE);
-   if (exclude == EXCLUDE_ALL)
-   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_ALL);
 }
+notmuch_query_set_omit_excluded (query, exclude_option_to_omit_excluded 
(exclude));
 
 switch (output) {
 default:
-- 
1.7.12.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v2 7/8] lib: add NOTMUCH_EXCLUDE_FLAG to notmuch_exclude_t

2013-05-13 Thread Peter Wang
On Sun, 12 May 2013 08:20:04 -0300, David Bremner  wrote:
> Mark Walters  writes:
> 
> > -   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FALSE);
> > +   notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_FLAG);
> 
> Shouldn't the documentation be updated to cover NOTMUCH_EXCLUDE_FLAG?  I
> realize it is not _wrong_ as written, but it is a bit confusing not to
> cover all possible values of the enum.

Actually, please drop patch 7 and patch 8.  It was wrong to combine the
CLI-level EXCLUDE_* constants with the lib-level NOTMUCH_EXCLUDE_* constants.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v5 07/12] insert: add --folder option

2013-04-28 Thread Peter Wang
On Sun, 28 Apr 2013 00:32:47 +0300, Jani Nikula  wrote:
> > +notmuch_opt_desc_t options[] = {
> > +   { NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
> > +   { NOTMUCH_OPT_END, 0, 0, 0, 0 }
> > +};
> > +
> > +opt_index = parse_arguments (argc, argv, options, 1);
> > +
> > +if (opt_index < 0) {
> > +   fprintf (stderr, "Error: bad argument to notmuch insert: %s\n",
> > +argv[-opt_index]);
> 
> I'm too tired to check what's correct, but argv[-opt_index] isn't.

Nice catch.  I'll just delete that message as parse_arguments
already prints a message, like other callers.

Peter


Re: [PATCH v5 07/12] insert: add --folder option

2013-04-27 Thread Peter Wang
On Sun, 28 Apr 2013 00:32:47 +0300, Jani Nikula  wrote:
> > +notmuch_opt_desc_t options[] = {
> > +   { NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
> > +   { NOTMUCH_OPT_END, 0, 0, 0, 0 }
> > +};
> > +
> > +opt_index = parse_arguments (argc, argv, options, 1);
> > +
> > +if (opt_index < 0) {
> > +   fprintf (stderr, "Error: bad argument to notmuch insert: %s\n",
> > +argv[-opt_index]);
> 
> I'm too tired to check what's correct, but argv[-opt_index] isn't.

Nice catch.  I'll just delete that message as parse_arguments
already prints a message, like other callers.

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[announce] Bower 0.5

2013-04-20 Thread Peter Wang
Hi,

Bower is yet another curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.5 (2013-04-20)
==

* Add 'a' add to addressbook action.
* Add 'B' resend message action.
* Add 'z' to cycle alternatives to highlighted part.
* Make 'O' toggle message ordering without prompting.
* Toggling message ordering need not call notmuch again.
* Add completion for search aliases.
* Add completion and expansion of tilde (~) to home directory in paths.
* Show approximate attachment sizes in pager (requires notmuch 0.15).
* Handle interrupts properly.
* ^C interrupts a running notmuch search without quitting bower.
* Don't abort when unable to parse invalid json output from notmuch.
* Pass sendmail -t option ourselves; disallow it in config.
* Tell lynx to output UTF-8 by default.
* Replace colons in generated filenames when saving.
* Fixes for URL detection algorithm.
* Use _NSGetEnviron to get environ on Darwin (reported by Blake Sweeney).

Peter


[announce] Bower 0.5

2013-04-19 Thread Peter Wang
Hi,

Bower is yet another curses frontend for the Notmuch email system.
I wrote it for me, but you might like it, too.

https://github.com/wangp/bower

Bower 0.5 (2013-04-20)
==

* Add 'a' add to addressbook action.
* Add 'B' resend message action.
* Add 'z' to cycle alternatives to highlighted part.
* Make 'O' toggle message ordering without prompting.
* Toggling message ordering need not call notmuch again.
* Add completion for search aliases.
* Add completion and expansion of tilde (~) to home directory in paths.
* Show approximate attachment sizes in pager (requires notmuch 0.15).
* Handle interrupts properly.
* ^C interrupts a running notmuch search without quitting bower.
* Don't abort when unable to parse invalid json output from notmuch.
* Pass sendmail -t option ourselves; disallow it in config.
* Tell lynx to output UTF-8 by default.
* Replace colons in generated filenames when saving.
* Fixes for URL detection algorithm.
* Use _NSGetEnviron to get environ on Darwin (reported by Blake Sweeney).

Peter
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v5 12/12] test: test insert --create-folder option

2013-04-03 Thread Peter Wang
Add tests for notmuch insert --create-folder option.
---
 test/insert | 24 
 1 file changed, 24 insertions(+)

diff --git a/test/insert b/test/insert
index 44e071c..24a61e1 100755
--- a/test/insert
+++ b/test/insert
@@ -83,4 +83,28 @@ gen_insert_msg
 test_expect_code 1 "Insert message, non-existent folder" \
 "notmuch insert --folder=nonesuch < $gen_msg_filename"

+test_begin_subtest "Insert message, create folder"
+gen_insert_msg
+notmuch insert --folder=F --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" "$MAIL_DIR/F/cur/${basename}"
+
+test_begin_subtest "Insert message, create subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:F/G/H/I/J tag:folder)
+basename=$(basename "$output")
+test_expect_equal_file "$gen_msg_filename" 
"${MAIL_DIR}/F/G/H/I/J/cur/${basename}"
+
+test_begin_subtest "Insert message, create existing subfolder"
+gen_insert_msg
+notmuch insert --folder=F/G/H/I/J --create-folder +folder < "$gen_msg_filename"
+output=$(notmuch count folder:F/G/H/I/J tag:folder)
+test_expect_equal "$output" "2"
+
+gen_insert_msg
+test_expect_code 1 "Insert message, create invalid subfolder" \
+"notmuch insert --folder=../G --create-folder $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v5 11/12] man: document insert --create-folder

2013-04-03 Thread Peter Wang
Add documentation for notmuch insert --create-folder option.
---
 man/man1/notmuch-insert.1 | 12 
 1 file changed, 12 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index 0d7bccd..74d6a3d 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -43,6 +43,18 @@ relative to the top-level directory given by the value of
 The default is to deliver to the top-level directory.

 .RE
+
+.RS 4
+.TP 4
+.B "--create-folder"
+
+Try to create the folder named by the
+.B "--folder"
+option, if it does not exist.
+Otherwise the folder must already exist for mail
+delivery to succeed.
+
+.RE
 .SH SEE ALSO

 \fBnotmuch\fR(1), \fBnotmuch-config\fR(1), \fBnotmuch-count\fR(1),
-- 
1.7.12.1



[PATCH v5 10/12] insert: add --create-folder option

2013-04-03 Thread Peter Wang
Allow the insert command to create the maildir folder
into which the new message should be delivered.
---
 notmuch-insert.c | 100 +++
 1 file changed, 100 insertions(+)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 778ac04..8ae5dc9 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -100,6 +100,99 @@ check_folder_name (const char *folder)
 }
 }

+/* Make the given directory, succeeding if it already exists. */
+static notmuch_bool_t
+make_directory (char *path, int mode)
+{
+notmuch_bool_t ret;
+char *slash;
+
+if (mkdir (path, mode) != 0)
+   return (errno == EEXIST);
+
+/* Sync the parent directory for durability. */
+ret = TRUE;
+slash = strrchr (path, '/');
+if (slash) {
+   *slash = '\0';
+   ret = sync_dir (path);
+   *slash = '/';
+}
+return ret;
+}
+
+/* Make the given directory including its parent directories as necessary.
+ * Return TRUE on success, FALSE on error. */
+static notmuch_bool_t
+make_directory_and_parents (char *path, int mode)
+{
+struct stat st;
+char *start;
+char *end;
+notmuch_bool_t ret;
+
+/* First check the common case: directory already exists. */
+if (stat (path, &st) == 0)
+   return S_ISDIR (st.st_mode) ? TRUE : FALSE;
+
+for (start = path; *start != '\0'; start = end + 1) {
+   /* start points to the first unprocessed character.
+* Find the next slash from start onwards. */
+   end = strchr (start, '/');
+
+   /* If there are no more slashes then all the parent directories
+* have been made.  Now attempt to make the whole path. */
+   if (end == NULL)
+   return make_directory (path, mode);
+
+   /* Make the path up to the next slash, unless the current
+* directory component is actually empty. */
+   if (end > start) {
+   *end = '\0';
+   ret = make_directory (path, mode);
+   *end = '/';
+   if (! ret)
+   return FALSE;
+   }
+}
+
+return TRUE;
+}
+
+/* Create the given maildir folder, i.e. dir and its subdirectories
+ * 'cur', 'new', 'tmp'. */
+static notmuch_bool_t
+maildir_create_folder (void *ctx, const char *dir)
+{
+const int mode = 0700;
+char *subdir;
+char *tail;
+
+/* Create 'cur' directory, including parent directories. */
+subdir = talloc_asprintf (ctx, "%s/cur", dir);
+if (! subdir) {
+   fprintf (stderr, "Out of memory.\n");
+   return FALSE;
+}
+if (! make_directory_and_parents (subdir, mode))
+   return FALSE;
+
+tail = subdir + strlen (subdir) - 3;
+
+/* Create 'new' directory. */
+strcpy (tail, "new");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+/* Create 'tmp' directory. */
+strcpy (tail, "tmp");
+if (! make_directory (subdir, mode))
+   return FALSE;
+
+talloc_free (subdir);
+return TRUE;
+}
+
 /* Open a unique file in the Maildir 'tmp' directory.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -305,6 +398,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
 const char *folder = NULL;
+notmuch_bool_t create_folder = FALSE;
 const char *maildir;
 int opt_index;
 unsigned int i;
@@ -312,6 +406,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])

 notmuch_opt_desc_t options[] = {
{ NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_BOOLEAN, &create_folder, "create-folder", 0, 0 },
{ NOTMUCH_OPT_END, 0, 0, 0, 0 }
 };

@@ -357,6 +452,11 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
fprintf (stderr, "Out of memory\n");
return 1;
}
+   if (create_folder && ! maildir_create_folder (config, maildir)) {
+   fprintf (stderr, "Error: creating maildir %s: %s\n",
+maildir, strerror (errno));
+   return 1;
+   }
 }

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
-- 
1.7.12.1



[PATCH v5 09/12] test: test insert --folder option

2013-04-03 Thread Peter Wang
Add tests for notmuch insert --folder option.
---
 test/insert | 17 +
 1 file changed, 17 insertions(+)

diff --git a/test/insert b/test/insert
index d880af9..44e071c 100755
--- a/test/insert
+++ b/test/insert
@@ -66,4 +66,21 @@ notmuch insert +custom -unread < "$gen_msg_filename"
 output=$(notmuch count tag:custom NOT tag:unread)
 test_expect_equal "$output" "1"

+test_begin_subtest "Insert message, folder"
+gen_insert_msg
+notmuch insert --folder=Drafts < "$gen_msg_filename"
+output=$(notmuch search --output=files folder:Drafts)
+dirname=$(dirname "$output")
+test_expect_equal "$dirname" "$MAIL_DIR/Drafts/cur"
+
+test_begin_subtest "Insert message, folder and tags"
+gen_insert_msg
+notmuch insert --folder=Drafts +draft -unread < "$gen_msg_filename"
+output=$(notmuch count folder:Drafts tag:draft NOT tag:unread)
+test_expect_equal "$output" "1"
+
+gen_insert_msg
+test_expect_code 1 "Insert message, non-existent folder" \
+"notmuch insert --folder=nonesuch < $gen_msg_filename"
+
 test_done
-- 
1.7.12.1



[PATCH v5 08/12] man: document insert --folder option

2013-04-03 Thread Peter Wang
Add documentation for notmuch insert --folder option.
---
 man/man1/notmuch-insert.1 | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/man/man1/notmuch-insert.1 b/man/man1/notmuch-insert.1
index 870e1bc..0d7bccd 100644
--- a/man/man1/notmuch-insert.1
+++ b/man/man1/notmuch-insert.1
@@ -4,6 +4,7 @@ notmuch-insert \- add a message to the maildir and notmuch 
database
 .SH SYNOPSIS

 .B notmuch insert
+.RI "[" options "]"
 .RI "[ +<" tag> "|\-<" tag "> ... ]"

 .SH DESCRIPTION
@@ -28,6 +29,19 @@ If the new message is a duplicate of an existing message in 
the database
 (it has same Message-ID), it will be added to the maildir folder and
 notmuch database, but the tags will not be changed.

+Option arguments must appear before any tag operation arguments.
+Supported options for
+.B insert
+include
+.RS 4
+.TP 4
+.BI "--folder=<" folder ">"
+
+Deliver the message to the specified folder,
+relative to the top-level directory given by the value of
+\fBdatabase.path\fR.
+The default is to deliver to the top-level directory.
+
 .RE
 .SH SEE ALSO

-- 
1.7.12.1



[PATCH v5 07/12] insert: add --folder option

2013-04-03 Thread Peter Wang
Allow the new message to be inserted into a folder within the Maildir
hierarchy instead of the top-level folder.
---
 notmuch-insert.c | 47 +--
 1 file changed, 45 insertions(+), 2 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 19b1cf9..778ac04 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -83,6 +83,23 @@ sync_dir (const char *dir)
 return ret;
 }

+/* Check the specified folder name does not contain a directory
+ * component ".." to prevent writes outside of the Maildir hierarchy. */
+static notmuch_bool_t
+check_folder_name (const char *folder)
+{
+const char *p = folder;
+
+for (;;) {
+   if ((p[0] == '.') && (p[1] == '.') && (p[2] == '\0' || p[2] == '/'))
+   return FALSE;
+   p = strchr (p, '/');
+   if (!p)
+   return TRUE;
+   p++;
+}
+}
+
 /* Open a unique file in the Maildir 'tmp' directory.
  * Returns the file descriptor on success, or -1 on failure.
  * On success, file paths for the message in the 'tmp' and 'new'
@@ -287,11 +304,25 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
 size_t new_tags_length;
 tag_op_list_t *tag_ops;
 char *query_string = NULL;
+const char *folder = NULL;
 const char *maildir;
-int opt_index = 1;
+int opt_index;
 unsigned int i;
 notmuch_bool_t ret;

+notmuch_opt_desc_t options[] = {
+   { NOTMUCH_OPT_STRING, &folder, "folder", 0, 0 },
+   { NOTMUCH_OPT_END, 0, 0, 0, 0 }
+};
+
+opt_index = parse_arguments (argc, argv, options, 1);
+
+if (opt_index < 0) {
+   fprintf (stderr, "Error: bad argument to notmuch insert: %s\n",
+argv[-opt_index]);
+   return 1;
+}
+
 db_path = notmuch_config_get_database_path (config);
 new_tags = notmuch_config_get_new_tags (config, &new_tags_length);

@@ -314,7 +345,19 @@ notmuch_insert_command (notmuch_config_t *config, int 
argc, char *argv[])
return 1;
 }

-maildir = db_path;
+if (folder == NULL) {
+   maildir = db_path;
+} else {
+   if (! check_folder_name (folder)) {
+   fprintf (stderr, "Error: bad folder name: %s\n", folder);
+   return 1;
+   }
+   maildir = talloc_asprintf (config, "%s/%s", db_path, folder);
+   if (! maildir) {
+   fprintf (stderr, "Out of memory\n");
+   return 1;
+   }
+}

 /* Setup our handler for SIGINT. We do not set SA_RESTART so that copying
  * from standard input may be interrupted. */
-- 
1.7.12.1



[PATCH v5 06/12] test: add tests for insert

2013-04-03 Thread Peter Wang
Add tests for new 'insert' command.
---
 test/insert   | 69 +++
 test/notmuch-test |  1 +
 2 files changed, 70 insertions(+)
 create mode 100755 test/insert

diff --git a/test/insert b/test/insert
new file mode 100755
index 000..d880af9
--- /dev/null
+++ b/test/insert
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+test_description='"notmuch insert"'
+. ./test-lib.sh
+
+# Create directories and database before inserting.
+mkdir -p "$MAIL_DIR"/{cur,new,tmp}
+mkdir -p "$MAIL_DIR"/Drafts/{cur,new,tmp}
+notmuch new > /dev/null
+
+# We use generate_message to create the temporary message files.
+# They happen to be in the mail directory already but that is okay
+# since we do not call notmuch new hereafter.
+
+gen_insert_msg() {
+generate_message \
+   "[subject]=\"insert-subject\"" \
+   "[date]=\"Sat, 01 Jan 2000 12:00:00 -\"" \
+   "[body]=\"insert-message\""
+}
+
+test_begin_subtest "Insert message, copied exactly"
+gen_insert_msg
+notmuch insert < "$gen_msg_filename"
+cur_msg_filename=$(notmuch search --output=files "subject:insert-subject")
+test_expect_equal_file "$cur_msg_filename" "$gen_msg_filename"
+
+test_begin_subtest "Insert message, default tags"
+output=$(notmuch show --format=json "subject:insert-subject")
+expected='[[[{
+ "id": "'"${gen_msg_id}"'",
+ "match": true,
+ "excluded": false,
+ "filename": "'"${cur_msg_filename}"'",
+ "timestamp": 946728000,
+ "date_relative": "2000-01-01",
+ "tags": ["inbox","unread"],
+ "headers": {
+  "Subject": "insert-subject",
+  "From": "Notmuch Test Suite ",
+  "To": "Notmuch Test Suite ",
+  "Date": "Sat, 01 Jan 2000 12:00:00 +"},
+ "body": [{"id": 1,
+  "content-type": "text/plain",
+  "content": "insert-message\n"}]},
+ ['
+test_expect_equal_json "$output" "$expected"
+
+test_begin_subtest "Insert message, duplicate message"
+notmuch insert +duptag -unread < "$gen_msg_filename"
+output=$(notmuch search --output=files "subject:insert-subject" | wc -l)
+test_expect_equal "$output" 2
+
+test_begin_subtest "Insert message, duplicate message does not change tags"
+output=$(notmuch search --format=json --output=tags "subject:insert-subject")
+test_expect_equal_json "$output" '["inbox", "unread"]'
+
+test_begin_subtest "Insert message, add tag"
+gen_insert_msg
+notmuch insert +custom < "$gen_msg_filename"
+output=$(notmuch count tag:custom)
+test_expect_equal "$output" "1"
+
+test_begin_subtest "Insert message, add/remove tag"
+gen_insert_msg
+notmuch insert +custom -unread < "$gen_msg_filename"
+output=$(notmuch count tag:custom NOT tag:unread)
+test_expect_equal "$output" "1"
+
+test_done
diff --git a/test/notmuch-test b/test/notmuch-test
index ca9c3dc..6952f0a 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -22,6 +22,7 @@ TESTS="
   config
   new
   count
+  insert
   search
   search-output
   search-by-folder
-- 
1.7.12.1



  1   2   3   4   5   >