[notmuch] Search body?

2010-03-18 Thread Sandra Snan
I  wrote:
> is there a way to search just the body?

To answer my own question (in case others have the same question)?no,
not currently. I was grepping the sources but it didn?t occur to me to
look in lib/.
database.cc had the answer.

Sandra


[notmuch] Search body?

2010-03-18 Thread Sandra Snan
I?m glad that the default search is free text including headers and
body, and that there are ways to search just the subject, or just the
sender and so on, but is there a way to search just the body? I
currently use a lot of ?sandra and not from:sandra? and so on.


[notmuch] [PATCH] Add count command to manual page

2010-03-18 Thread Sandra Snan
Just found out that I?ve been piping things through wc for no reason.
Heh. Here?s a quick patch, if you want to add this to the manual page.

Signed-off-by: Sandra Snan 
---
 notmuch.1 |   18 +-
 1 files changed, 17 insertions(+), 1 deletions(-)

diff --git a/notmuch.1 b/notmuch.1
index e573749..4d300c0 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -126,7 +126,7 @@ syntax. See the
 section below for more details on the supported syntax.

 The
-.BR search " and "show
+.BR "search" ", " "show" " and " "count"
 commands are used to query the email database.
 .RS 4
 .TP 4
@@ -239,6 +239,22 @@ See the
 .B "SEARCH SYNTAX"
 section below for details of the supported syntax for .
 .RE
+.TP
+.BR count " ..."
+
+Count messages matching the search terms.
+
+The number of matching messages is output to stdout.
+
+A common use of
+.B notmuch count
+is to display the count of messages matching both a specific tag and
+either inbox or unread
+
+See the
+.B "SEARCH SYNTAX"
+section below for details of the supported syntax for .
+.RE
 .RE

 The
-- 
1.7.0



[notmuch] [PATCH 4/4] Tests for maildir-based mailstore

2010-03-18 Thread Michal Sojka
Signed-off-by: Michal Sojka 
---
 test/t0006-maildir.sh |  113 +
 test/test-lib.sh  |7 ++-
 2 files changed, 118 insertions(+), 2 deletions(-)
 create mode 100755 test/t0006-maildir.sh

diff --git a/test/t0006-maildir.sh b/test/t0006-maildir.sh
new file mode 100755
index 000..e584908
--- /dev/null
+++ b/test/t0006-maildir.sh
@@ -0,0 +1,113 @@
+#!/bin/bash
+
+test_description="Test maildir mailstore"
+
+. ./test-lib.sh
+
+filter_output() {
+grep -v -E -e "$NOTMUCH_IGNORED_OUTPUT_REGEXP" | sed -e 
"$NOTMUCH_THREAD_ID_SQUELCH"
+}
+
+filter_show() {
+sed -e 's/, /,\n/g'|sed -e '/^"filename"/ s/:2,[A-Z]*//' -e '/^"tags"/d'
+}
+
+cat >> "$NOTMUCH_CONFIG" < expected < actual &&
+test_cmp expected actual
+#emacs --eval "(gdb \"gdb --annotate=3 --args $(which notmuch) new\")"
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+test_expect_success 'Tag the seen messages as replied' '
+notmuch tag +replied -inbox tag:inbox and not tag:unread
+'
+
+cat > expected < actual &&
+test_cmp expected actual
+'
+echo -n '[[[{"id": "msg-001 at notmuch-test-suite",
+"match": true,
+"filename": "/home/wsh/src/notmuch/test/trash 
directory.t0006-maildir/mail/msg-001",
+"headers": {"Subject": "test message",
+"From": "Notmuch Test Suite ",
+"To": "Notmuch Test Suite ",
+"Cc": "",
+"Bcc": "",
+"Date": "Sat,
+01 Jan 2000 12:00:00 -"},
+"body": [{"id": 1,
+"content-type": "text/plain",
+"content": "This is just a test message at /home/wsh/src/notmuch/test/trash 
directory.t0006-maildir/mail/msg-001:2,\n"}]},
+[' > show-expected
+
+test_expect_success 'Renamed message can be shown without running notmuch new' 
'
+notmuch show --format=json id:msg-001 at notmuch-test-suite | filter_show > 
show-actual &&
+test_cmp show-expected show-actual
+'
+
+test_expect_success 'Test that we can reply to the renamed message' '
+notmuch reply id:msg-001 at notmuch-test-suite
+'
+
+echo "No new mail." > expected
+test_expect_success 'No rename should be detected by notmuch new' '
+increment_mtime "$(dirname "${gen_msg_filename}")" &&
+notmuch new > actual &&
+test_cmp expected actual
+'
+test_done
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5417fe7..917631b 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -257,8 +257,11 @@ generate_message ()
 local additional_headers

 gen_msg_cnt=$((gen_msg_cnt + 1))
-gen_msg_name=msg-$(printf "%03d" $gen_msg_cnt)
-gen_msg_id="${gen_msg_name}@notmuch-test-suite"
+if [ -z "${template[filename]}" ]; then
+   template[filename]="msg-$(printf "%03d" $gen_msg_cnt)"
+fi
+gen_msg_name=${template[filename]}
+gen_msg_id="${gen_msg_name%:2,*}@notmuch-test-suite"

 if [ -z "${template[dir]}" ]; then
gen_msg_filename="${MAIL_DIR}/$gen_msg_name"
-- 
1.7.0



[notmuch] [PATCH 3/4] Add maildir-based mailstore

2010-03-18 Thread Michal Sojka
This mailstore allows bi-directional synchronization between maildir
flags and certain tags. The flag-to-tag mapping is defined by flag2tag
array.

The synchronization works this way:

1) Whenever notmuch new is executed, then for every new/renamed
   message the tags defined in flag2tag are either added or removed
   depending on the flags from the file name.

2) Whenever notmuch tag is executed, a new set of flags based on the
   tags is constructed and if the new flags differ from that stored in
   the file name, the file is renamed and notmuch database is updated
   to contain the new name for the file.

This mailstore is enabled by putting
[mailstore]
type=maildir
to your .notmuch-config.

Signed-off-by: Michal Sojka 
---
 lib/database.cc   |7 ++
 lib/mailstore-files.c |  167 -
 lib/mailstore.c   |1 +
 lib/message.cc|   41 -
 lib/notmuch-private.h |4 +
 lib/notmuch.h |1 +
 6 files changed, 217 insertions(+), 4 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 93c8d0f..33ef889 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1471,6 +1471,13 @@ notmuch_database_add_message (notmuch_database_t 
*notmuch,

_notmuch_message_add_filename (message, filename);

+   /* This is a new message or it has a new filename and as such,
+* its tags in database either do not exists or might be out
+* of date. Mailstore assigns the tags later in index_new(),
+* but until then we should not synchronize the tags back to
+* the mailstore. */
+   notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, 
TRUE);
+
/* Is this a newly created message object? */
if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
_notmuch_message_add_term (message, "type", "mail");
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index ace2664..d89e183 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -24,6 +24,9 @@
 #include "notmuch.h"
 #include "mailstore-private.h"
 #include 
+#include 
+
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))

 typedef struct _filename_node {
 char *filename;
@@ -69,8 +72,9 @@ _filename_list_add (_filename_list_t *list,
 }

 static void
-tag_inbox_and_unread (notmuch_message_t *message)
+tag_inbox_and_unread (notmuch_message_t *message, const char *filename)
 {
+(void)filename;
 notmuch_message_add_tag (message, "inbox");
 notmuch_message_add_tag (message, "unread");
 }
@@ -117,6 +121,141 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
 }


+struct mailstore_priv {
+void (*tag_new)(notmuch_message_t *message, const char *filename);
+void (*tag_renamed)(notmuch_message_t *message, const char *filename);
+};
+
+struct maildir_flag_tag {
+char flag;
+const char *tag;
+bool inverse;
+};
+
+/* ASCII ordered table of Maildir flags and assiciated tags */
+struct maildir_flag_tag flag2tag[] = {
+{ 'D', "draft",   false},
+{ 'F', "flagged", false},
+{ 'P', "passed",  false},
+{ 'R', "replied", false},
+{ 'S', "unread",  true },
+{ 'T', "delete",  false},
+};
+
+static void
+tag_from_maildir_flags(notmuch_message_t *message, const char *filename)
+{
+const char *flags, *p;
+char f;
+bool valid, unread;
+unsigned i;
+
+flags = strstr(filename, ":2,");
+if (!flags)
+   return;
+flags += 3;
+
+/*  Check that the letters are valid Maildir flags */
+f = 0;
+valid = true;
+for (p=flags; valid && *p; p++) {
+   switch (*p) {
+   case 'P':
+   case 'R':
+   case 'S':
+   case 'T':
+   case 'D':
+   case 'F':
+   if (*p > f) f=*p;
+   else valid = false;
+   break;
+   default:
+   valid = false;
+   }
+}
+if (!valid) {
+   /* fprintf(stderr, "Note: Invalid maildir flags: %s\n", 
message->filename); */
+   return;
+}
+
+notmuch_message_freeze(message);
+unread = true;
+for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+   if ((strchr(flags, flag2tag[i].flag) != NULL) ^ flag2tag[i].inverse) {
+   notmuch_message_add_tag (message, flag2tag[i].tag);
+   } else {
+   notmuch_message_remove_tag (message, flag2tag[i].tag);
+   }
+}
+notmuch_message_thaw(message);
+
+/* From now on, we can synchronize the tags from the database to
+ * the mailstore. */
+notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, 
FALSE);
+}
+
+/* Store maildir-related tags as maildir flags */
+static notmuch_private_status_t
+maildir_sync_tags(notmuch_mailstore_t *mailstore,
+ notmuch_message_t *message)
+{
+notmuch_tags_t *tags;
+const char *tag;
+char flags[ARRAY_SIZE(flag2tag)+1];
+unsigned i;
+char *p;
+const char *filename;
+char *filename_new;
+
+(void)mailstore;
+for

[notmuch] [PATCH 2/4] Convert mailstore abstraction

2010-03-18 Thread Michal Sojka
The code for detection of new files in the mailstore and their
addition to the database is moved from notmuch-new.c to
lib/mailstore-files.c, where it is called by the abstract mailstore
interface.

The code was changed to allow the progress reporting function to be
implemented outside of notmuch library.

Signed-off-by: Michal Sojka 
---
 lib/mailstore-files.c |  590 +++
 notmuch-new.c |  611 ++---
 2 files changed, 615 insertions(+), 586 deletions(-)

diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index 92d7f5d..ace2664 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -20,9 +20,596 @@
  * Michal Sojka 
  */

+#define _GNU_SOURCE/* For asprintf() */
 #include "notmuch.h"
 #include "mailstore-private.h"
+#include 

+typedef struct _filename_node {
+char *filename;
+struct _filename_node *next;
+} _filename_node_t;
+
+typedef struct _filename_list {
+_filename_node_t *head;
+_filename_node_t **tail;
+} _filename_list_t;
+
+typedef struct _indexing_context_priv {
+_filename_list_t *removed_files;
+_filename_list_t *removed_directories;
+} _indexing_context_priv_t;
+
+static _filename_list_t *
+_filename_list_create (const void *ctx)
+{
+_filename_list_t *list;
+
+list = talloc (ctx, _filename_list_t);
+if (list == NULL)
+   return NULL;
+
+list->head = NULL;
+list->tail = &list->head;
+
+return list;
+}
+
+static void
+_filename_list_add (_filename_list_t *list,
+   const char *filename)
+{
+_filename_node_t *node = talloc (list, _filename_node_t);
+
+node->filename = talloc_strdup (list, filename);
+node->next = NULL;
+
+*(list->tail) = node;
+list->tail = &node->next;
+}
+
+static void
+tag_inbox_and_unread (notmuch_message_t *message)
+{
+notmuch_message_add_tag (message, "inbox");
+notmuch_message_add_tag (message, "unread");
+}
+
+static int
+dirent_sort_inode (const struct dirent **a, const struct dirent **b)
+{
+return ((*a)->d_ino < (*b)->d_ino) ? -1 : 1;
+}
+
+static int
+dirent_sort_strcmp_name (const struct dirent **a, const struct dirent **b)
+{
+return strcmp ((*a)->d_name, (*b)->d_name);
+}
+
+/* Test if the directory looks like a Maildir directory.
+ *
+ * Search through the array of directory entries to see if we can find all
+ * three subdirectories typical for Maildir, that is "new", "cur", and "tmp".
+ *
+ * Return 1 if the directory looks like a Maildir and 0 otherwise.
+ */
+static int
+_entries_resemble_maildir (struct dirent **entries, int count)
+{
+int i, found = 0;
+
+for (i = 0; i < count; i++) {
+   if (entries[i]->d_type != DT_DIR && entries[i]->d_type != DT_UNKNOWN)
+   continue;
+
+   if (strcmp(entries[i]->d_name, "new") == 0 ||
+   strcmp(entries[i]->d_name, "cur") == 0 ||
+   strcmp(entries[i]->d_name, "tmp") == 0)
+   {
+   found++;
+   if (found == 3)
+   return 1;
+   }
+}
+
+return 0;
+}
+
+
+/* Examine 'path' recursively as follows:
+ *
+ *   o Ask the filesystem for the mtime of 'path' (fs_mtime)
+ *   o Ask the database for its timestamp of 'path' (db_mtime)
+ *
+ *   o Ask the filesystem for files and directories within 'path'
+ * (via scandir and stored in fs_entries)
+ *   o Ask the database for files and directories within 'path'
+ * (db_files and db_subdirs)
+ *
+ *   o Pass 1: For each directory in fs_entries, recursively call into
+ * this same function.
+ *
+ *   o Pass 2: If 'fs_mtime' > 'db_mtime', then walk fs_entries
+ * simultaneously with db_files and db_subdirs. Look for one of
+ * three interesting cases:
+ *
+ *1. Regular file in fs_entries and not in db_files
+ *This is a new file to add_message into the database.
+ *
+ * 2. Filename in db_files not in fs_entries.
+ *This is a file that has been removed from the mail store.
+ *
+ * 3. Directory in db_subdirs not in fs_entries
+ *This is a directory that has been removed from the mail store.
+ *
+ * Note that the addition of a directory is not interesting here,
+ * since that will have been taken care of in pass 1. Also, we
+ * don't immediately act on file/directory removal since we must
+ * ensure that in the case of a rename that the new filename is
+ * added before the old filename is removed, (so that no
+ * information is lost from the database).
+ *
+ *   o Tell the database to update its time of 'path' to 'fs_mtime'
+ */
+static notmuch_status_t
+add_files_recursive (notmuch_mailstore_t *mailstore,
+const char *path,
+notmuch_indexing_context_t *state)
+{
+DIR *dir = NULL;
+struct dirent *entry = NULL;
+char *next = NULL;
+time_t fs_mtime, db_mtime;
+notmuch_status_t status, ret = NOTMUCH_STATUS_SUCCESS;
+

[notmuch] [PATCH 1/4] Mailstore abstraction interface

2010-03-18 Thread Michal Sojka
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. This
patchseries implements two types of mailstore - plain files and
maildir. It is expected that additional git-based mailstore will
follow.

This patch contains only the interface changes. No functionality is
added, removed or changed.

A new configuration group [mailstore] is defined. Currently, there is
only one key 'type' whose value determines the used mailstore. The
default value of this option is the plain-file mailstore currently
implemented in notmuch.

Signed-off-by: Michal Sojka 
---
 lib/Makefile.local  |2 +
 lib/database-private.h  |1 +
 lib/database.cc |   15 ++--
 lib/mailstore-files.c   |   43 
 lib/mailstore-private.h |   59 
 lib/mailstore.c |   77 ++
 lib/message.cc  |   13 +++
 lib/notmuch.h   |   85 --
 notmuch-client.h|7 
 notmuch-config.c|   34 +++
 notmuch-count.c |3 +-
 notmuch-dump.c  |3 +-
 notmuch-new.c   |6 ++-
 notmuch-reply.c |3 +-
 notmuch-restore.c   |3 +-
 notmuch-search-tags.c   |3 +-
 notmuch-search.c|3 +-
 notmuch-show.c  |3 +-
 notmuch-tag.c   |3 +-
 19 files changed, 348 insertions(+), 18 deletions(-)
 create mode 100644 lib/mailstore-files.c
 create mode 100644 lib/mailstore-private.h
 create mode 100644 lib/mailstore.c

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 495b27e..fc8883d 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -3,6 +3,8 @@ extra_cflags += -I$(dir)

 libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+   $(dir)/mailstore.c  \
+   $(dir)/mailstore-files.c\
$(dir)/message-file.c   \
$(dir)/messages.c   \
$(dir)/sha1.c   \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;

+notmuch_mailstore_t *mailstore;
 };

 /* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
  */

 #include "database-private.h"
+#include "mailstore-private.h"

 #include 

@@ -438,7 +439,7 @@ parse_references (void *ctx,
 }

 notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
 }

 notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
 notmuch_database_upgrade (notmuch, NULL, NULL);

   DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t 
*notmuch)

 notmuch_database_t *
 notmuch_database_open (const char *path,
-  notmuch_database_mode_t mode)
+  notmuch_database_mode_t mode,
+  notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = &PROBABILISTIC_PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+   notmuch->mailstore = mailstore;
+   mailstore->notmuch = notmuch;
 } catch (const Xapian::Error &error) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
 error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,

   DONE:
 if (message) {
-   if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+   if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
new file mode 100644
index 000..92d7f5d
--- /dev/null
+++ b/lib/mailstore-files.c
@@ -0,0 +1,43 @@
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
+ *
+ * Copyright ?? 2009 Carl Worth
+ *
+ * This program is free software: you can redistri

[notmuch] Mailstore abstraction & maildir synchronization

2010-03-18 Thread Michal Sojka
Hi all,

I've finally found some time to implement the mailstore abstraction
was initially described in id:87ljecmnbd.fsf at steelpick.localdomain and
id:87eijqlz54.fsf at steelpick.localdomain.

The idea is to allow notmuch operate on different types of mail
storage (e.g. mail in git repositories) and to store the tags in the
storage together with mails. The aim is the ability to synchronize
mails and tags between computers. The following patch series is the
first version which I'm able to use on daily basis so I'd like to get
some feedback.

In the current form, the patch series implements two mail stores:
1. plain files (compatible with the current notmuch)
2. maildir (synchronizes certain tags with maildir flags)

The series passes the test suite. For the maildir store, there are
additional tests, but you need need the modularized testsuite (taken
from git). The whole patch series is also available from
git://rtime.felk.cvut.cz/notmuch.git mailstore-abstraction-v1 (this
branch wont be rebased).

Known bugs and limitations:

- Only file-based storage is suported. Notmuch access the files
  directly, and not via the mailstore interface.

- (maildir) Viewing/storing of attachments of unread messages doesn't
  work. The reason is that when you view the message it its unread tag
  is removed which leads to rename of the file, but Emacs still uses
  the original name to access the attachment.

  Workaround: close the message and open it again.

Maildir howto:

1. Backup you emails
2. Apply the patches (at least 1-3)
3. Configure notmuch to use maildir store
cat > ~/.notmuch-config <

Re: [notmuch] Search body?

2010-03-18 Thread Sandra Snan
I  wrote:
> is there a way to search just the body?

To answer my own question (in case others have the same question)—no,
not currently. I was grepping the sources but it didn’t occur to me to
look in lib/.
database.cc had the answer.

Sandra
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Tag search peculiarities

2010-03-18 Thread Sandra Snan
Hey, Ben.
Ben Gamari  wrote:
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new

Is there a new line between the calls? Like:
notmuch tag -new tag:new and tag:list # removes new from list
notmuch tag -new +inbox tag:new # replaces those that still are new with inbox

> However, I found that mailing list traffic was still getting through.

What do you mean by ?through?? Do you mean that there still are
messages tagged both inbox and list? Even though you can?t search for
them?

If a message is tagged both inbox and list, but not new then the two
lines of your script that you posted so far wouldn?t change it, and it
would still bo both inbox and list.

It?s risky being so dependent on the new tag.

> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of 
> tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of?

Yes, many versions of the reading client (for example, the version of
notmuch.el that ships with the notmuch package in debian) remove the
inbox tag from messages once you scroll past them or when you press
the key that?s bound to notmuch-show-archive-thread (default is ?a?).

> Is my index just FUBAR? (the ladder would be very strange
> considering it's only a few days old and I can't think of any
> crashes, etc. that might have corrupted it) Any ideas for debugging?

Add the tags manually to a few messages, search for them again, and be
sure to not run your tagging script while looking at this problem.

Michal?s tip, looking at a few lines of notmuch dump, is a good idea
too. Notmuch dump is fast, so don?t be afraid (notmuch restore on the
other hand? but even that completes fast enough.)

I?ve often been surprised at weird tag situations then realized that
it was old versions of my own scripts that had ran in the background
and I?ve forgotten about it.

Boolean algebra and set theory is tricky stuff! Good luck.

Sandra


[notmuch] Tag search peculiarities

2010-03-18 Thread Michal Sojka
On Wed, 17 Mar 2010, Ben Gamari wrote:
> While trying perfect my initial tagging script, I have run into a very strange
> set of issue. In my script, I use the following to exclude mailing list 
> traffic
> from my inbox,
> 
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new
> 
> However, I found that mailing list traffic was still getting through.
> 
> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of 
> tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of? Is my index just FUBAR? (the 
> ladder
> would be very strange considering it's only a few days old and I can't think 
> of
> any crashes, etc. that might have corrupted it) Any ideas for debugging?

Hi Ben,

I've never had similar problems. AFAIK there is nothing special on inbox
tag. It seems you use your configurable tags patch so I suspected that
patch, but it seems OK.

I'd try the following:
notmuch dump|grep "inbox.*${TAG}"
notmuch dump|grep "${TAG}.*inbox"

It if outputs something, then it's really strange.

-Michal


[notmuch] [PATCH] To use compose-mail and mail-citation-hook

2010-03-18 Thread Sandra Snan
Michal Sojka  wrote:
> it seems that many peaple are not happy with the current notmuch reply.
> It would be probably better to ignore notmuch reply completely and
> implement replying only in elisp.

I?ll give it another look.

> > I?ll send another e-mail with the patch for just compose-mail without
> Next time please word-wrap the commit message.

Thanks for the heads up; I didn?t realize that the part that I wrote
in the mail also was used as the commit message.
I?m always welcome to more feedback on how I format and send my
patches because that?s a part of development that?s new to me.

Sandra


[notmuch] [PATCH] To use compose-mail and mail-citation-hook

2010-03-18 Thread Michal Sojka
On Tue, 16 Mar 2010, Sandra Snan wrote:
> Emacs has an interface called compose-mail which uses whatever mailing
> mode that you?ve selected in mail-user-agent so if you like the
> message mode that?s been hard-coded into notmuch.el, (setq
> mail-user-agent 'message-user-agent) and this will use that.
>
> This version of the patch also tries to yank the body text, calling
> mail-citation-hook as it does so it works with mu-cite, supercite,
> trivialcite and so on. I guess I started yak-shaving a bit too much
> because I kinda began to reconstruct the output of notmuch reply via
> emacs lisp.

Hi Sandra,

it seems that many peaple are not happy with the current notmuch reply.
It would be probably better to ignore notmuch reply completely and
implement replying only in elisp.

> I?ll send another e-mail with the patch for just compose-mail without
> the mail-citation-hook, too. These patches are mutually exclusive. I?m
> not used to git so I hope this is all right.

Next time please word-wrap the commit message.

Thanks
Michal


[notmuch] Search body?

2010-03-18 Thread Sandra Snan
I’m glad that the default search is free text including headers and
body, and that there are ways to search just the subject, or just the
sender and so on, but is there a way to search just the body? I
currently use a lot of “sandra and not from:sandra” and so on.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH] Add count command to manual page

2010-03-18 Thread Sandra Snan
Just found out that I’ve been piping things through wc for no reason.
Heh. Here’s a quick patch, if you want to add this to the manual page.

Signed-off-by: Sandra Snan 
---
 notmuch.1 |   18 +-
 1 files changed, 17 insertions(+), 1 deletions(-)

diff --git a/notmuch.1 b/notmuch.1
index e573749..4d300c0 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -126,7 +126,7 @@ syntax. See the
 section below for more details on the supported syntax.
 
 The
-.BR search " and "show
+.BR "search" ", " "show" " and " "count"
 commands are used to query the email database.
 .RS 4
 .TP 4
@@ -239,6 +239,22 @@ See the
 .B "SEARCH SYNTAX"
 section below for details of the supported syntax for .
 .RE
+.TP
+.BR count " ..."
+
+Count messages matching the search terms.
+
+The number of matching messages is output to stdout.
+
+A common use of
+.B notmuch count
+is to display the count of messages matching both a specific tag and
+either inbox or unread
+
+See the
+.B "SEARCH SYNTAX"
+section below for details of the supported syntax for .
+.RE
 .RE
 
 The
-- 
1.7.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Yet another python binding

2010-03-18 Thread Sebastian Spaeth
I've been bragging on IRC over the last couple of days already, so some
might already be annoyed by this, but here it goes:

I've been binding the notmuch shared library to python and so far it
works really nice. It "only" requires that you have a libnotmuch.so or
libnotmuch.so.1 installed in some library path and a python>=2.5.

API docs of bound methods: http://spaetz.bitbucket.org/docs/html/index.html
Source code: http://bitbucket.org/spaetz/cnotmuch/

-
I've reimplemented a python-based notmuch "binary" (to be able to run the
test suite on it), that implements the following commands:
(The performance is comparable to that of the real notmuch)

- show: outputs message stubs, formatting is still missing; no
  --entire-threads option
- count, tag, dump, search-tags

Missing is still:
 setup, reply
 new (this is  still hard)
 search (still missing the notmuch_threads* bindings)
 restore (trivial, but not done yet)
-

Get it via "hg clone https://spaetz at bitbucket.org/spaetz/cnotmuch/".
(Switch to the branch with the current static documentation with "hg up
docs" and back to the code with "hg up default".)

If you are interested in using notmuch from python, give it a look. I
plan to convert notmuchsync to this soon and hope to get a nice boost
from it.

spaetz


[notmuch] [PATCH 2/4] Convert mailstore abstraction

2010-03-18 Thread Michal Sojka
The code for detection of new files in the mailstore and their
addition to the database is moved from notmuch-new.c to
lib/mailstore-files.c, where it is called by the abstract mailstore
interface.

The code was changed to allow the progress reporting function to be
implemented outside of notmuch library.

Signed-off-by: Michal Sojka 
---
 lib/mailstore-files.c |  590 +++
 notmuch-new.c |  611 ++---
 2 files changed, 615 insertions(+), 586 deletions(-)

diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index 92d7f5d..ace2664 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -20,9 +20,596 @@
  * Michal Sojka 
  */
 
+#define _GNU_SOURCE/* For asprintf() */
 #include "notmuch.h"
 #include "mailstore-private.h"
+#include 
 
+typedef struct _filename_node {
+char *filename;
+struct _filename_node *next;
+} _filename_node_t;
+
+typedef struct _filename_list {
+_filename_node_t *head;
+_filename_node_t **tail;
+} _filename_list_t;
+
+typedef struct _indexing_context_priv {
+_filename_list_t *removed_files;
+_filename_list_t *removed_directories;
+} _indexing_context_priv_t;
+
+static _filename_list_t *
+_filename_list_create (const void *ctx)
+{
+_filename_list_t *list;
+
+list = talloc (ctx, _filename_list_t);
+if (list == NULL)
+   return NULL;
+
+list->head = NULL;
+list->tail = &list->head;
+
+return list;
+}
+
+static void
+_filename_list_add (_filename_list_t *list,
+   const char *filename)
+{
+_filename_node_t *node = talloc (list, _filename_node_t);
+
+node->filename = talloc_strdup (list, filename);
+node->next = NULL;
+
+*(list->tail) = node;
+list->tail = &node->next;
+}
+
+static void
+tag_inbox_and_unread (notmuch_message_t *message)
+{
+notmuch_message_add_tag (message, "inbox");
+notmuch_message_add_tag (message, "unread");
+}
+
+static int
+dirent_sort_inode (const struct dirent **a, const struct dirent **b)
+{
+return ((*a)->d_ino < (*b)->d_ino) ? -1 : 1;
+}
+
+static int
+dirent_sort_strcmp_name (const struct dirent **a, const struct dirent **b)
+{
+return strcmp ((*a)->d_name, (*b)->d_name);
+}
+
+/* Test if the directory looks like a Maildir directory.
+ *
+ * Search through the array of directory entries to see if we can find all
+ * three subdirectories typical for Maildir, that is "new", "cur", and "tmp".
+ *
+ * Return 1 if the directory looks like a Maildir and 0 otherwise.
+ */
+static int
+_entries_resemble_maildir (struct dirent **entries, int count)
+{
+int i, found = 0;
+
+for (i = 0; i < count; i++) {
+   if (entries[i]->d_type != DT_DIR && entries[i]->d_type != DT_UNKNOWN)
+   continue;
+
+   if (strcmp(entries[i]->d_name, "new") == 0 ||
+   strcmp(entries[i]->d_name, "cur") == 0 ||
+   strcmp(entries[i]->d_name, "tmp") == 0)
+   {
+   found++;
+   if (found == 3)
+   return 1;
+   }
+}
+
+return 0;
+}
+
+
+/* Examine 'path' recursively as follows:
+ *
+ *   o Ask the filesystem for the mtime of 'path' (fs_mtime)
+ *   o Ask the database for its timestamp of 'path' (db_mtime)
+ *
+ *   o Ask the filesystem for files and directories within 'path'
+ * (via scandir and stored in fs_entries)
+ *   o Ask the database for files and directories within 'path'
+ * (db_files and db_subdirs)
+ *
+ *   o Pass 1: For each directory in fs_entries, recursively call into
+ * this same function.
+ *
+ *   o Pass 2: If 'fs_mtime' > 'db_mtime', then walk fs_entries
+ * simultaneously with db_files and db_subdirs. Look for one of
+ * three interesting cases:
+ *
+ *1. Regular file in fs_entries and not in db_files
+ *This is a new file to add_message into the database.
+ *
+ * 2. Filename in db_files not in fs_entries.
+ *This is a file that has been removed from the mail store.
+ *
+ * 3. Directory in db_subdirs not in fs_entries
+ *This is a directory that has been removed from the mail store.
+ *
+ * Note that the addition of a directory is not interesting here,
+ * since that will have been taken care of in pass 1. Also, we
+ * don't immediately act on file/directory removal since we must
+ * ensure that in the case of a rename that the new filename is
+ * added before the old filename is removed, (so that no
+ * information is lost from the database).
+ *
+ *   o Tell the database to update its time of 'path' to 'fs_mtime'
+ */
+static notmuch_status_t
+add_files_recursive (notmuch_mailstore_t *mailstore,
+const char *path,
+notmuch_indexing_context_t *state)
+{
+DIR *dir = NULL;
+struct dirent *entry = NULL;
+char *next = NULL;
+time_t fs_mtime, db_mtime;
+notmuch_status_t status, ret = NOTMUCH_STATUS_SUCCESS;

[notmuch] [PATCH 4/4] Tests for maildir-based mailstore

2010-03-18 Thread Michal Sojka
Signed-off-by: Michal Sojka 
---
 test/t0006-maildir.sh |  113 +
 test/test-lib.sh  |7 ++-
 2 files changed, 118 insertions(+), 2 deletions(-)
 create mode 100755 test/t0006-maildir.sh

diff --git a/test/t0006-maildir.sh b/test/t0006-maildir.sh
new file mode 100755
index 000..e584908
--- /dev/null
+++ b/test/t0006-maildir.sh
@@ -0,0 +1,113 @@
+#!/bin/bash
+
+test_description="Test maildir mailstore"
+
+. ./test-lib.sh
+
+filter_output() {
+grep -v -E -e "$NOTMUCH_IGNORED_OUTPUT_REGEXP" | sed -e 
"$NOTMUCH_THREAD_ID_SQUELCH"
+}
+
+filter_show() {
+sed -e 's/, /,\n/g'|sed -e '/^"filename"/ s/:2,[A-Z]*//' -e '/^"tags"/d'
+}
+
+cat >> "$NOTMUCH_CONFIG" < expected < actual &&
+test_cmp expected actual
+#emacs --eval "(gdb \"gdb --annotate=3 --args $(which notmuch) new\")"
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+cat > expected < actual &&
+test_cmp expected actual
+'
+test_expect_success 'Tag the seen messages as replied' '
+notmuch tag +replied -inbox tag:inbox and not tag:unread
+'
+
+cat > expected < actual &&
+test_cmp expected actual
+'
+echo -n '[[[{"id": "msg-...@notmuch-test-suite",
+"match": true,
+"filename": "/home/wsh/src/notmuch/test/trash 
directory.t0006-maildir/mail/msg-001",
+"headers": {"Subject": "test message",
+"From": "Notmuch Test Suite ",
+"To": "Notmuch Test Suite ",
+"Cc": "",
+"Bcc": "",
+"Date": "Sat,
+01 Jan 2000 12:00:00 -"},
+"body": [{"id": 1,
+"content-type": "text/plain",
+"content": "This is just a test message at /home/wsh/src/notmuch/test/trash 
directory.t0006-maildir/mail/msg-001:2,\n"}]},
+[' > show-expected
+
+test_expect_success 'Renamed message can be shown without running notmuch new' 
'
+notmuch show --format=json id:msg-...@notmuch-test-suite | filter_show > 
show-actual &&
+test_cmp show-expected show-actual
+'
+
+test_expect_success 'Test that we can reply to the renamed message' '
+notmuch reply id:msg-...@notmuch-test-suite
+'
+
+echo "No new mail." > expected
+test_expect_success 'No rename should be detected by notmuch new' '
+increment_mtime "$(dirname "${gen_msg_filename}")" &&
+notmuch new > actual &&
+test_cmp expected actual
+'
+test_done
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5417fe7..917631b 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -257,8 +257,11 @@ generate_message ()
 local additional_headers
 
 gen_msg_cnt=$((gen_msg_cnt + 1))
-gen_msg_name=msg-$(printf "%03d" $gen_msg_cnt)
-gen_msg_id="${gen_msg_na...@notmuch-test-suite"
+if [ -z "${template[filename]}" ]; then
+   template[filename]="msg-$(printf "%03d" $gen_msg_cnt)"
+fi
+gen_msg_name=${template[filename]}
+gen_msg_id="${gen_msg_name%:2,*...@notmuch-test-suite"
 
 if [ -z "${template[dir]}" ]; then
gen_msg_filename="${MAIL_DIR}/$gen_msg_name"
-- 
1.7.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH 1/4] Mailstore abstraction interface

2010-03-18 Thread Michal Sojka
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. This
patchseries implements two types of mailstore - plain files and
maildir. It is expected that additional git-based mailstore will
follow.

This patch contains only the interface changes. No functionality is
added, removed or changed.

A new configuration group [mailstore] is defined. Currently, there is
only one key 'type' whose value determines the used mailstore. The
default value of this option is the plain-file mailstore currently
implemented in notmuch.

Signed-off-by: Michal Sojka 
---
 lib/Makefile.local  |2 +
 lib/database-private.h  |1 +
 lib/database.cc |   15 ++--
 lib/mailstore-files.c   |   43 
 lib/mailstore-private.h |   59 
 lib/mailstore.c |   77 ++
 lib/message.cc  |   13 +++
 lib/notmuch.h   |   85 --
 notmuch-client.h|7 
 notmuch-config.c|   34 +++
 notmuch-count.c |3 +-
 notmuch-dump.c  |3 +-
 notmuch-new.c   |6 ++-
 notmuch-reply.c |3 +-
 notmuch-restore.c   |3 +-
 notmuch-search-tags.c   |3 +-
 notmuch-search.c|3 +-
 notmuch-show.c  |3 +-
 notmuch-tag.c   |3 +-
 19 files changed, 348 insertions(+), 18 deletions(-)
 create mode 100644 lib/mailstore-files.c
 create mode 100644 lib/mailstore-private.h
 create mode 100644 lib/mailstore.c

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 495b27e..fc8883d 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -3,6 +3,8 @@ extra_cflags += -I$(dir)
 
 libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+   $(dir)/mailstore.c  \
+   $(dir)/mailstore-files.c\
$(dir)/message-file.c   \
$(dir)/messages.c   \
$(dir)/sha1.c   \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;
 
+notmuch_mailstore_t *mailstore;
 };
 
 /* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
  */
 
 #include "database-private.h"
+#include "mailstore-private.h"
 
 #include 
 
@@ -438,7 +439,7 @@ parse_references (void *ctx,
 }
 
 notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
 }
 
 notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
 notmuch_database_upgrade (notmuch, NULL, NULL);
 
   DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t 
*notmuch)
 
 notmuch_database_t *
 notmuch_database_open (const char *path,
-  notmuch_database_mode_t mode)
+  notmuch_database_mode_t mode,
+  notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = &PROBABILISTIC_PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+   notmuch->mailstore = mailstore;
+   mailstore->notmuch = notmuch;
 } catch (const Xapian::Error &error) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
 error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
 
   DONE:
 if (message) {
-   if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+   if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
new file mode 100644
index 000..92d7f5d
--- /dev/null
+++ b/lib/mailstore-files.c
@@ -0,0 +1,43 @@
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
+ *
+ * Copyright © 2009 Carl Worth
+ *
+ * This program is free software: you c

[notmuch] [PATCH 3/4] Add maildir-based mailstore

2010-03-18 Thread Michal Sojka
This mailstore allows bi-directional synchronization between maildir
flags and certain tags. The flag-to-tag mapping is defined by flag2tag
array.

The synchronization works this way:

1) Whenever notmuch new is executed, then for every new/renamed
   message the tags defined in flag2tag are either added or removed
   depending on the flags from the file name.

2) Whenever notmuch tag is executed, a new set of flags based on the
   tags is constructed and if the new flags differ from that stored in
   the file name, the file is renamed and notmuch database is updated
   to contain the new name for the file.

This mailstore is enabled by putting
[mailstore]
type=maildir
to your .notmuch-config.

Signed-off-by: Michal Sojka 
---
 lib/database.cc   |7 ++
 lib/mailstore-files.c |  167 -
 lib/mailstore.c   |1 +
 lib/message.cc|   41 -
 lib/notmuch-private.h |4 +
 lib/notmuch.h |1 +
 6 files changed, 217 insertions(+), 4 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 93c8d0f..33ef889 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1471,6 +1471,13 @@ notmuch_database_add_message (notmuch_database_t 
*notmuch,
 
_notmuch_message_add_filename (message, filename);
 
+   /* This is a new message or it has a new filename and as such,
+* its tags in database either do not exists or might be out
+* of date. Mailstore assigns the tags later in index_new(),
+* but until then we should not synchronize the tags back to
+* the mailstore. */
+   notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, 
TRUE);
+
/* Is this a newly created message object? */
if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
_notmuch_message_add_term (message, "type", "mail");
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index ace2664..d89e183 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -24,6 +24,9 @@
 #include "notmuch.h"
 #include "mailstore-private.h"
 #include 
+#include 
+
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
 
 typedef struct _filename_node {
 char *filename;
@@ -69,8 +72,9 @@ _filename_list_add (_filename_list_t *list,
 }
 
 static void
-tag_inbox_and_unread (notmuch_message_t *message)
+tag_inbox_and_unread (notmuch_message_t *message, const char *filename)
 {
+(void)filename;
 notmuch_message_add_tag (message, "inbox");
 notmuch_message_add_tag (message, "unread");
 }
@@ -117,6 +121,141 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
 }
 
 
+struct mailstore_priv {
+void (*tag_new)(notmuch_message_t *message, const char *filename);
+void (*tag_renamed)(notmuch_message_t *message, const char *filename);
+};
+
+struct maildir_flag_tag {
+char flag;
+const char *tag;
+bool inverse;
+};
+
+/* ASCII ordered table of Maildir flags and assiciated tags */
+struct maildir_flag_tag flag2tag[] = {
+{ 'D', "draft",   false},
+{ 'F', "flagged", false},
+{ 'P', "passed",  false},
+{ 'R', "replied", false},
+{ 'S', "unread",  true },
+{ 'T', "delete",  false},
+};
+
+static void
+tag_from_maildir_flags(notmuch_message_t *message, const char *filename)
+{
+const char *flags, *p;
+char f;
+bool valid, unread;
+unsigned i;
+
+flags = strstr(filename, ":2,");
+if (!flags)
+   return;
+flags += 3;
+
+/*  Check that the letters are valid Maildir flags */
+f = 0;
+valid = true;
+for (p=flags; valid && *p; p++) {
+   switch (*p) {
+   case 'P':
+   case 'R':
+   case 'S':
+   case 'T':
+   case 'D':
+   case 'F':
+   if (*p > f) f=*p;
+   else valid = false;
+   break;
+   default:
+   valid = false;
+   }
+}
+if (!valid) {
+   /* fprintf(stderr, "Note: Invalid maildir flags: %s\n", 
message->filename); */
+   return;
+}
+
+notmuch_message_freeze(message);
+unread = true;
+for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+   if ((strchr(flags, flag2tag[i].flag) != NULL) ^ flag2tag[i].inverse) {
+   notmuch_message_add_tag (message, flag2tag[i].tag);
+   } else {
+   notmuch_message_remove_tag (message, flag2tag[i].tag);
+   }
+}
+notmuch_message_thaw(message);
+
+/* From now on, we can synchronize the tags from the database to
+ * the mailstore. */
+notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, 
FALSE);
+}
+
+/* Store maildir-related tags as maildir flags */
+static notmuch_private_status_t
+maildir_sync_tags(notmuch_mailstore_t *mailstore,
+ notmuch_message_t *message)
+{
+notmuch_tags_t *tags;
+const char *tag;
+char flags[ARRAY_SIZE(flag2tag)+1];
+unsigned i;
+char *p;
+const char *filename;
+char *filename_new;
+
+(void)mailstore;
+ 

[notmuch] Mailstore abstraction & maildir synchronization

2010-03-18 Thread Michal Sojka
Hi all,

I've finally found some time to implement the mailstore abstraction
was initially described in id:87ljecmnbd@steelpick.localdomain and
id:87eijqlz54@steelpick.localdomain.

The idea is to allow notmuch operate on different types of mail
storage (e.g. mail in git repositories) and to store the tags in the
storage together with mails. The aim is the ability to synchronize
mails and tags between computers. The following patch series is the
first version which I'm able to use on daily basis so I'd like to get
some feedback.

In the current form, the patch series implements two mail stores:
1. plain files (compatible with the current notmuch)
2. maildir (synchronizes certain tags with maildir flags)

The series passes the test suite. For the maildir store, there are
additional tests, but you need need the modularized testsuite (taken
from git). The whole patch series is also available from
git://rtime.felk.cvut.cz/notmuch.git mailstore-abstraction-v1 (this
branch wont be rebased).

Known bugs and limitations:

- Only file-based storage is suported. Notmuch access the files
  directly, and not via the mailstore interface.

- (maildir) Viewing/storing of attachments of unread messages doesn't
  work. The reason is that when you view the message it its unread tag
  is removed which leads to rename of the file, but Emacs still uses
  the original name to access the attachment.

  Workaround: close the message and open it again.

Maildir howto:

1. Backup you emails
2. Apply the patches (at least 1-3)
3. Configure notmuch to use maildir store
cat > ~/.notmuch-config 

Re: [notmuch] Yet another python binding

2010-03-18 Thread Ben Gamari
On Thu, 18 Mar 2010 09:59:01 +0100, "Sebastian Spaeth"  
wrote:
> I've been bragging on IRC over the last couple of days already, so some
> might already be annoyed by this, but here it goes:
> 
> I've been binding the notmuch shared library to python and so far it
> works really nice. It "only" requires that you have a libnotmuch.so or
> libnotmuch.so.1 installed in some library path and a python>=2.5.
> 
Looks awesome. I like this far better than my own SWIG bindings. 

- Ben
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Yet another python binding

2010-03-18 Thread Ben Gamari
On Thu, 18 Mar 2010 09:59:01 +0100, "Sebastian Spaeth"  wrote:
> I've been bragging on IRC over the last couple of days already, so some
> might already be annoyed by this, but here it goes:
> 
> I've been binding the notmuch shared library to python and so far it
> works really nice. It "only" requires that you have a libnotmuch.so or
> libnotmuch.so.1 installed in some library path and a python>=2.5.
> 
Looks awesome. I like this far better than my own SWIG bindings. 

- Ben


Re: [notmuch] Tag search peculiarities

2010-03-18 Thread Sandra Snan
Hey, Ben.
Ben Gamari  wrote:
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new

Is there a new line between the calls? Like:
notmuch tag -new tag:new and tag:list # removes new from list
notmuch tag -new +inbox tag:new # replaces those that still are new with inbox

> However, I found that mailing list traffic was still getting through.

What do you mean by “through”? Do you mean that there still are
messages tagged both inbox and list? Even though you can’t search for
them?

If a message is tagged both inbox and list, but not new then the two
lines of your script that you posted so far wouldn’t change it, and it
would still bo both inbox and list.

It’s risky being so dependent on the new tag.

> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of 
> tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of?

Yes, many versions of the reading client (for example, the version of
notmuch.el that ships with the notmuch package in debian) remove the
inbox tag from messages once you scroll past them or when you press
the key that’s bound to notmuch-show-archive-thread (default is “a”).

> Is my index just FUBAR? (the ladder would be very strange
> considering it's only a few days old and I can't think of any
> crashes, etc. that might have corrupted it) Any ideas for debugging?

Add the tags manually to a few messages, search for them again, and be
sure to not run your tagging script while looking at this problem.

Michal’s tip, looking at a few lines of notmuch dump, is a good idea
too. Notmuch dump is fast, so don’t be afraid (notmuch restore on the
other hand… but even that completes fast enough.)

I’ve often been surprised at weird tag situations then realized that
it was old versions of my own scripts that had ran in the background
and I’ve forgotten about it.

Boolean algebra and set theory is tricky stuff! Good luck.

Sandra
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Tag search peculiarities

2010-03-18 Thread Michal Sojka
On Wed, 17 Mar 2010, Ben Gamari wrote:
> While trying perfect my initial tagging script, I have run into a very strange
> set of issue. In my script, I use the following to exclude mailing list 
> traffic
> from my inbox,
> 
> notmuch tag -new tag:new and tag:list notmuch tag -new +inbox tag:new
> 
> However, I found that mailing list traffic was still getting through.
> 
> After investigating further, I found that any query in the form of "tag:inbox
> and tag:$TAG" would return no results. Strangely, all other combinations of 
> tag
> searches (i.e. "tag:lkml and tag:unread") seem to work just fine.
> 
> Has anyone else noticed this sort of behavior? Does the inbox tag have some
> special signifigance that I should know of? Is my index just FUBAR? (the 
> ladder
> would be very strange considering it's only a few days old and I can't think 
> of
> any crashes, etc. that might have corrupted it) Any ideas for debugging?

Hi Ben,

I've never had similar problems. AFAIK there is nothing special on inbox
tag. It seems you use your configurable tags patch so I suspected that
patch, but it seems OK.

I'd try the following:
notmuch dump|grep "inbox.*${TAG}"
notmuch dump|grep "${TAG}.*inbox"

It if outputs something, then it's really strange.

-Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] To use compose-mail and mail-citation-hook

2010-03-18 Thread Sandra Snan
Michal Sojka  wrote:
> it seems that many peaple are not happy with the current notmuch reply.
> It would be probably better to ignore notmuch reply completely and
> implement replying only in elisp.

I’ll give it another look.

> > I’ll send another e-mail with the patch for just compose-mail without
> Next time please word-wrap the commit message.

Thanks for the heads up; I didn’t realize that the part that I wrote
in the mail also was used as the commit message.
I’m always welcome to more feedback on how I format and send my
patches because that’s a part of development that’s new to me.

Sandra
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] To use compose-mail and mail-citation-hook

2010-03-18 Thread Michal Sojka
On Tue, 16 Mar 2010, Sandra Snan wrote:
> Emacs has an interface called compose-mail which uses whatever mailing
> mode that you’ve selected in mail-user-agent so if you like the
> message mode that’s been hard-coded into notmuch.el, (setq
> mail-user-agent 'message-user-agent) and this will use that.
>
> This version of the patch also tries to yank the body text, calling
> mail-citation-hook as it does so it works with mu-cite, supercite,
> trivialcite and so on. I guess I started yak-shaving a bit too much
> because I kinda began to reconstruct the output of notmuch reply via
> emacs lisp.

Hi Sandra,

it seems that many peaple are not happy with the current notmuch reply.
It would be probably better to ignore notmuch reply completely and
implement replying only in elisp.

> I’ll send another e-mail with the patch for just compose-mail without
> the mail-citation-hook, too. These patches are mutually exclusive. I’m
> not used to git so I hope this is all right.

Next time please word-wrap the commit message.

Thanks
Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Yet another python binding

2010-03-18 Thread Sebastian Spaeth
I've been bragging on IRC over the last couple of days already, so some
might already be annoyed by this, but here it goes:

I've been binding the notmuch shared library to python and so far it
works really nice. It "only" requires that you have a libnotmuch.so or
libnotmuch.so.1 installed in some library path and a python>=2.5.

API docs of bound methods: http://spaetz.bitbucket.org/docs/html/index.html
Source code: http://bitbucket.org/spaetz/cnotmuch/

-
I've reimplemented a python-based notmuch "binary" (to be able to run the
test suite on it), that implements the following commands:
(The performance is comparable to that of the real notmuch)

- show: outputs message stubs, formatting is still missing; no
  --entire-threads option
- count, tag, dump, search-tags

Missing is still:
 setup, reply
 new (this is  still hard)
 search (still missing the notmuch_threads* bindings)
 restore (trivial, but not done yet)
-

Get it via "hg clone https://spa...@bitbucket.org/spaetz/cnotmuch/";.
(Switch to the branch with the current static documentation with "hg up
docs" and back to the code with "hg up default".)

If you are interested in using notmuch from python, give it a look. I
plan to convert notmuchsync to this soon and hope to get a nice boost
from it.

spaetz
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch