Re: [PATCH] Test: fix draft test for emacs23

[David Bremner]

Mark Walters  writes:

> emacs24 and emacs23 have different secure tag defaults: in particular,
> mml-secure-message-sign only signs the part on emacs23 but the whole
> message on emacs24. This difference makes one of the draft tests fail
> (which causes a cascade of later failures) on emacs23. It seems that
> travis uses emacs23 so it is useful to fix this.
>
> We do this by forcing the whole message to be signed in either case --
> the code snippet is extracted from mml-secure-message-sign on emacs24.
> ---

Pushed to master

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[Patch v2] lib: regexp matching in 'subject' and 'from'

[David Bremner]

the idea is that you can run

% notmuch search re:subject:
% notmuch search re:from:'

or

% notmuch search subject:"your usual phrase search"
% notmuch search from:"usual phrase search"

This should also work with bindings, since it extends the query parser.

This is trivial to extend for other value slots, but currently the only
value slots are date, message_id, from, subject, and last_mod. Date is
already searchable, and message_id is not obviously useful to regex
match.

This was originally written by Austin Clements, and ported to Xapian
field processors (from Austin's custom query parser) by yours truly.
---

rebase of id:1467034387-16885-1-git-send-email-da...@tethera.net against master

 doc/man7/notmuch-search-terms.rst |  17 +-
 lib/Makefile.local|   1 +
 lib/database-private.h|   1 +
 lib/database.cc   |   5 ++
 lib/regexp-fields.cc  | 125 ++
 lib/regexp-fields.h   |  77 +++
 test/T630-regexp-query.sh |  91 +++
 7 files changed, 316 insertions(+), 1 deletion(-)
 create mode 100644 lib/regexp-fields.cc
 create mode 100644 lib/regexp-fields.h
 create mode 100755 test/T630-regexp-query.sh

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index de93d73..4c7afc2 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -60,6 +60,8 @@ indicate user-supplied values):
 
 -  property:=
 
+- re:{subject,from}:
+
 The **from:** prefix is used to match the name or address of the sender
 of an email message.
 
@@ -146,6 +148,12 @@ The **property:** prefix searches for messages with a 
particular
 (and extensions) to add metadata to messages. A given key can be
 present on a given message with several different values.
 
+The **re::** prefix can be used to restrict the results to
+those whose  matches the given regular expression (see
+**regex(7)**). Regular expression searches are only available if
+notmuch is built with **Xapian Field Processors** (see below), and
+currently only for the Subject and From fields.
+
 Operators
 -
 
@@ -220,13 +228,19 @@ Boolean and Probabilistic Prefixes
 --
 
 Xapian (and hence notmuch) prefixes are either **boolean**, supporting
-exact matches like "tag:inbox"  or **probabilistic**, supporting a more 
flexible **term** based searching. The prefixes currently supported by notmuch 
are as follows.
+exact matches like "tag:inbox" or **probabilistic**, supporting a more
+flexible **term** based searching. Certain **special** prefixes are
+processed by notmuch in a way not stricly fitting either of Xapian's
+built in styles. The prefixes currently supported by notmuch are as
+follows.
 
 
 Boolean
**tag:**, **id:**, **thread:**, **folder:**, **path:**, **property:**
 Probabilistic
**from:**, **to:**, **subject:**, **attachment:**, **mimetype:**
+Special
+   **query:**, **re:**
 
 Terms and phrases
 -
@@ -396,6 +410,7 @@ Currently the following features require field processor 
support:
 
 - non-range date queries, e.g. "date:today"
 - named queries e.g. "query:my_special_query"
+- regular expression searches, e.g. "re:subject:^\\[SPAM\\]"
 
 SEE ALSO
 
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 3d1030a..ccd32ab 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -53,6 +53,7 @@ libnotmuch_cxx_srcs = \
$(dir)/query.cc \
$(dir)/query-fp.cc  \
$(dir)/config.cc\
+   $(dir)/regexp-fields.cc \
$(dir)/thread.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database-private.h b/lib/database-private.h
index ca71a92..900a989 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -186,6 +186,7 @@ struct _notmuch_database {
 #if HAVE_XAPIAN_FIELD_PROCESSOR
 Xapian::FieldProcessor *date_field_processor;
 Xapian::FieldProcessor *query_field_processor;
+Xapian::FieldProcessor *re_field_processor;
 #endif
 Xapian::ValueRangeProcessor *last_mod_range_processor;
 };
diff --git a/lib/database.cc b/lib/database.cc
index 2d19f20..851a62d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -21,6 +21,7 @@
 #include "database-private.h"
 #include "parse-time-vrp.h"
 #include "query-fp.h"
+#include "regexp-fields.h"
 #include "string-util.h"
 
 #include 
@@ -1042,6 +1043,8 @@ notmuch_database_open_verbose (const char *path,
notmuch->query_parser->add_boolean_prefix("date", 
notmuch->date_field_processor);
notmuch->query_field_processor = new QueryFieldProcessor 
(*notmuch->query_parser, notmuch);
notmuch->query_parser->add_boolean_prefix("query", 
notmuch->query_field_processor);
+   notmuch->re_field_processor = new RegexpFieldProcessor 
(*notmuch->query_parser, notmuch);
+   

Re: [Paul Wise] Bug#843127: notmuch: race condition in `notmuch new`?

[David Bremner]

"J. Lewis Muir"  writes:

> On 11/12, Austin Clements wrote:
>> Quoth David Bremner on Nov 04 at  1:26 pm:
>> > I agree it looks like a race condition. inotify sounds a bit
>> > overcomplicated and perhaps non-portable? It should probably just
>> > tolerate disappearing files better, consider that a warning.
>> 
>> Inotify really *is* the solution.
>
> I don't see how inotify can be the solution unless the idea is to make
> Notmuch run on Linux only.  Inotify is a Linux kernel API.  Some other
> OSes have their own native file event notification facilities, but not
> all of them have it, and most (if not all) only support file event
> notifications for certain file systems and not for others (e.g., not for
> NFS).

Yeah, it's worth saying that, even if I think Austin knows. I was
thinking that an alternative approach might be to have either notmuch
new or notmuch insert take a file name on the command line, and let the
user call it via what ever kind of directory watcher utility works on
their system.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [Paul Wise] Bug#843127: notmuch: race condition in `notmuch new`?

[J. Lewis Muir]

On 11/12, Austin Clements wrote:
> Quoth David Bremner on Nov 04 at  1:26 pm:
> > I agree it looks like a race condition. inotify sounds a bit
> > overcomplicated and perhaps non-portable? It should probably just
> > tolerate disappearing files better, consider that a warning.
> 
> Inotify really *is* the solution.

I don't see how inotify can be the solution unless the idea is to make
Notmuch run on Linux only.  Inotify is a Linux kernel API.  Some other
OSes have their own native file event notification facilities, but not
all of them have it, and most (if not all) only support file event
notifications for certain file systems and not for others (e.g., not for
NFS).

Regards,

Lewis
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] fix out of tree build

[Tomi Ollila]

With working tests.
---

This is v2 of id:1479066903-28310-1-git-send-email-tomi.oll...@iki.fi

with this ${srcdir} being absolute path will also work (relevant in tests!)

... and commit message is fixed: I accuse qt for "shadow build" ;)

Out of tree builds breaks every now and then; perhaps it is broken less
often in the future as I plan to start doing my "production" builds
out of tree...

 configure   | 6 --
 test/test-lib-common.sh | 7 +++
 test/test-lib.sh| 6 +++---
 3 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index 2a0ce9e..8174383 100755
--- a/configure
+++ b/configure
@@ -342,8 +342,8 @@ int main(void) {
 return 0;
 }
 EOF
-if ${CC} ${CFLAGS} _libversion.c -o _libversion > /dev/null 2>&1 && \
-   ./_libversion > _libversion.sh && . ./_libversion.sh
+if ${CC} ${CFLAGS} -I"$srcdir" _libversion.c -o _libversion > /dev/null 2>&1 \
+   && ./_libversion > _libversion.sh && . ./_libversion.sh
 then
 printf "OK.\n"
 else
@@ -1158,6 +1158,8 @@ cat > sh.config < OUTPUT
 }
 
 test_ruby() {
-MAIL_DIR=$MAIL_DIR ruby -I $TEST_DIRECTORY/../bindings/ruby> OUTPUT
+MAIL_DIR=$MAIL_DIR ruby -I $SOURCE_DIRECTORY/bindings/ruby> OUTPUT
 }
 
 test_C () {
 exec_file="test${test_count}"
 test_file="${exec_file}.c"
 cat > ${test_file}
-${TEST_CC} ${TEST_CFLAGS} -I${TEST_DIRECTORY} -I${TEST_DIRECTORY}/../lib 
-o ${exec_file} ${test_file} -L${TEST_DIRECTORY}/../lib/ -lnotmuch -ltalloc
+${TEST_CC} ${TEST_CFLAGS} -I${TEST_DIRECTORY} -I${SOURCE_DIRECTORY}/lib -o 
${exec_file} ${test_file} -L${TEST_DIRECTORY}/../lib/ -lnotmuch -ltalloc
 echo "== stdout ==" > OUTPUT.stdout
 echo "== stderr ==" > OUTPUT.stderr
 ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr
-- 
2.7.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Test: fix draft test for emacs23

[Tomi Ollila]

On Mon, Nov 14 2016, David Bremner  wrote:

> Mark Walters  writes:
>
>> emacs24 and emacs23 have different secure tag defaults: in particular,
>> mml-secure-message-sign only signs the part on emacs23 but the whole
>> message on emacs24. This difference makes one of the draft tests fail
>> (which causes a cascade of later failures) on emacs23. It seems that
>> travis uses emacs23 so it is useful to fix this.
>>
>> We do this by forcing the whole message to be signed in either case --
>> the code snippet is extracted from mml-secure-message-sign on emacs24.
>> ---
>
> Fixes the tests on emacs 23 for me, without the other other emacs23
> compatibility patch
>
> It's a little ugly, but it's the test suite. So I guess we go for it.

If you refer tabs converted to spaces then I agree on ugliness >;)

+1

>
> d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Test: fix draft test for emacs23

[David Bremner]

Mark Walters  writes:

> emacs24 and emacs23 have different secure tag defaults: in particular,
> mml-secure-message-sign only signs the part on emacs23 but the whole
> message on emacs24. This difference makes one of the draft tests fail
> (which causes a cascade of later failures) on emacs23. It seems that
> travis uses emacs23 so it is useful to fix this.
>
> We do this by forcing the whole message to be signed in either case --
> the code snippet is extracted from mml-secure-message-sign on emacs24.
> ---

Fixes the tests on emacs 23 for me, without the other other emacs23
compatibility patch

It's a little ugly, but it's the test suite. So I guess we go for it.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Test: fix draft test for emacs23

[Mark Walters]

emacs24 and emacs23 have different secure tag defaults: in particular,
mml-secure-message-sign only signs the part on emacs23 but the whole
message on emacs24. This difference makes one of the draft tests fail
(which causes a cascade of later failures) on emacs23. It seems that
travis uses emacs23 so it is useful to fix this.

We do this by forcing the whole message to be signed in either case --
the code snippet is extracted from mml-secure-message-sign on emacs24.
---

The main thing to note with this is that the motivation for fixing is
that travis seems to use emacs23.

Best wishes

Mark



test/T630-emacs-draft.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/test/T630-emacs-draft.sh b/test/T630-emacs-draft.sh
index 46fc356..cd9e33a 100755
--- a/test/T630-emacs-draft.sh
+++ b/test/T630-emacs-draft.sh
@@ -32,7 +32,9 @@ test_begin_subtest "Saving a signed draft adds header"
 test_emacs '(notmuch-mua-mail)
(message-goto-subject)
(insert "draft-test-0003")
-   (mml-secure-message-sign)
+;; We would use (mml-secure-message-sign) but on emacs23
+;; that only signs the part, not the whole message.
+(mml-secure-message mml-secure-method '\''sign)
(notmuch-draft-save)
(test-output)'
 header_count=$(notmuch show --format=raw subject:draft-test-0003 | grep -c 
^X-Notmuch-Emacs-Secure)
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 12/11/16 00:20, David Bremner wrote:
> Matthew Lear  writes:
> 
>> I can trigger the bug if I discard text/plain though. Did you try that? 
> 
> I'm not sure what you mean by "discard", but closing it isn't enough. I
> think more of your settings are needed to replicate the problem with a
> multipart/alternative message.
> 

Apologies, David. I wasn't clear at all. I meant that if I discourage
text/plain in a multipart/alternative message, I can trigger the bug.
If I use the following tiny-init.el...

(require 'shr)
(setq mm-text-html-renderer 'shr)
(setq mime-view-text/html-previewer 'shr)
(setq mm-html-inhibit-images nil)
(setq mm-html-blocked-images nil)

(require 'notmuch)
(global-set-key [f1]  'notmuch)
(setq notmuch-mua-compose-in 'new-frame)
(setq notmuch-multipart/alternative-discouraged '("text/plain"))


...and load it using 'emacs-25 -q --load ~/.emacs.d/tiny-init.el',
searching for any mimetype:text/html and replying to it triggers the bug
for me.

Commenting out the setting of notmuch-mua-compose-in is enough to
prevent the problem from occurring.
Cheers,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch