[PATCH] notmuch-emacs-mua: avoid extra separators at the end of the line

[Jani Nikula]

Currently the --to/--cc/--bcc options add "u...@example.com, " to the
message headers, with the the unnecessary ", " separator after the
last address, regardless of how many addresses are being added.

This used to be fine, but with recent emacs mm, trying to send the
email with the trailing commas leads to prompt:

  Email address  looks invalid; send anyway? (y or n)

Fix this by only adding the commas between addresses, avoiding the
trailing commas.
---
 emacs/notmuch-emacs-mua | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-emacs-mua b/emacs/notmuch-emacs-mua
index a521497784ec..254e6407cece 100755
--- a/emacs/notmuch-emacs-mua
+++ b/emacs/notmuch-emacs-mua
@@ -41,6 +41,9 @@ CREATE_FRAME=
 ELISP=
 MAILTO=
 HELLO=
+TO_SEP=
+CC_SEP=
+BCC_SEP=
 
 # Short options compatible with mutt(1).
 while getopts :s:c:b:i:h opt; do
@@ -86,13 +89,16 @@ while getopts :s:c:b:i:h opt; do
ELISP="${ELISP} (message-goto-subject) (insert \"${OPTARG}\")"
;;
--to)
-   ELISP="${ELISP} (message-goto-to) (insert \"${OPTARG}, \")"
+   ELISP="${ELISP} (message-goto-to) (insert \"${TO_SEP}${OPTARG}\")"
+   TO_SEP=", "
;;
--cc|c)
-   ELISP="${ELISP} (message-goto-cc) (insert \"${OPTARG}, \")"
+   ELISP="${ELISP} (message-goto-cc) (insert \"${CC_SEP}${OPTARG}\")"
+   CC_SEP=", "
;;
--bcc|b)
-   ELISP="${ELISP} (message-goto-bcc) (insert \"${OPTARG}, \")"
+   ELISP="${ELISP} (message-goto-bcc) (insert \"${BCC_SEP}${OPTARG}\")"
+   BCC_SEP=", "
;;
--body|i)
ELISP="${ELISP} (message-goto-body) (insert-file \"${OPTARG}\")"
-- 
2.39.2

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [RFC 3/5] lib: remove commented out NOTMUCH_DEPRECATED()

[Jani Nikula]

On Wed, 13 Oct 2021, David Bremner  wrote:
> Jani Nikula  writes:
>
>> Having comments between the documentation comment and the object being
>> documented throws Hawkmoth off. Remove the placeholder
>> NOTMUCH_DEPRECATED() comments.
>>
>> ---
>>
>> They were added in commit e5f3c3ed5024 ("lib: add stub for
>> notmuch_database_open_with_config"); maybe the comment markers should
>> be removed instead?
>
> Yes, those should be uncommented, I think.

Thanks, sent separately. Though it does give deprecation warnings now.

BR,
Jani.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [RFC 2/5] lib: remove enum names from typedefs

[Jani Nikula]

On Wed, 13 Oct 2021, Tomi Ollila  wrote:
> On Wed, Oct 13 2021, David Bremner wrote:
>
>> Jani Nikula  writes:
>>
>>> Given the named enum typedef:
>>>
>>> typedef enum _name_t { ... } name_t;
>>>
>>> Hawkmoth uses _name_t as the name for documentation. Typedef the
>>> anonymous enum:
>>>
>>> typedef enum { ... } name_t;
>>>
>>> instead to use name_t. We don't really need the enum names for
>>> anything, and not all of the enum typedefs have them anyway.
>>> ---
>> I guess that change would be fine independent of the rest?
>
> I had the same thought.

Added more typedef enums outside of notmuch.h, removed hawkmoth
references, and resent separately.

BR,
Jani.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] lib: fix commented out NOTMUCH_DEPRECATED()

[Jani Nikula]

Remove the comment markers from the placeholder NOTMUCH_DEPRECATED(),
added in commit e5f3c3ed5024 ("lib: add stub for
notmuch_database_open_with_config").

---

This does lead to deprecation warnings now.
---
 lib/notmuch.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index 546643e80cbb..b2227daef6ad 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -323,7 +323,7 @@ typedef enum {
  * config_path="" and error_message=NULL
  * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
  */
-/* NOTMUCH_DEPRECATED(5, 4) */
+NOTMUCH_DEPRECATED(5, 4)
 notmuch_status_t
 notmuch_database_open (const char *path,
   notmuch_database_mode_t mode,
@@ -335,7 +335,7 @@ notmuch_database_open (const char *path,
  * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
  *
  */
-/* NOTMUCH_DEPRECATED(5, 4) */
+NOTMUCH_DEPRECATED(5, 4)
 notmuch_status_t
 notmuch_database_open_verbose (const char *path,
   notmuch_database_mode_t mode,
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH 2/2] cli: remove enum names from typedefs

[Jani Nikula]

There are some enum typedefs with the enum name:

typedef enum _name_t { ... } name_t;

We don't need or use the enum names _name_t for anything, and not all
of the enum typedefs have them. We have the typedefs specifically to
use the typedef name.

Use the anonymous enum in the typedefs:

typedef enum { ... } name_t;
---
 notmuch-client.h  | 4 ++--
 util/hex-escape.h | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 96d81166d2f5..82ae44e429fc 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -426,13 +426,13 @@ mime_node_seek_dfs (mime_node_t *node, int n);
 const _notmuch_message_crypto_t *
 mime_node_get_message_crypto_status (mime_node_t *node);
 
-typedef enum dump_formats {
+typedef enum {
 DUMP_FORMAT_AUTO,
 DUMP_FORMAT_BATCH_TAG,
 DUMP_FORMAT_SUP
 } dump_format_t;
 
-typedef enum dump_includes {
+typedef enum {
 DUMP_INCLUDE_TAGS  = 1,
 DUMP_INCLUDE_CONFIG= 2,
 DUMP_INCLUDE_PROPERTIES= 4
diff --git a/util/hex-escape.h b/util/hex-escape.h
index 8703334cda8d..83a4c6f146fe 100644
--- a/util/hex-escape.h
+++ b/util/hex-escape.h
@@ -5,7 +5,7 @@
 extern "C" {
 #endif
 
-typedef enum hex_status {
+typedef enum {
 HEX_SUCCESS = 0,
 HEX_SYNTAX_ERROR,
 HEX_OUT_OF_MEMORY
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH 1/2] lib: remove enum names from typedefs

[Jani Nikula]

There are some enum typedefs with the enum name:

typedef enum _name_t { ... } name_t;

We don't need or use the enum names _name_t for anything, and not all
of the enum typedefs have them. We have the typedefs specifically to
use the typedef name.

Use the anonymous enum in the typedefs:

typedef enum { ... } name_t;
---
 lib/database-private.h | 2 +-
 lib/notmuch-private.h  | 4 ++--
 lib/notmuch.h  | 6 +++---
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 8b9d67feec70..7a0450515f44 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -160,7 +160,7 @@ operator&= (_notmuch_features , _notmuch_features b)
 
 /*
  * Configuration options for xapian database fields */
-typedef enum notmuch_field_flags {
+typedef enum {
 NOTMUCH_FIELD_NO_FLAGS = 0,
 NOTMUCH_FIELD_EXTERNAL = 1 << 0,
 NOTMUCH_FIELD_PROBABILISTIC = 1 << 1,
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 093c29b1bbc5..e9ce74a4b178 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -121,7 +121,7 @@ typedef enum {
  */
 #define NOTMUCH_MESSAGE_ID_MAX (200 - sizeof 
(NOTMUCH_METADATA_THREAD_ID_PREFIX))
 
-typedef enum _notmuch_private_status {
+typedef enum {
 /* First, copy all the public status values. */
 NOTMUCH_PRIVATE_STATUS_SUCCESS = 
NOTMUCH_STATUS_SUCCESS,
 NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY   = 
NOTMUCH_STATUS_OUT_OF_MEMORY,
@@ -173,7 +173,7 @@ typedef enum _notmuch_private_status {
  (notmuch_status_t) private_status)
 
 /* Flags shared by various lookup functions. */
-typedef enum _notmuch_find_flags {
+typedef enum {
 /* Lookup without creating any documents.  This is the default
  * behavior. */
 NOTMUCH_FIND_LOOKUP = 0,
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 546643e80cbb..ca5878a949bc 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -112,7 +112,7 @@ typedef int notmuch_bool_t;
  * A zero value (NOTMUCH_STATUS_SUCCESS) indicates that the function
  * completed without error. Any other value indicates an error.
  */
-typedef enum _notmuch_status {
+typedef enum {
 /**
  * No error occurred.
  */
@@ -1678,7 +1678,7 @@ notmuch_message_reindex (notmuch_message_t *message,
 /**
  * Message flags.
  */
-typedef enum _notmuch_message_flag {
+typedef enum {
 NOTMUCH_MESSAGE_FLAG_MATCH,
 NOTMUCH_MESSAGE_FLAG_EXCLUDED,
 
@@ -2524,7 +2524,7 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 /**
  * Configuration keys known to libnotmuch
  */
-typedef enum _notmuch_config_key {
+typedef enum {
 NOTMUCH_CONFIG_FIRST,
 NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
 NOTMUCH_CONFIG_MAIL_ROOT,
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [RFC 0/5] doc: api docs overhaul

[Jani Nikula]

On Wed, 13 Oct 2021, David Bremner  wrote:
> Jani Nikula  writes:
>
>> The main downside is that Hawkmoth is not available via distro
>> packaging, only PyPI.
>
> I guess another issue is the new dependency on Clang +
> bindings. Although maybe that's a fair trade for dropping doxygen.

Doxygen already depends on libclang:

$ dpkg -s doxygen | grep "^Depends:"
Depends: libc6 (>= 2.29), libclang-cpp11 (>= 1:11.0.1), libclang1-11 (>=
  1:5.0~svn298832-1~), libgcc-s1 (>= 3.0), libllvm11 (>=
  1:9~svn298832-1~), libstdc++6 (>= 7), libxapian30 (>= 1.4.14~)

The difference is that the libclang python bindings depend on the dev
packages:

$ dpkg -s python3-clang-11 | grep "^Depends:"
Depends: python3, libclang-11-dev

$ dpkg -s libclang-11-dev | grep "^Depends:"
Depends: libstdc++-10-dev, libgcc-10-dev, libobjc-10-dev, libclang1-11
  (= 1:11.0.1-2), libclang-common-11-dev (= 1:11.0.1-2)

I think you're probably in a better position to judge the relative sizes
of the dependency trees than me, but the libclang dependency is already
there with Doxygen. The Python parts are pretty small, it's the dev
packages vs. Doxygen that makes the difference.

> What would be the plan for transition while distros package hawkmoth?
> Embed it in notmuch? Ship the compiled docs?

I'm afraid I don't have an answer to that. What do projects do when they
depend on packages from PyPI? It's a kind of chicken and egg problem.

I saw that you made the Debian packaging request; many thanks! But is
that enough? What is your stance on imposing new dependencies on other
distros?


BR,
Jani.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [RFC 1/5] doc: replace doxygen with hawkmoth sphinx extension for api docs

[Jani Nikula]

On Wed, 13 Oct 2021, Tomi Ollila  wrote:
> Interesting stuff...
>
> On Tue, Oct 12 2021, Jani Nikula wrote:
>
>> Hawkmoth is a Sphinx extension for incorporating documentation
>> comments from C source to Sphinx. For Notmuch API documentation needs,
>> Doxygen is overkill, and Hawkmoth also directly integrates with
>> Sphinx.
>
> // stuff deleted //
>
>> +
>>  # -- Options for HTML output --
>>  
>>  # The theme to use for HTML and HTML Help pages.  See the documentation for
>> @@ -172,6 +178,13 @@ man_pages = [
>>   [notmuch_authors], 1),
>>  ]
>>  
>> +if tags.has('WITH_API_DOCS'):
>> +man_pages += [
>> +('man3/notmuch', 'notmuch',
>> + u'notmuch library',
>
> öld stuff...??? (the u'' i mean) :O

Yeah, it's still used throughout in man_pages, decided to follow suit
instead of making that cleanup now.

BR,
Jani.

>
> Tomi 
>
>
>> + [notmuch_authors], 3),
>> +]
>> +
>>  # If true, show URL addresses after external links.
>>  #man_show_urls = False
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[RFC 5/5] lib: documentation comment overhaul for hawkmoth

[Jani Nikula]

Convert from Doxygen to Hawkmoth, and improve while at it.

- Use rst :param: field lists for parameters.

- Use :deprecated: and :since:.

- Cross-reference all constructs.

- Use preformatted text for TRUE, FALSE, NULL, parameter references,
  etc.

Comment changes only.
---
 lib/notmuch.h | 1971 +++--
 1 file changed, 1075 insertions(+), 896 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index 700e9b7b25c1..9d78ff64d7c7 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -19,13 +19,12 @@
  */
 
 /**
- * @defgroup notmuch The notmuch API
+ * The notmuch API
  *
  * Not much of an email library, (just index and search)
- *
- * @{
  */
 
+/**/
 #ifndef NOTMUCH_H
 #define NOTMUCH_H
 
@@ -72,23 +71,25 @@ NOTMUCH_BEGIN_DECLS
 /**
  * Check the version of the notmuch library being compiled against.
  *
- * Return true if the library being compiled against is of the
- * specified version or above. For example:
+ * :param major: major version
+ * :param minor: minor version
+ * :param micro: micro version
+ *
+ * :return: true if the library being compiled against is of the
+ *  specified version or above.
+ *
+ * For example::
  *
- * @code
- * #if LIBNOTMUCH_CHECK_VERSION(3, 1, 0)
- * (code requiring libnotmuch 3.1.0 or above)
- * #endif
- * @endcode
+ *   #if LIBNOTMUCH_CHECK_VERSION(3, 1, 0)
+ *   // (code requiring libnotmuch 3.1.0 or above)
+ *   #endif
  *
  * LIBNOTMUCH_CHECK_VERSION has been defined since version 3.1.0; to
- * check for versions prior to that, use:
+ * check for versions prior to that, use::
  *
- * @code
- * #if !defined(NOTMUCH_CHECK_VERSION)
- * (code requiring libnotmuch prior to 3.1.0)
- * #endif
- * @endcode
+ *   #if !defined(NOTMUCH_CHECK_VERSION)
+ *   // (code requiring libnotmuch prior to 3.1.0)
+ *   #endif
  */
 #define LIBNOTMUCH_CHECK_VERSION(major, minor, micro)   \
 (LIBNOTMUCH_MAJOR_VERSION > (major) || 
 \
@@ -104,8 +105,9 @@ typedef int notmuch_bool_t;
 /**
  * Status codes used for the return values of most functions.
  *
- * A zero value (NOTMUCH_STATUS_SUCCESS) indicates that the function
- * completed without error. Any other value indicates an error.
+ * A zero value (:c:enumerator:`NOTMUCH_STATUS_SUCCESS`) indicates
+ * that the function completed without error. Any other value
+ * indicates an error.
  */
 typedef enum {
 /**
@@ -124,9 +126,13 @@ typedef enum {
 /**
  * A Xapian exception occurred.
  *
- * @todo We don't really want to expose this lame XAPIAN_EXCEPTION
- * value. Instead we should map to things like DATABASE_LOCKED or
- * whatever.
+ * .. note::
+ *
+ *   We don't really want to expose this lame
+ *   :c:enumerator:`NOTMUCH_STATUS_XAPIAN_EXCEPTION`
+ *   value. Instead we should map to things like
+ *   :c:enumerator:`NOTMUCH_STATUS_READ_ONLY_DATABASE` or
+ *   whatever.
  */
 NOTMUCH_STATUS_XAPIAN_EXCEPTION,
 /**
@@ -145,22 +151,22 @@ typedef enum {
  */
 NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID,
 /**
- * The user erroneously passed a NULL pointer to a notmuch
+ * The user erroneously passed a ``NULL`` pointer to a notmuch
  * function.
  */
 NOTMUCH_STATUS_NULL_POINTER,
 /**
- * A tag value is too long (exceeds NOTMUCH_TAG_MAX).
+ * A tag value is too long (exceeds :c:macro:`NOTMUCH_TAG_MAX`).
  */
 NOTMUCH_STATUS_TAG_TOO_LONG,
 /**
- * The notmuch_message_thaw function has been called more times
- * than notmuch_message_freeze.
+ * The :c:func:`notmuch_message_thaw` function has been called
+ * more times than :c:func:`notmuch_message_freeze`.
  */
 NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW,
 /**
- * notmuch_database_end_atomic has been called more times than
- * notmuch_database_begin_atomic.
+ * :c:func:`notmuch_database_end_atomic` has been called more
+ * times than :c:func:`notmuch_database_begin_atomic`.
  */
 NOTMUCH_STATUS_UNBALANCED_ATOMIC,
 /**
@@ -227,7 +233,7 @@ typedef enum {
 } notmuch_status_t;
 
 /**
- * Get a string representation of a notmuch_status_t value.
+ * Get a string representation of a :c:type:`notmuch_status_t` value.
  *
  * The result is read-only.
  */
@@ -236,61 +242,78 @@ notmuch_status_to_string (notmuch_status_t status);
 
 /* Various opaque data types. For each notmuch__t see the various
  * notmuch_ functions below. */
+/** Notmuch opaque database type. */
 typedef struct _notmuch_database notmuch_database_t;
+/** Notmuch opaque query type. */
 typedef struct _notmuch_query notmuch_query_t;
+/** Notmuch opaque threads type. */
 typedef struct _notmuch_threads notmuch_threads_t;
+/** Notmuch opaque thread type. */
 typedef struct _notmuch_thread notmuch_thread_t;
+/** Notmuch opaque messages type. */
 typedef struct _notmuch_messages notmuch_messages_t;
+/** Notmuch opaque message type. */
 typedef struct 

[RFC 4/5] lib: remove #ifndef __DOXYGEN__ conditions

[Jani Nikula]

These are now unnecessary. Hawkmoth does not require similar
constructs.
---
 lib/notmuch.h | 7 ---
 1 file changed, 7 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index abbc5ace9d1b..700e9b7b25c1 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -29,8 +29,6 @@
 #ifndef NOTMUCH_H
 #define NOTMUCH_H
 
-#ifndef __DOXYGEN__
-
 #ifdef  __cplusplus
 # define NOTMUCH_BEGIN_DECLS  extern "C" {
 # define NOTMUCH_END_DECLS}
@@ -71,9 +69,6 @@ NOTMUCH_BEGIN_DECLS
 #define NOTMUCH_DEPRECATED(major, minor) __attribute__ ((deprecated))
 #endif
 
-
-#endif /* __DOXYGEN__ */
-
 /**
  * Check the version of the notmuch library being compiled against.
  *
@@ -241,7 +236,6 @@ notmuch_status_to_string (notmuch_status_t status);
 
 /* Various opaque data types. For each notmuch__t see the various
  * notmuch_ functions below. */
-#ifndef __DOXYGEN__
 typedef struct _notmuch_database notmuch_database_t;
 typedef struct _notmuch_query notmuch_query_t;
 typedef struct _notmuch_threads notmuch_threads_t;
@@ -255,7 +249,6 @@ typedef struct _notmuch_config_list notmuch_config_list_t;
 typedef struct _notmuch_config_values notmuch_config_values_t;
 typedef struct _notmuch_config_pairs notmuch_config_pairs_t;
 typedef struct _notmuch_indexopts notmuch_indexopts_t;
-#endif /* __DOXYGEN__ */
 
 /**
  * Create a new, empty notmuch database located at 'path'.
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[RFC 3/5] lib: remove commented out NOTMUCH_DEPRECATED()

[Jani Nikula]

Having comments between the documentation comment and the object being
documented throws Hawkmoth off. Remove the placeholder
NOTMUCH_DEPRECATED() comments.

---

They were added in commit e5f3c3ed5024 ("lib: add stub for
notmuch_database_open_with_config"); maybe the comment markers should
be removed instead?
---
 lib/notmuch.h | 2 --
 1 file changed, 2 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index ca5878a949bc..abbc5ace9d1b 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -323,7 +323,6 @@ typedef enum {
  * config_path="" and error_message=NULL
  * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
  */
-/* NOTMUCH_DEPRECATED(5, 4) */
 notmuch_status_t
 notmuch_database_open (const char *path,
   notmuch_database_mode_t mode,
@@ -335,7 +334,6 @@ notmuch_database_open (const char *path,
  * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
  *
  */
-/* NOTMUCH_DEPRECATED(5, 4) */
 notmuch_status_t
 notmuch_database_open_verbose (const char *path,
   notmuch_database_mode_t mode,
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[RFC 2/5] lib: remove enum names from typedefs

[Jani Nikula]

Given the named enum typedef:

typedef enum _name_t { ... } name_t;

Hawkmoth uses _name_t as the name for documentation. Typedef the
anonymous enum:

typedef enum { ... } name_t;

instead to use name_t. We don't really need the enum names for
anything, and not all of the enum typedefs have them anyway.
---
 lib/notmuch.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index 546643e80cbb..ca5878a949bc 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -112,7 +112,7 @@ typedef int notmuch_bool_t;
  * A zero value (NOTMUCH_STATUS_SUCCESS) indicates that the function
  * completed without error. Any other value indicates an error.
  */
-typedef enum _notmuch_status {
+typedef enum {
 /**
  * No error occurred.
  */
@@ -1678,7 +1678,7 @@ notmuch_message_reindex (notmuch_message_t *message,
 /**
  * Message flags.
  */
-typedef enum _notmuch_message_flag {
+typedef enum {
 NOTMUCH_MESSAGE_FLAG_MATCH,
 NOTMUCH_MESSAGE_FLAG_EXCLUDED,
 
@@ -2524,7 +2524,7 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 /**
  * Configuration keys known to libnotmuch
  */
-typedef enum _notmuch_config_key {
+typedef enum {
 NOTMUCH_CONFIG_FIRST,
 NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST,
 NOTMUCH_CONFIG_MAIL_ROOT,
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[RFC 1/5] doc: replace doxygen with hawkmoth sphinx extension for api docs

[Jani Nikula]

Hawkmoth is a Sphinx extension for incorporating documentation
comments from C source to Sphinx. For Notmuch API documentation needs,
Doxygen is overkill, and Hawkmoth also directly integrates with
Sphinx.

Hawkmoth is currently not packaged for distros, just PyPI. To try this
out on Debian:

$ sudo apt install python3-clang-11 python3-venv
$ python3 -m venv --system-site-packages .venv
$ . .venv/bin/activate
$ pip install hawkmoth
$ ./configure
$ make build-man sphinx-html

Note that Doxygen also depends on libclang; the additional dependency
are the clang python bindings.

Initially, there will be lots of warnings due to documentation comment
formatting. These will be fixed in separate changes.

For more information on Hawkmoth, see
https://github.com/jnikula/hawkmoth
---
 configure|  27 ++--
 doc/.gitignore   |   1 -
 doc/Makefile.local   |  54 +++-
 doc/conf.py  |  13 ++
 doc/doxygen.cfg  | 298 ---
 doc/index.rst|   1 +
 doc/man3/notmuch.rst |   5 +
 7 files changed, 53 insertions(+), 346 deletions(-)
 delete mode 100644 doc/doxygen.cfg
 create mode 100644 doc/man3/notmuch.rst

diff --git a/configure b/configure
index 6c3a38f1098a..4d6c62645561 100755
--- a/configure
+++ b/configure
@@ -867,17 +867,6 @@ if [ $WITH_EMACS = "1" ]; then
 fi
 fi
 
-have_doxygen=0
-if [ $WITH_API_DOCS = "1" ] ; then
-printf "Checking if doxygen is available... "
-if command -v doxygen > /dev/null; then
-   printf "Yes.\n"
-   have_doxygen=1
-else
-   printf "No (so will not install api docs)\n"
-fi
-fi
-
 have_ruby_dev=0
 if [ $WITH_RUBY = "1" ] ; then
 printf "Checking for ruby development files... "
@@ -914,6 +903,15 @@ if [ $WITH_DOCS = "1" ] ; then
 else
printf "No (so will not install info pages).\n"
 fi
+if [ $WITH_API_DOCS = "1" ] ; then
+   printf "Checking if hawkmoth is available... "
+   if ${python} -m hawkmoth -h > /dev/null 2>&1; then
+   printf "Yes.\n"
+   else
+   printf "No (so will not install api docs)\n"
+   WITH_API_DOCS=0
+   fi
+fi
 fi
 
 if [ $WITH_DESKTOP = "1" ]; then
@@ -1372,8 +1370,8 @@ HAVE_MAKEINFO=${have_makeinfo}
 # Whether there's an install-info binary available for installing info format 
documentation
 HAVE_INSTALL_INFO=${have_install_info}
 
-# Whether there's a doxygen binary available for building api documentation
-HAVE_DOXYGEN=${have_doxygen}
+# Whether there's hawkmoth sphinx extension available for building api 
documentation
+WITH_API_DOCS=${WITH_API_DOCS}
 
 # The directory to which desktop files should be installed
 desktop_dir = \$(prefix)/share/applications
@@ -1576,6 +1574,9 @@ EOF
 if [ $WITH_PYTHON_DOCS = "1" ]; then
echo "tags.add('WITH_PYTHON')"
 fi
+if [ $WITH_API_DOCS = "1" ]; then
+echo "tags.add('WITH_API_DOCS')"
+fi
 printf "rsti_dir = '%s'\n" "$(cd emacs && pwd -P)"
 } > sphinx.config
 
diff --git a/doc/.gitignore b/doc/.gitignore
index bbb749fabda3..f25d695cccbf 100644
--- a/doc/.gitignore
+++ b/doc/.gitignore
@@ -1,3 +1,2 @@
 *.pyc
 /_build
-/config.dox
diff --git a/doc/Makefile.local b/doc/Makefile.local
index f476d1da46c9..10281a82efff 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -9,26 +9,31 @@ DOCBUILDDIR  := $(dir)/_build
 
 # Internal variables.
 ALLSPHINXOPTS   := $(SPHINXOPTS) $(srcdir)/$(dir)
-APIMAN := $(DOCBUILDDIR)/man/man3/notmuch.3
-DOXYFILE   := $(srcdir)/$(dir)/doxygen.cfg
 
 MAN1_RST := $(wildcard $(srcdir)/doc/man1/*.rst)
+ifeq ($(WITH_API_DOCS),1)
+   MAN3_RST := $(wildcard $(srcdir)/doc/man3/*.rst)
+else
+   MAN3_RST :=
+endif
 MAN5_RST := $(wildcard $(srcdir)/doc/man5/*.rst)
 MAN7_RST := $(wildcard $(srcdir)/doc/man7/*.rst)
-MAN_RST_FILES := $(MAN1_RST) $(MAN5_RST) $(MAN7_RST)
+MAN_RST_FILES := $(MAN1_RST) $(MAN3_RST) $(MAN5_RST) $(MAN7_RST)
 ALL_RST_FILES := $(MAN_RST_FILES) $(srcdir)/doc/notmuch-emacs.rst
 
 MAN1_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN1_RST:.rst=.1))
+MAN3_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN3_RST:.rst=.3))
 MAN5_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN5_RST:.rst=.5))
 MAN7_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN7_RST:.rst=.7))
-MAN_ROFF_FILES := $(MAN1_ROFF) $(MAN5_ROFF) $(MAN7_ROFF)
+MAN_ROFF_FILES := $(MAN1_ROFF) $(MAN3_ROFF) $(MAN5_ROFF) $(MAN7_ROFF)
 
 MAN_GZIP_FILES := $(addsuffix .gz,${MAN_ROFF_FILES})
 
 MAN1_TEXI := $(patsubst 
$(srcdir)/doc/man1/%.rst,$(DOCBUILDDIR)/texinfo/%.texi,$(MAN1_RST))
+MAN3_TEXI := $(patsubst 
$(srcdir)/doc/man3/%.rst,$(DOCBUILDDIR)/texinfo/%.texi,$(MAN3_RST))
 MAN5_TEXI := $(patsubst 
$(srcdir)/doc/man5/%.rst,$(DOCBUILDDIR)/texinfo/%.texi,$(MAN5_RST))
 MAN7_TEXI := $(patsubst 
$(srcdir)/doc/man7/%.rst,$(DOCBUILDDIR)/texinfo/%.texi,$(MAN7_RST))
-INFO_TEXI_FILES := $(MAN1_TEXI) $(MAN5_TEXI) $(MAN7_TEXI)
+INFO_TEXI_FILES := $(MAN1_TEXI) 

[RFC 0/5] doc: api docs overhaul

[Jani Nikula]

I have a pet project to incorporate C documentation comments written in
reStructuredText to Sphinx based documentation [1]. For Notmuch needs,
it replaces Doxygen with something that directly integrates with Sphinx
through an extension.

I've split the series to configure/build changes, functional code
changes in notmuch.h, and comment changes in notmuch.h.

With this, the 'make sphinx-html' build includes the API, with
cross-references working across the man pages, etc. We didn't have this
before.

The main downside is that Hawkmoth is not available via distro
packaging, only PyPI.


BR,
Jani.


[1] https://github.com/jnikula/hawkmoth


Jani Nikula (5):
  doc: replace doxygen with hawkmoth sphinx extension for api docs
  lib: remove enum names from typedefs
  lib: remove commented out NOTMUCH_DEPRECATED()
  lib: remove #ifndef __DOXYGEN__ conditions
  lib: documentation comment overhaul for hawkmoth

 configure|   27 +-
 doc/.gitignore   |1 -
 doc/Makefile.local   |   54 +-
 doc/conf.py  |   13 +
 doc/doxygen.cfg  |  298 ---
 doc/index.rst|1 +
 doc/man3/notmuch.rst |5 +
 lib/notmuch.h| 1986 +++---
 8 files changed, 1131 insertions(+), 1254 deletions(-)
 delete mode 100644 doc/doxygen.cfg
 create mode 100644 doc/man3/notmuch.rst

-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH 0/5] doc: man page cleanup

[Jani Nikula]

On Sat, 22 May 2021, David Bremner  wrote:
> Jani Nikula  writes:
>> On Sat, 22 May 2021, David Bremner  wrote:
>> I was wondering about that myself. Should I respin with that?
>>
>
> Either that or an addon to the series, whatever is easiest.

If I get to choose, I'll take the latter. ;)

BR,
Jani.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH 0/5] doc: man page cleanup

[Jani Nikula]

On Sat, 22 May 2021, David Bremner  wrote:
> Jani Nikula  writes:
>
>> I saw [1] and decided to give it some love. :)
>>
>> Mostly this is all about adding html cross-references all over the place
>> while trying to keep the roff man pages roughly the same. Also updating
>> the man page rst becomes easier by setting a clean example.
>>
>> BR,
>> Jani.
>>
>
> Series LGTM, based on a bit of sampling of the outputs.  I did wonder
> if the FILES section in notmuch-config should also move to notmuch(1). I
> can see arguments for both options.

I was wondering about that myself. Should I respin with that?

BR,
Jani.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH 4/5] doc: use program and option directives to document options

[Jani Nikula]

Use the program and option directives to document the subcommand
options. This unifies a lot of option documentation throughout.

This also makes it possible to reference options with :option:`--foo`
(within .. program::) or :option:`subcommand --foo` (globally). This
is left for later work.

See 
https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html#directive-program

Note: There is a lot of indentation change, but intentionally there is
no reflow. Using 'git diff -w' or 'git show -w' to ignore white space
changes makes this a very easy change to review.
---
 doc/man1/notmuch-address.rst   | 174 +
 doc/man1/notmuch-compact.rst   |  20 +-
 doc/man1/notmuch-config.rst|  45 +++--
 doc/man1/notmuch-count.rst |  71 ---
 doc/man1/notmuch-dump.rst  | 146 +++---
 doc/man1/notmuch-emacs-mua.rst |  80 
 doc/man1/notmuch-insert.rst| 100 +-
 doc/man1/notmuch-new.rst   |  67 ---
 doc/man1/notmuch-reindex.rst   |  47 ++---
 doc/man1/notmuch-reply.rst | 117 +--
 doc/man1/notmuch-restore.rst   | 118 ++--
 doc/man1/notmuch-search.rst| 234 +++---
 doc/man1/notmuch-show.rst  | 343 +
 doc/man1/notmuch-tag.rst   |  37 ++--
 doc/man1/notmuch.rst   |  44 +++--
 15 files changed, 874 insertions(+), 769 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index 9eae65cbae8f..7423b6295508 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -20,89 +20,97 @@ See :any:`notmuch-search-terms(7)` for details of the 
supported syntax for
 
 Supported options for **address** include
 
-``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**)
-Presents the results in either JSON, S-Expressions, newline
-character separated plain-text (default), or null character
-separated plain-text (compatible with :manpage:`xargs(1)` -0
-option where available).
-
-``--format-version=N``
-Use the specified structured output format version. This is
-intended for programs that invoke :any:`notmuch(1)` internally. If
-omitted, the latest supported version will be used.
-
-``--output=(sender|recipients|count|address)``
-Controls which information appears in the output. This option can
-be given multiple times to combine different outputs.  When
-neither ``--output=sender`` nor ``--output=recipients`` is
-given, ``--output=sender`` is implied.
-
-**sender**
-Output all addresses from the *From* header.
-
-Note: Searching for **sender** should be much faster than
-searching for **recipients**, because sender addresses are
-cached directly in the database whereas other addresses need
-to be fetched from message files.
-
-**recipients**
-Output all addresses from the *To*, *Cc* and *Bcc* headers.
-
-**count**
-Print the count of how many times was the address encountered
-during search.
-
-Note: With this option, addresses are printed only after the
-whole search is finished. This may take long time.
-
-**address**
-Output only the email addresses instead of the full mailboxes
-with names and email addresses. This option has no effect on
-the JSON or S-Expression output formats.
-
-``--deduplicate=(no|mailbox|address)``
-Control the deduplication of results.
-
-**no**
-Output all occurrences of addresses in the matching
-messages. This is not applicable with ``--output=count``.
-
-**mailbox**
-Deduplicate addresses based on the full, case sensitive name
-and email address, or mailbox. This is effectively the same as
-piping the ``--deduplicate=no`` output to **sort | uniq**, except
-for the order of results. This is the default.
-
-**address**
-Deduplicate addresses based on the case insensitive address
-part of the mailbox. Of all the variants (with different name
-or case), print the one occurring most frequently among the
-matching messages. If ``--output=count`` is specified, include all
-variants in the count.
-
-``--sort=``\ (**newest-first**\ \|\ **oldest-first**)
-This option can be used to present results in either chronological
-order (**oldest-first**) or reverse chronological order
-(**newest-first**).
-
-By default, results will be displayed in reverse chronological
-order, (that is, the newest results will be displayed first).
-
-However, if either ``--output=count`` or ``--deduplicate=address`` is
-specified, this option is ignored and the order of the results is
-unspecified.
-
-``--exclude=(true|false)``
-A message is called "excluded" if it matches at least one tag in
-search.exclude\_tags that does not appear explicitly in the search
-terms. This option specifies whether to omit excluded 

[PATCH 5/5] doc: example command-line option reference

[Jani Nikula]

Example reference to a command-line option using the option role
reference. This creates a hyperlink in html, and the usual boldface
style in man page. This could be used throughout the man pages.
---
 doc/man1/notmuch-config.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index af126289a97f..56f4a1600153 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -248,7 +248,7 @@ CONFIGURATION
 
 Notmuch configuration file search order:
 
-1. File specified by ``--config=FILE`` global option; see
+1. File specified by :option:`notmuch --config` global option; see
:any:`notmuch(1)`.
 
 2. File specified by :envvar:`NOTMUCH_CONFIG` environment variable.
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH 3/5] doc: use envvar directive and role for environment variables

[Jani Nikula]

Make man1/notmuch.rst the single point of truth for describing notmuch
environment variables. Use the envvar directive for that, and
reference them with the envvar role.

Drive-by cleanup configuration file and hook directory search order
documentation.
---
 doc/man1/notmuch-compact.rst   | 10 
 doc/man1/notmuch-config.rst| 42 +-
 doc/man1/notmuch-emacs-mua.rst | 10 
 doc/man1/notmuch.rst   | 36 +
 4 files changed, 49 insertions(+), 49 deletions(-)

diff --git a/doc/man1/notmuch-compact.rst b/doc/man1/notmuch-compact.rst
index 85f611bf44e5..3e3e70c532ca 100644
--- a/doc/man1/notmuch-compact.rst
+++ b/doc/man1/notmuch-compact.rst
@@ -35,16 +35,6 @@ Supported options for **compact** include
 ``--quiet``
 Do not report database compaction progress to stdout.
 
-ENVIRONMENT
-===
-
-The following environment variables can be used to control the behavior
-of notmuch.
-
-**NOTMUCH\_CONFIG**
-Specifies the location of the notmuch configuration file. Notmuch
-will use ${HOME}/.notmuch-config if this variable is not set.
-
 SEE ALSO
 
 
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 5c980a8a4bb8..129d4b810261 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -235,39 +235,39 @@ paths are presumed relative to `$HOME` for items in 
section
 :any:`notmuch-search-terms(7)` for more information about named
 queries.
 
-ENVIRONMENT
-===
-
-The following environment variables can be used to control the behavior
-of notmuch.
-
-**NOTMUCH\_CONFIG**
-Specifies the location of the notmuch configuration file.
-
-**NOTMUCH_PROFILE**
-Selects among notmuch configurations.
-
 FILES
 =
 
 CONFIGURATION
 -
 
-If ``NOTMUCH_CONFIG`` is unset, notmuch tries (in order)
+Notmuch configuration file search order:
 
-- ``$XDG_CONFIG_HOME/notmuch//config`` where  is
-  defined by ``$NOTMUCH_PROFILE`` or "default"
-- ``${HOME}/.notmuch-config`` where  is
-  ``.$NOTMUCH_PROFILE`` or ""
+1. File specified by ``--config=FILE`` global option; see
+   :any:`notmuch(1)`.
+
+2. File specified by :envvar:`NOTMUCH_CONFIG` environment variable.
+
+3. ``$XDG_CONFIG_HOME/notmuch//config`` where 
+   is defined by :envvar:`NOTMUCH_PROFILE` environment variable if
+   set, ``$XDG_CONFIG_HOME/notmuch/default/config`` otherwise.
+
+4. ``$HOME/.notmuch-config.`` where  is defined
+   by :envvar:`NOTMUCH_PROFILE` environment variable if set,
+   ``$HOME/.notmuch-config`` otherwise.
 
 Hooks
 -
 
-If ``database.hook_dir`` is unset, notmuch tries (in order)
+Notmuch hook directory search order:
+
+1. Directory specified by ``database.hook_dir`` configuration option.
+
+2. ``$XDG_CONFIG_HOME/notmuch//hooks`` where 
+   is defined by :envvar:`NOTMUCH_PROFILE` environment variable if
+   set, ``$XDG_CONFIG_HOME/notmuch/default/hooks`` otherwise.
 
-- ``$XDG_CONFIG_HOME/notmuch//hooks`` where  is
-  defined by ``$NOTMUCH_PROFILE`` or "default"
-- ``/.notmuch/hooks``
+3. ``/.notmuch/hooks``
 
 SEE ALSO
 
diff --git a/doc/man1/notmuch-emacs-mua.rst b/doc/man1/notmuch-emacs-mua.rst
index a599b6ebc7bf..c0d5b1a7c476 100644
--- a/doc/man1/notmuch-emacs-mua.rst
+++ b/doc/man1/notmuch-emacs-mua.rst
@@ -72,11 +72,13 @@ Options may be specified multiple times.
 ENVIRONMENT VARIABLES
 =
 
-**EMACS**
-Name of emacs command to invoke. Defaults to "emacs".
+.. envvar:: EMACS
 
-**EMACSCLIENT**
-Name of emacsclient command to invoke. Defaults to "emacsclient".
+   Name of emacs command to invoke. Defaults to "emacs".
+
+.. envvar:: EMACSCLIENT
+
+   Name of emacsclient command to invoke. Defaults to "emacsclient".
 
 SEE ALSO
 
diff --git a/doc/man1/notmuch.rst b/doc/man1/notmuch.rst
index 7b84ef447935..93135bdd6abb 100644
--- a/doc/man1/notmuch.rst
+++ b/doc/man1/notmuch.rst
@@ -52,7 +52,7 @@ Supported global options for ``notmuch`` include
 
 ``--config=FILE``
 Specify the configuration file to use. This overrides any
-configuration file specified by ${NOTMUCH\_CONFIG}. The empty
+configuration file specified by :envvar:`NOTMUCH_CONFIG`. The empty
 string is a permitted and sometimes useful value of *FILE*, which
 tells ``notmuch`` to use only configuration metadata from the database.
 
@@ -79,7 +79,7 @@ use, (or to reconfigure it later).
 The setup command will prompt for your full name, your primary email
 address, any alternate email addresses you use, and the directory
 containing your email archives. Your answers will be written to a
-configuration file in ${NOTMUCH\_CONFIG} (if set) or
+configuration file in :envvar:`NOTMUCH_CONFIG` (if set) or
 ${HOME}/.notmuch-config . This configuration file will be created with
 descriptive comments, making it easy to edit by hand later to change the
 configuration. Or you can run **notmuch setup** again to change the
@@ -128,8 +128,8 

[PATCH 2/5] doc: cross-reference notmuch man pages with actual links

[Jani Nikula]

Add internal hyperlink targets for man pages and cross-reference them
using the any role reference. There are a number of alternatives to
accomplish this, but this seems like the combination that retains the
man page section number and the same boldface style in the man pages.

As a bonus, we get sanity checking on the links; for example
notmuch-search-terms.rst had a reference to notmuch-properties(1)
i.e. the wrong section.

The obvious semantic follow-up change would be to only have meaningful
"see also" references instead of having them all everywhere.
---
 doc/man1/notmuch-address.rst  | 32 
 doc/man1/notmuch-compact.rst  | 26 +++--
 doc/man1/notmuch-config.rst   | 60 +++--
 doc/man1/notmuch-count.rst| 28 +++---
 doc/man1/notmuch-dump.rst | 40 ++-
 doc/man1/notmuch-emacs-mua.rst|  4 +-
 doc/man1/notmuch-insert.rst   | 30 ---
 doc/man1/notmuch-new.rst  | 49 ---
 doc/man1/notmuch-reindex.rst  | 34 
 doc/man1/notmuch-reply.rst| 30 ---
 doc/man1/notmuch-restore.rst  | 38 +-
 doc/man1/notmuch-search.rst   | 32 
 doc/man1/notmuch-show.rst | 37 ++
 doc/man1/notmuch-tag.rst  | 30 ---
 doc/man1/notmuch.rst  | 64 +--
 doc/man5/notmuch-hooks.rst| 54 +-
 doc/man7/notmuch-properties.rst   | 36 +
 doc/man7/notmuch-search-terms.rst | 48 ---
 18 files changed, 359 insertions(+), 313 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index 9193aefe1db6..9eae65cbae8f 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -1,3 +1,5 @@
+.. _notmuch-address(1):
+
 ===
 notmuch-address
 ===
@@ -13,7 +15,7 @@ DESCRIPTION
 Search for messages matching the given search terms, and display the
 addresses from them. Duplicate addresses are filtered out.
 
-See **notmuch-search-terms(7)** for details of the supported syntax for
+See :any:`notmuch-search-terms(7)` for details of the supported syntax for
 .
 
 Supported options for **address** include
@@ -26,7 +28,7 @@ Supported options for **address** include
 
 ``--format-version=N``
 Use the specified structured output format version. This is
-intended for programs that invoke **notmuch(1)** internally. If
+intended for programs that invoke :any:`notmuch(1)` internally. If
 omitted, the latest supported version will be used.
 
 ``--output=(sender|recipients|count|address)``
@@ -116,16 +118,16 @@ This command supports the following special exit status 
codes
 SEE ALSO
 
 
-**notmuch(1)**,
-**notmuch-config(1)**,
-**notmuch-count(1)**,
-**notmuch-dump(1)**,
-**notmuch-hooks(5)**,
-**notmuch-insert(1)**,
-**notmuch-new(1)**,
-**notmuch-reply(1)**,
-**notmuch-restore(1)**,
-**notmuch-search-terms(7)**,
-**notmuch-show(1)**,
-**notmuch-tag(1)**,
-**notmuch-search(1)**
+:any:`notmuch(1)`,
+:any:`notmuch-config(1)`,
+:any:`notmuch-count(1)`,
+:any:`notmuch-dump(1)`,
+:any:`notmuch-hooks(5)`,
+:any:`notmuch-insert(1)`,
+:any:`notmuch-new(1)`,
+:any:`notmuch-reply(1)`,
+:any:`notmuch-restore(1)`,
+:any:`notmuch-search(1)`,
+:any:`notmuch-search-terms(7)`,
+:any:`notmuch-show(1)`,
+:any:`notmuch-tag(1)`
diff --git a/doc/man1/notmuch-compact.rst b/doc/man1/notmuch-compact.rst
index b05593ec2213..85f611bf44e5 100644
--- a/doc/man1/notmuch-compact.rst
+++ b/doc/man1/notmuch-compact.rst
@@ -1,3 +1,5 @@
+.. _notmuch-compact(1):
+
 ===
 notmuch-compact
 ===
@@ -46,15 +48,15 @@ of notmuch.
 SEE ALSO
 
 
-**notmuch(1)**,
-**notmuch-count(1)**,
-**notmuch-dump(1)**,
-**notmuch-hooks(5)**,
-**notmuch-insert(1)**,
-**notmuch-new(1)**,
-**notmuch-reply(1)**,
-**notmuch-restore(1)**,
-**notmuch-search(1)**,
-**notmuch-search-terms(7)**,
-**notmuch-show(1)**,
-**notmuch-tag(1)**
+:any:`notmuch(1)`,
+:any:`notmuch-count(1)`,
+:any:`notmuch-dump(1)`,
+:any:`notmuch-hooks(5)`,
+:any:`notmuch-insert(1)`,
+:any:`notmuch-new(1)`,
+:any:`notmuch-reply(1)`,
+:any:`notmuch-restore(1)`,
+:any:`notmuch-search(1)`,
+:any:`notmuch-search-terms(7)`,
+:any:`notmuch-show(1)`,
+:any:`notmuch-tag(1)`
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 75c59ff97188..5c980a8a4bb8 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -1,3 +1,5 @@
+.. _notmuch-config(1):
+
 ==
 notmuch-config
 ==
@@ -76,7 +78,7 @@ paths are presumed relative to `$HOME` for items in section
 **database.hook_dir**
 
 Directory containing hooks run by notmuch commands. See
-**notmuch-hooks(5)**.
+:any:`notmuch-hooks(5)`.
 
 **user.name**
 Your full name.
@@ -103,7 +105,7 @@ paths are presumed relative to `$HOME` for items in section
 
 

[PATCH 1/5] doc: use manpage role references to external man pages

[Jani Nikula]

Using manpage role references generates helpful links in html
documentation, while retaining the same boldface style in the man
pages.

The external man page site is configurable. The Debian manpage site
seems like a good fit for Notmuch.
---
 doc/conf.py   |  5 +
 doc/man1/notmuch-address.rst  |  4 ++--
 doc/man1/notmuch-dump.rst | 16 
 doc/man1/notmuch-emacs-mua.rst| 15 +--
 doc/man1/notmuch-reply.rst|  2 +-
 doc/man1/notmuch-restore.rst  |  6 +++---
 doc/man1/notmuch-search.rst   |  4 ++--
 doc/man1/notmuch-show.rst |  2 +-
 doc/man1/notmuch.rst  |  5 +++--
 doc/man7/notmuch-search-terms.rst |  2 +-
 10 files changed, 35 insertions(+), 26 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index d0f7f66ce83e..4a4a34212cc3 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -80,6 +80,11 @@ htmlhelp_basename = 'notmuchdoc'
 # Despite the name, this actually affects manual pages as well.
 html_use_smartypants = False
 
+# See:
+# - 
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-manpages_url
+# - https://manpages.debian.org/
+manpages_url = 'https://manpages.debian.org/{page}.{section}.html'
+
 # -- Options for manual page output ---
 
 # One entry per manual page. List of tuples
diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index 2a7df6f0d80f..9193aefe1db6 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -21,8 +21,8 @@ Supported options for **address** include
 ``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**)
 Presents the results in either JSON, S-Expressions, newline
 character separated plain-text (default), or null character
-separated plain-text (compatible with **xargs(1)** -0 option where
-available).
+separated plain-text (compatible with :manpage:`xargs(1)` -0
+option where available).
 
 ``--format-version=N``
 Use the specified structured output format version. This is
diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst
index ec6335b2febc..0a7a23876cf3 100644
--- a/doc/man1/notmuch-dump.rst
+++ b/doc/man1/notmuch-dump.rst
@@ -27,7 +27,7 @@ the remaining arguments are search terms.
 Supported options for **dump** include
 
 ``--gzip``
-Compress the output in a format compatible with **gzip(1)**.
+Compress the output in a format compatible with :manpage:`gzip(1)`.
 
 ``--format=(sup|batch-tag)``
 Notmuch restore supports two plain text dump formats, both with
@@ -36,8 +36,8 @@ Supported options for **dump** include
 **batch-tag**
 The default **batch-tag** dump format is intended to more
 robust against malformed message-ids and tags containing
-whitespace or non-\ **ascii(7)** characters. Each line has the
-form::
+whitespace or non-\ :manpage:`ascii(7)` characters. Each line
+has the form::
 
  +<*encoded-tag*\ > +<*encoded-tag*\ > ... -- id:<*quoted-message-id*\ 
>
 
@@ -54,11 +54,11 @@ Supported options for **dump** include
 
 **sup**
 The **sup** dump file format is specifically chosen to be
-compatible with the format of files produced by sup-dump. So
-if you've previously been using sup for mail, then the
-**notmuch restore** command provides you a way to import all
-of your tags (or labels as sup calls them). Each line has the
-following form::
+compatible with the format of files produced by
+:manpage:`sup-dump(1)`. So if you've previously been using sup
+for mail, then the **notmuch restore** command provides you a
+way to import all of your tags (or labels as sup calls
+them). Each line has the following form::
 
   <*message-id*\ > **(** <*tag*\ > ... **)**
 
diff --git a/doc/man1/notmuch-emacs-mua.rst b/doc/man1/notmuch-emacs-mua.rst
index a0476136f503..a1e65eddd9f1 100644
--- a/doc/man1/notmuch-emacs-mua.rst
+++ b/doc/man1/notmuch-emacs-mua.rst
@@ -41,8 +41,9 @@ Supported options for **emacs-mua** include
 Even if a window system is available, use the current terminal.
 
 ``--client``
-Use **emacsclient**, rather than **emacs**. For **emacsclient** to
-work, you need an already running Emacs with a server, or use
+Use :manpage:`emacsclient(1)`, rather than
+:manpage:`emacs(1)`. For :manpage:`emacsclient(1)` to work, you
+need an already running Emacs with a server, or use
 ``--auto-daemon``.
 
 ``--auto-daemon``
@@ -60,9 +61,9 @@ Supported options for **emacs-mua** include
 Output the resulting elisp to stdout instead of evaluating it.
 
 The supported positional parameters and short options are a compatible
-subset of the **mutt** MUA command-line options. The options and
-positional parameters modifying the message can't be combined with the
-mailto: URL.
+subset of the :manpage:`mutt(1)` MUA command-line 

[PATCH 0/5] doc: man page cleanup

[Jani Nikula]

I saw [1] and decided to give it some love. :)

Mostly this is all about adding html cross-references all over the place
while trying to keep the roff man pages roughly the same. Also updating
the man page rst becomes easier by setting a clean example.

BR,
Jani.



[1] https://notmuchmail.org/doc/latest/


Jani Nikula (5):
  doc: use manpage role references to external man pages
  doc: cross-reference notmuch man pages with actual links
  doc: use envvar directive and role for environment variables
  doc: use program and option directives to document options
  doc: example command-line option reference

 doc/conf.py   |   5 +
 doc/man1/notmuch-address.rst  | 204 
 doc/man1/notmuch-compact.rst  |  48 ++--
 doc/man1/notmuch-config.rst   | 147 ++--
 doc/man1/notmuch-count.rst|  87 ---
 doc/man1/notmuch-dump.rst | 176 +++---
 doc/man1/notmuch-emacs-mua.rst| 101 
 doc/man1/notmuch-insert.rst   | 128 +-
 doc/man1/notmuch-new.rst  | 102 
 doc/man1/notmuch-reindex.rst  |  71 +++---
 doc/man1/notmuch-reply.rst| 121 +-
 doc/man1/notmuch-restore.rst  | 154 ++--
 doc/man1/notmuch-search.rst   | 264 +++--
 doc/man1/notmuch-show.rst | 380 --
 doc/man1/notmuch-tag.rst  |  63 ++---
 doc/man1/notmuch.rst  | 139 ++-
 doc/man5/notmuch-hooks.rst|  54 +++--
 doc/man7/notmuch-properties.rst   |  36 +--
 doc/man7/notmuch-search-terms.rst |  50 ++--
 19 files changed, 1245 insertions(+), 1085 deletions(-)

-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] emacs: update default mailing list archives for stashing links

[Jani Nikula]

Gmane web interface is long gone, remove it. Make MARC the new
default. Update LKML to Lore, where it already redirects anyway. Also
add Notmuch web archive.

---

Note: I observed M-x customize-variable RET
notmuch-show-stash-mlarchive-link-default RET does not take user
customized notmuch-show-stash-mlarchive-link-alist into account. It
should, but I don't know how to fix that.
---
 emacs/notmuch-show.el | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index ba93febb34ff..a415a95c25bf 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -189,10 +189,10 @@ each attachment handler is logged in buffers with names 
beginning
 ;;; Options
 
 (defcustom notmuch-show-stash-mlarchive-link-alist
-  '(("Gmane" . "https://mid.gmane.org/;)
-("MARC" . "https://marc.info/?i=;)
+  '(("MARC" . "https://marc.info/?i=;)
 ("Mail Archive, The" . "https://mid.mail-archive.com/;)
-("LKML" . "https://lkml.kernel.org/r/;)
+("Lore" . "https://lore.kernel.org/r/;)
+("Notmuch" . "https://nmbug.notmuchmail.org/nmweb/show/;)
 ;; FIXME: can these services be searched by `Message-Id' ?
 ;; ("MarkMail" . "http://markmail.org/;)
 ;; ("Nabble" . "http://nabble.com/;)
@@ -217,7 +217,7 @@ return the ML archive reference URI."
 (function :tag "Function returning the URL")))
   :group 'notmuch-show)
 
-(defcustom notmuch-show-stash-mlarchive-link-default "Gmane"
+(defcustom notmuch-show-stash-mlarchive-link-default "MARC"
   "Default Mailing List Archive to use when stashing links.
 
 This is used when `notmuch-show-stash-mlarchive-link' isn't
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH] doc: fix man page build for Sphinx 4.x

[Jani Nikula]

On Tue, 18 May 2021, David Bremner  wrote:
> Jani Nikula  writes:
>
>>
>> The Sphinx documentation on this [1] is confusing, and has the change
>> backwards. Git history says the default changed from False to True.
>>
>> [1] 
>> https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
>
> Maybe the docs were corrected?
>
> Changed in version 4.0: The default is changed to False from True.
 ^^   

Were you tricked by the funny word order? I know I was at first. :)

I looked deeper into the rabbit hole, and it really is a mess for such a
small change. They first added the config to create directories man/1
etc. in 3.3. Then they made it the default in 4.0. Then they realized
making it the default was a mistake, and are going to roll it back in
upcoming v4.0.2. Then they realized the directories should be named
man/man1 (not man/1) etc. and are going to change that in upcoming v4.1.

The patch at hand fixes the issue for v4.0.0 and v4.0.1 (the current
latest release). Are those worth catering for, I don't know. Up to you.

Actually using the feature would be really nice to avoid doing it
manually in the makefile, but then that would require Sphinx v3.3 and
having to take into account the directory name change in v4.1.

*facepalm*


BR,
Jani.


Some references:

https://github.com/sphinx-doc/sphinx/commit/178c05b0597e144fdc2117c7b46ac15723a290e8
https://github.com/sphinx-doc/sphinx/commit/a843fd3db8b588be93b576049a8a291bc4b3ec9d
https://github.com/sphinx-doc/sphinx/issues/7996
https://github.com/sphinx-doc/sphinx/issues/9217
https://github.com/sphinx-doc/sphinx/pull/9232
https://github.com/sphinx-doc/sphinx/pull/9231
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] doc: fix man page build for Sphinx 4.x

[Jani Nikula]

Sphinx 4.0 changed the default value of man_make_section_directory
from False to True. We create the section directories and move the
files manually, so fix the immediate man build failure by disabling
the feature.

The Sphinx documentation on this [1] is confusing, and has the change
backwards. Git history says the default changed from False to True.

[1] 
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-man_make_section_directory
---
 doc/conf.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/conf.py b/doc/conf.py
index 1a5c217bb91e..d0f7f66ce83e 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -87,6 +87,8 @@ html_use_smartypants = False
 
 notmuch_authors = u'Carl Worth and many others'
 
+man_make_section_directory = False
+
 man_pages = [
 ('man1/notmuch', 'notmuch',
  u'thread-based email index, search, and tagging',
-- 
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: What do people use for calendar invites?

[Jani Nikula]

On Wed, 20 Jan 2021, David Mazieres  
wrote:
> Calendar invites and the text/calendar mime type seem to be getting
> increasingly important.  I asked about this five years ago and didn't
> get a good response, so I apologize for the repeat question, but I'm
> wondering if anything has changed since then.
>
> If you have found a good solution for integrating your calendar with
> notmuchmail, would you mind sharing what you are doing?  Some specific
> questions:
>
>   * What's a good way to generate text/calendar attachments from within
> notmuch (especially the emacs interface, which I use)?
>
>   * What's a good way to apply text/calendar attachments to a calendar
> from within notmuch?
>
>   * For those of us who hate gmail and love notmuch, but are still stuck
> using and hating google calendar, what alternative solutions should
> we be considering?

TL;DR I don't have a solution yet.

---

I've had this draft idea for integrating calendar support for
notmuch-emacs for a long time now. You'd have a command-line tool for
actually managing the calendar, and some elisp glue code to integrate
that to notmuch-emacs, as well as with something to pretty display of
the calendar. (I'm personally not a fan of org-mode, so I pretty much
ruled that out from the start for anything I'd use myself. YMMV.)

I got as far as writing the beginnings of a notmuch-like command-line
tool, "late", for calendar search [1]. It's actually usable for
searching, but it doesn't do much else. I'd like it to be able to
generate calendar invites and replies to them, in a way that's easy to
use from notmuch-emacs.

For the display of calendar information in emacs, I thought calfw looked
pretty [2]. It does seem to be a bit abandoned, unfortunately, and I
haven't checked the status lately. It wasn't too hard to make
notmuch-emacs display calender information using calfw, but to make it
more useful, you'd have to take the calendar in the email, combine that
with the user's calendar using the cli tool, and then feed it to calfw
to show the calendar event in context. And add bindings to actually do
stuff instead of just view.

I ran out of steam and time with this a bit, and anyway I don't have the
elisp skills for integrating this nicely to notmuch-emacs.

So the project is, uh, late. :p

BR,
Jani.


[1] https://github.com/jnikula/late
[2] https://github.com/kiwanami/emacs-calfw
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [RFC PATCH] build: add meson build system

[Jani Nikula]

On Sun, 12 Jan 2020, David Bremner  wrote:
> Jani Nikula  writes:
>
>> This is a draft patch adding basic configure, build and test support
>> for the binaries. Everything else is left out for now. It would be a
>> considerable amount of work to convert everything, and I don't expect
>> it to be possible in one go anyway. If there's interest in adding
>> meson support, it would have to happen gradually, side-by-side with
>> the current system, with a reasonably long transition period. But
>> there's no point in going beyond the patch at hand if folks decide the
>> focus should remain on the current system.
>
> Personally I think the idea is worth pursuing, but I admit I don't have
> much experience with meson/ninja.  How much churn can we expect from
> meson changes? It seems there is something like one meson point release
> per month.

I build projects with meson regularly, but I don't develop the build
files on a regular basis. So take this with a grain of salt. But I think
overall the experience has been pleasant. They do keep adding features
all the time, but AFAIK the version currently in e.g. Debian stable is
usable, and the new features don't break existing stuff.

If you want to hold back to an older version, you might have to put more
effort to things that are easier in newer versions - but the solution
should work in all versions. Any churn would be caused by taking
advantage of the new features.

> Are the tests supposed to be working fully in this version? When I run
>
> % meson && cd build && ninja && ninja test
>
> I get failures in T010, T351, T356, T357, T391, T395, and T710.  At a
> glance it looks like mainly out-of-tree related problems to finding
> e.g. json_check_nodes.py and message-id-parse. It also looks like a few
> variables like TEST_RUBY and NOTMUCH_HAVE_PYTHON3_CFFI are not being
> set.

I get those failures too, as I haven't bothered with debugging in more
detail yet. I just brought this to a point that mostly works.

If we decide we can start merging meson support, it's going to be
experimental and not fully featured initially. I think we'd have to
decide what's "good enough". Obviously I think the tests must all pass,
either by fixing the issues or gracefully disabling features. For
example, I don't currently build any of notmuch-emacs or documentation -
they will require custom build rules, while all the C and C++ work out
of the box.

I think mostly meson support can be added in parallel to the existing
system. Some things might have to be fixed for both, e.g. meson only
works with out-of-tree builds. The annoying part will be the transition
period, where you have to maintain both. And if we decide we'll want
both ad infinitum, it'll be an annoyance ad infinitum... ;)

BR,
Jani.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[RFC PATCH] build: add meson build system

[Jani Nikula]

Add initial support for configuring and building notmuch using the
Meson Build System [1].

The "home-brew" configure script in notmuch has grown rather
complicated, although the goal was originally to avoid the complexity
of using autotools. Since then, meson has emerged as a viable option
for replacing autotools and make.

The benefits compared to the current configure are most evident in the
feature tests. Many things that require tens of lines of shell and/or
C feature test code are simple one-liners in meson. Some
self-explanatory examples:

 - dep_gmime = dependency('gmime-3.0', version: '>=3.0.3')
 - cmd_python = find_program('python3', 'python', 'python2')
 - have_d_type = cc.has_member('struct dirent', 'd_type',
   prefix: '#include ')

Additionally, meson provides basic install targets automatically, as
well as a test runner with built-in parallelization.

This is a draft patch adding basic configure, build and test support
for the binaries. Everything else is left out for now. It would be a
considerable amount of work to convert everything, and I don't expect
it to be possible in one go anyway. If there's interest in adding
meson support, it would have to happen gradually, side-by-side with
the current system, with a reasonably long transition period. But
there's no point in going beyond the patch at hand if folks decide the
focus should remain on the current system.

[1] https://mesonbuild.com/
---
 compat/config.h.in|  11 +++
 compat/meson.build| 169 ++
 lib/meson.build   |  36 
 meson.build   |  77 
 parse-time-string/meson.build |   4 +
 sh.config.in  |  31 +++
 test/meson.build  | 133 ++
 util/meson.build  |  19 
 8 files changed, 480 insertions(+)
 create mode 100644 compat/config.h.in
 create mode 100644 compat/meson.build
 create mode 100644 lib/meson.build
 create mode 100644 meson.build
 create mode 100644 parse-time-string/meson.build
 create mode 100644 sh.config.in
 create mode 100644 test/meson.build
 create mode 100644 util/meson.build

diff --git a/compat/config.h.in b/compat/config.h.in
new file mode 100644
index ..22269a84c8a4
--- /dev/null
+++ b/compat/config.h.in
@@ -0,0 +1,11 @@
+#define HAVE_CANONICALIZE_FILE_NAME @HAVE_CANONICALIZE_FILE_NAME@
+#define HAVE_D_TYPE @HAVE_D_TYPE@
+#define HAVE_GETLINE @HAVE_GETLINE@
+#define HAVE_STRCASESTR @HAVE_STRCASESTR@
+#define HAVE_STRSEP @HAVE_STRSEP@
+#define HAVE_TIMEGM @HAVE_TIMEGM@
+#define HAVE_XAPIAN_COMPACT @HAVE_XAPIAN_COMPACT@
+#define HAVE_XAPIAN_DB_RETRY_LOCK @HAVE_XAPIAN_DB_RETRY_LOCK@
+#define HAVE_XAPIAN_FIELD_PROCESSOR @HAVE_XAPIAN_FIELD_PROCESSOR@
+#define STD_ASCTIME @STD_ASCTIME@
+#define STD_GETPWUID @STD_GETPWUID@
diff --git a/compat/meson.build b/compat/meson.build
new file mode 100644
index ..ed79bd2ec83c
--- /dev/null
+++ b/compat/meson.build
@@ -0,0 +1,169 @@
+notmuch_compat_srcs = [
+]
+
+config = configuration_data()
+
+config.set_quoted('NOTMUCH_SRCDIR', meson.source_root())
+
+# FIXME: this differs from the legacy build
+config.set('PLATFORM', build_machine.system())
+
+cc = meson.get_compiler('c')
+cxx = meson.get_compiler('cpp')
+
+have_canonicalize_file_name = cc.has_function('canonicalize_file_name',
+ prefix: '#include ',
+ args: '-D_GNU_SOURCE')
+config.set10('HAVE_CANONICALIZE_FILE_NAME', have_canonicalize_file_name)
+if not have_canonicalize_file_name
+   notmuch_compat_srcs += 'canonicalize_file_name.c'
+endif
+
+have_getline = cc.has_function('getline', prefix: '''#include 
+#include 
+''')
+config.set10('HAVE_GETLINE', have_getline)
+if not have_getline
+   notmuch_compat_srcs += [ 'getline.c', 'getdelim.c' ]
+endif
+
+have_strcasestr = cc.has_function('strcasestr', prefix: '#include ',
+ args: '-D_GNU_SOURCE')
+config.set10('HAVE_STRCASESTR', have_strcasestr)
+if not have_strcasestr
+   notmuch_compat_srcs += 'strcasestr.c'
+endif
+
+have_strsep = cc.has_function('strsep', prefix: '#include ')
+config.set10('HAVE_STRSEP', have_strsep)
+if not have_strsep
+   notmuch_compat_srcs += 'strsep.c'
+endif
+
+have_timegm = cc.has_function('timegm', prefix: '#include ')
+config.set10('HAVE_TIMEGM', have_timegm)
+if not have_timegm
+   notmuch_compat_srcs += 'timegm.c'
+endif
+
+have_d_type = cc.has_member('struct dirent', 'd_type',
+   prefix: '#include ')
+config.set10('HAVE_D_TYPE', have_d_type)
+
+getpwuid_code = '''
+#include 
+#include 
+int main()
+{
+   struct passwd passwd, *ignored;
+   getpwuid_r (0, , NULL, 0, );
+}
+'''
+std_getpwuid = cc.compiles(getpwuid_code, name: 'standard getpwuid_r')
+config.set10('STD_GETPWUID', std_getpwuid)
+
+asctime_code = '''
+#include 
+#include 
+int main()
+{
+   

[PATCH 1/2] doc: pass specified configure options as tags to Sphinx

[Jani Nikula]

Add generic support for passing enabled configure options as tags to
Sphinx. Add TAGS make variable to specify the configure options, and
use the sphinx-build -t option to add the options equal to 1 as tags.

The tags may be used in conf.py to conditionally adjust the
configuration [1], and to conditionally include content blocks in rst
files using the Sphinx "only" directive [2].

Initially, convert WITH_EMACS to a tag.

[1] https://www.sphinx-doc.org/en/master/usage/configuration.html#conf-tags
[2] 
https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#tags
---
 doc/Makefile.local | 9 +++--
 doc/conf.py| 2 +-
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index b4e0c9558101..c78aea49a108 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -4,11 +4,16 @@ dir := doc
 
 # You can set these variables from the command line.
 SPHINXOPTS:= -q
-SPHINXBUILD   = WITH_EMACS=${WITH_EMACS} sphinx-build
+SPHINXBUILD   = sphinx-build
 DOCBUILDDIR  := $(dir)/_build
 
+# Configure options to be added as Sphinx tags.
+# Add "-t " for each make variable in TAGS that equals 1.
+TAGS := WITH_EMACS
+TAGOPTS := $(patsubst %=1,-t %,$(filter %=1,$(foreach 
tag,$(TAGS),$(tag)=$(value $(tag)
+
 # Internal variables.
-ALLSPHINXOPTS   := $(SPHINXOPTS) $(srcdir)/$(dir)
+ALLSPHINXOPTS   := $(SPHINXOPTS) $(TAGOPTS) $(srcdir)/$(dir)
 APIMAN := $(DOCBUILDDIR)/man/man3/notmuch.3
 DOXYFILE   := $(srcdir)/$(dir)/doxygen.cfg
 
diff --git a/doc/conf.py b/doc/conf.py
index fc9738ff092a..2ee063d77f10 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -32,7 +32,7 @@ exclude_patterns = ['_build']
 # If we don't have emacs (or the user configured --without-emacs),
 # don't build the notmuch-emacs docs, as they need emacs to generate
 # the docstring include files
-if os.environ.get('WITH_EMACS') != '1':
+if not tags.has('WITH_EMACS'):
 exclude_patterns.append('notmuch-emacs.rst')
 
 # The name of the Pygments (syntax highlighting) style to use.
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] doc: add a hint about field processor support availability

[Jani Nikula]

In most cases the documentation and the notmuch binary will be built
on the same machine and the same dependencies. Hint at the field
processor support availability in the man page based on the
documentation build host.
---
 doc/Makefile.local| 2 +-
 doc/man7/notmuch-search-terms.rst | 8 
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index c78aea49a108..cb01306c0b7c 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -9,7 +9,7 @@ DOCBUILDDIR  := $(dir)/_build
 
 # Configure options to be added as Sphinx tags.
 # Add "-t " for each make variable in TAGS that equals 1.
-TAGS := WITH_EMACS
+TAGS := WITH_EMACS HAVE_XAPIAN_FIELD_PROCESSOR
 TAGOPTS := $(patsubst %=1,-t %,$(filter %=1,$(foreach 
tag,$(TAGS),$(tag)=$(value $(tag)
 
 # Internal variables.
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1dd2dc5813f5..9e3b9398c58d 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -457,6 +457,14 @@ notmuch was built against a sufficiently recent version of 
Xapian by running
 
   % notmuch config get built_with.field_processor
 
+.. only:: HAVE_XAPIAN_FIELD_PROCESSOR
+
+   The documentation was built on a host with field processor support.
+
+.. only:: not HAVE_XAPIAN_FIELD_PROCESSOR
+
+   The documentation was built on a host without field processor support.
+
 Currently the following features require field processor support:
 
 - non-range date queries, e.g. "date:today"
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Inconsistent query results

[Jani Nikula]

On Sun, 05 Jan 2020, Nicolas Bock  wrote:
> When I run the following query I get no results:
>
> $ notmuch search --output=files folder:"mail_account/Deleted
> Items" and not tag:deleted
>
> When I run that same query in emacs/notmuch I find one file:
>
> query: folder:"mail_account/Deleted Items" and not tag:deleted
> file: /home/nbock/Mail/mail_account/Deleted
> Items/cur/5d5aca30-2ef6-11ea-9e47-28f10e4fd9ec,U=984:2,S
>
> Am I missing something?

Yes, quotes. ;)

Emacs does the quoting for you, but in shell you need to ensure the
double quotes go all the way to Xapian. Try wrapping the folder search
in single quotes: 'folder:"mail_account/Deleted Items"'.

You can use the NOTMUCH_DEBUG_QUERY=1 environment variable to debug
queries. You can see the difference.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Fix format_headers_sprinter to return all headers

[Jani Nikula]

On Sat, 09 Nov 2019, David Bremner  wrote:
> Jani Nikula  writes:
>
>> On Sat, 09 Nov 2019, Johan Parin  wrote:
>>> Modify format_headers_sprinter so that it returns all headers in the
>>> sexp, instead of a fixed set of headers.
>>
>> I have to deal with plenty of long threads that already take a very long
>> time to open in notmuch-emacs. How's this going to impact the emacs
>> interface performance? For example the patch mail I'm replying to has
>> more header content than body content. There are tons of headers that I
>> can't imagine being useful to anyone.
>
>  Can we benchmark this somehow? I think we have suitable threads in the
>  LKML corpus, but the challenge is timing emacs in some sensible way.

notmuch-emacs performance tests? :o

I can of course try this stuff to get a gut feeling, but it's subjective
and doesn't really scale. :)

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Fix format_headers_sprinter to return all headers

[Jani Nikula]

On Sat, 09 Nov 2019, Johan Parin  wrote:
> Jani Nikula  writes:
>
>> On Sat, 09 Nov 2019, Johan Parin  wrote:
>>> Modify format_headers_sprinter so that it returns all headers in the
>>> sexp, instead of a fixed set of headers.
>>
>> I have to deal with plenty of long threads that already take a very
>> long time to open in notmuch-emacs. How's this going to impact the
>> emacs interface performance? For example the patch mail I'm replying
>> to has more header content than body content. There are tons of
>> headers that I can't imagine being useful to anyone.
>>
>
> How long are these threads may I ask? Would it be possible for you to
> apply the patch and see if you see a noticable difference?

I'll need to try this with my work email which is the bigger problem.

> I have a hard time seeing that the C code would cause any noticable
> performance impact, but perhaps the elisp. But my general understanding
> of the code base is low, so I may be wrong.

It's indeed the lisp part that I'm worried about.

> I guess it's in the following type of code the performance concern lies?
>
> (mapc (lambda (header)
>   (let* ((header-symbol (intern (concat ":" header)))
>  (header-value (plist-get headers header-symbol)))
> (if (and header-value
>  (not (string-equal "" header-value)))
> (notmuch-show-insert-header header header-value
> notmuch-message-headers)
>
> Don't know how efficient plists are in general.
>
>> I'm wondering if the right thing to do would be to make it possible to
>> specify which additional headers to include in notmuch-show
>> output. This would be based on notmuch-message-headers. We already
>> have options to include/exclude body content and html parts for much
>> the same reasons. (Though the command-line arguments for those are
>> incoherent at best.)
>>
>
> This would be possible of course.
>
> Another option I guess is to hard code some extra "interesting"
> headers. But what is interesting is subjective and it is very nice to
> have the flexibility to control this is in elisp.
>
> mu4e has the customization variable `mu4e-view-fields', but it can only
> take on values from a restricted list, namely:
>
> :from :to :cc :subject :flags :date :maildir :mailing-list :tags
> :attachments :signature :decryption :user-agent
>
> So a compromise would be to restrict the returned headers to a list like
> this. This of course is if we think the potential performance issue is
> in the elisp.
>
> Personally I would be content for the moment to be able to see
> user-agent and x-mailer. But that's just me.

I failed to mention that I do think the feature itself would be useful
and welcome. Currently I just hit V and find the header in the raw
message...

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Fix format_headers_sprinter to return all headers

[Jani Nikula]

On Sat, 09 Nov 2019, Johan Parin  wrote:
> Modify format_headers_sprinter so that it returns all headers in the
> sexp, instead of a fixed set of headers.

I have to deal with plenty of long threads that already take a very long
time to open in notmuch-emacs. How's this going to impact the emacs
interface performance? For example the patch mail I'm replying to has
more header content than body content. There are tons of headers that I
can't imagine being useful to anyone.

I'm wondering if the right thing to do would be to make it possible to
specify which additional headers to include in notmuch-show output. This
would be based on notmuch-message-headers. We already have options to
include/exclude body content and html parts for much the same
reasons. (Though the command-line arguments for those are incoherent at
best.)

It is, of course, much more work than the patch at hand.

BR,
Jani.


>
> This is required in order for the elisp variable
>`notmuch-message-headers' to work.
>
> See this bug report:
>
>   https://notmuchmail.org/pipermail/notmuch/2017/026069.html
> ---
>  notmuch-show.c | 27 +++
>  1 file changed, 27 insertions(+)
>
> diff --git a/notmuch-show.c b/notmuch-show.c
> index 21792a57..9652ed09 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -205,6 +205,15 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
> *message,
>  char *recipients_string;
>  const char *reply_to_string;
>  void *local = talloc_new (sp);
> +GMimeHeaderList *header_list;
> +GMimeHeader *header;
> +int header_count;
> +/* Headers requiring some special treatment in formatting */
> +const char* special_headers[] = {"Subject", "From", "To", "Cc", "Bcc",
> +  "Reply-To", "In-reply-to", "References",
> +  "Date"};
> +const int special_header_count = sizeof(special_headers) / sizeof(char 
> *);
> +bool special;
>  
>  sp->begin_map (sp);
>  
> @@ -255,6 +264,24 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage 
> *message,
>   sp->string (sp, g_mime_message_get_date_string (sp, message));
>  }
>  
> +header_list  = g_mime_object_get_header_list (GMIME_OBJECT(message));
> +header_count = g_mime_header_list_get_count(header_list);
> +
> +for (int i = 0; i < header_count; i++) {
> +header = g_mime_header_list_get_header_at(header_list, i);
> + special = false;
> + for (int k = 0; k < special_header_count; k++)
> + if (!STRNCMP_LITERAL(g_mime_header_get_name(header),
> +  special_headers[k])) {
> + special = true;
> + break;
> + }
> + if (special)
> + continue;
> + sp->map_key (sp, g_mime_header_get_name(header));
> + sp->string (sp, g_mime_header_get_value(header));
> +}
> +
>  sp->end (sp);
>  talloc_free (local);
>  }
> -- 
> 2.21.0 (Apple Git-122)
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] doc: add support for adding configure options as Sphinx tags

[Jani Nikula]

Add the configure options specified in $(TAGS) that equal 1 as tags on
the Sphinx command line using the -t option. The tags may be used to
conditionally include documentation using the Sphinx "only" directive
[1].

As an example, indicate in the documentation whether the Xapian field
processor is likely to be available (assuming the notmuch binary was
built in the same environment as the documentation).

[1] 
http://www.sphinx-doc.org/en/stable/markup/misc.html#including-content-based-on-tags

---

This is something I wrote two years ago, ISTR David asking about
something like this on IRC, but I've already forgotten what it was
about...
---
 doc/Makefile.local| 7 ++-
 doc/man7/notmuch-search-terms.rst | 8 
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index bab3d0d286ce..95df9ae82b4f 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -7,8 +7,13 @@ SPHINXOPTS:= -q
 SPHINXBUILD   = sphinx-build
 DOCBUILDDIR  := $(dir)/_build
 
+# Configure options to be added as Sphinx tags.
+# Add "-t " for each make variable in TAGS that equals 1.
+TAGS := HAVE_XAPIAN_FIELD_PROCESSOR HAVE_XAPIAN_COMPACT
+TAGOPTS := $(patsubst %=1,-t %,$(filter %=1,$(foreach 
tag,$(TAGS),$(tag)=$(value $(tag)
+
 # Internal variables.
-ALLSPHINXOPTS   := -d $(DOCBUILDDIR)/doctrees $(SPHINXOPTS) $(srcdir)/$(dir)
+ALLSPHINXOPTS   := -d $(DOCBUILDDIR)/doctrees $(SPHINXOPTS) $(TAGOPTS) 
$(srcdir)/$(dir)
 APIMAN := $(DOCBUILDDIR)/man/man3/notmuch.3
 DOXYFILE   := $(srcdir)/$(dir)/doxygen.cfg
 
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index f7a39ceb9df4..4840f03743aa 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -451,6 +451,14 @@ notmuch was built against a sufficiently recent version of 
Xapian by running
 
   % notmuch config get built_with.field_processor
 
+.. only:: HAVE_XAPIAN_FIELD_PROCESSOR
+
+   The documentation was built on a host with field processor support.
+
+.. only:: not HAVE_XAPIAN_FIELD_PROCESSOR
+
+   The documentation was built on a host without field processor support.
+
 Currently the following features require field processor support:
 
 - non-range date queries, e.g. "date:today"
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/3] python: fix threads.__str__ automethod documentation

[Jani Nikula]

Indent the directive properly to attach it to Threads autoclass
documentation.

Fixes:

WARNING: don't know which module to import for autodocumenting
'__str__' (try placing a "module" or "currentmodule" directive in the
document, or giving an explicit module name)
---
 bindings/python/docs/source/threads.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bindings/python/docs/source/threads.rst 
b/bindings/python/docs/source/threads.rst
index 4324ac82a389..46ce5be5412a 100644
--- a/bindings/python/docs/source/threads.rst
+++ b/bindings/python/docs/source/threads.rst
@@ -11,4 +11,4 @@
   iterator and broke list(Threads()). Use `len(list(msgs))`
   instead.
 
-.. automethod:: __str__
+   .. automethod:: __str__
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] python: fix documentation title underline

[Jani Nikula]

Fix documentation build sphinx warning:

filesystem.rst:18: WARNING: Title underline too short.
---
 bindings/python/docs/source/filesystem.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bindings/python/docs/source/filesystem.rst 
b/bindings/python/docs/source/filesystem.rst
index 558c93de592a..13fe11946c46 100644
--- a/bindings/python/docs/source/filesystem.rst
+++ b/bindings/python/docs/source/filesystem.rst
@@ -15,7 +15,7 @@ Files and directories
   instead.
 
 :class:`Directory` -- A directory entry in the database
---
+---
 
 .. autoclass:: Directory
 
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/3] python: fix documentation build with python 3.7

[Jani Nikula]

From: Jani Nikula 

The simplistic mocking in conf.py falls short on python 3.7. Just use
unittest.mock instead.

Fixes:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/config.py", line 368, in 
eval_config_file
execfile_(filename, namespace)
  File "/usr/lib/python3/dist-packages/sphinx/util/pycompat.py", line 150, in 
execfile_
exec_(code, _globals)
  File "/path/to/notmuch/bindings/python/docs/source/conf.py", line 39, in 

from notmuch import __VERSION__,__AUTHOR__
  File "/path/to/notmuch/bindings/python/notmuch/__init__.py", line 54, in 

from .database import Database
  File "/path/to/notmuch/bindings/python/notmuch/database.py", line 25, in 

from .globals import (
  File "/path/to/notmuch/bindings/python/notmuch/globals.py", line 48, in 

class NotmuchDatabaseS(Structure):
TypeError: __mro_entries__ must return a tuple
---
 bindings/python/docs/source/conf.py | 13 ++---
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/bindings/python/docs/source/conf.py 
b/bindings/python/docs/source/conf.py
index 5b901c4ec4d8..8b43c5ca3f9f 100644
--- a/bindings/python/docs/source/conf.py
+++ b/bindings/python/docs/source/conf.py
@@ -13,22 +13,13 @@
 
 import sys, os
 
+from unittest.mock import Mock
+
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 sys.path.insert(0,os.path.abspath('../..'))
 
-class Mock(object):
-def __init__(self, *args, **kwargs):
-pass
-
-def __call__(self, *args, **kwargs):
-return Mock()
-
-@classmethod
-def __getattr__(self, name):
-return Mock() if name not in ('__file__', '__path__') else '/dev/null'
-
 MOCK_MODULES = [
 'ctypes',
 ]
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [Bug] Cannot build python3 bindings documentation of notmuch 0.28.1

[Jani Nikula]

On Wed, 06 Feb 2019, Dan Čermák  wrote:
> Hi list,
>
> the python3 bindings documentation seems to be broken:
>
> on OpenSUSE Tumbleweed and Fedora 29 (both have Python 3.7 and sphinx
> 1.7.6) I get this:

Looks like a change in Python 3.7 makes the ctypes mocking in
bindings/python/docs/source/conf.py fall short. If you just remove
'ctypes' from MOCK_MODULES, it'll work.

However, that would probably break the documentation build at
readthedocs.org, which won't have libnotmuch around.

Short term, I guess someone(tm) has to hack the mocking.

Long term, I'd love to see the python bindings Sphinx documentation
merged to the main Sphinx notmuch documentation, and have all of that
get automatically built and updated to notmuchmail.org.


BR,
Jani.


>
> $ cd bindings/python/docs/
> $ make dirhtml
> sphinx-build -b dirhtml -d build/doctrees   source build/dirhtml
> Running Sphinx v1.7.6
>
> Configuration error:
> There is a programable error in your configuration file:
>
> Traceback (most recent call last):
>   File "/usr/lib/python3.7/site-packages/sphinx/config.py", line 161, in 
> __init__
> execfile_(filename, config)
>   File "/usr/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 150, 
> in execfile_
> exec_(code, _globals)
>   File "conf.py", line 39, in 
> from notmuch import __VERSION__,__AUTHOR__
>   File 
> "/home/dan/packages/git.notmuchmail.org/git/notmuch/bindings/python/notmuch/__init__.py",
>  line 54, in 
> from .database import Database
>   File 
> "/home/dan/packages/git.notmuchmail.org/git/notmuch/bindings/python/notmuch/database.py",
>  line 25, in 
> from .globals import (
>   File 
> "/home/dan/packages/git.notmuchmail.org/git/notmuch/bindings/python/notmuch/globals.py",
>  line 48, in 
> class NotmuchDatabaseS(Structure):
> TypeError: __mro_entries__ must return a tuple
>
> make: *** [Makefile:38: dirhtml] Error 2
>
>
> Cheers,
>
> Dan
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 1/4] test: dynamically generate parser tests

[Jani Nikula]

Apologies for the noise, I sent the entire series to the wrong list. :(

BR,
Jani.

On Mon, 28 Jan 2019, Jani Nikula  wrote:
> It's impossible to have expected failures or other unittest decorators
> at subtest granularity. They only work at the test method level. On the
> other hand we don't want to be manually adding test methods when all of
> the tests are defined in terms of input files and expected results.
>
> Generate test methods dynamically from the input files, and assign to
> the test class. Running code at import time to do this is less than
> stellar, but it needs to be done early to have unittest test discovery
> find the methods.
>
> The alternative would be to add a load_tests protocol function [1], but
> that seems like more boilerplate. Can be added later as needed.
>
> Finally, one massive upside to this is the ability to run individual
> named testcases. For example, to test enum.c and typedef-enum.c, use:
>
> $ test/test_hawkmoth.py ParserTest.test_enum ParserTest.test_typedef_enum
>
> [1] https://docs.python.org/3/library/unittest.html#load-tests-protocol
> ---
>  test/test_hawkmoth.py | 26 +++---
>  test/testenv.py   | 29 +
>  2 files changed, 36 insertions(+), 19 deletions(-)
>
> diff --git a/test/test_hawkmoth.py b/test/test_hawkmoth.py
> index 1fe02efc004d..75eebbe35eef 100755
> --- a/test/test_hawkmoth.py
> +++ b/test/test_hawkmoth.py
> @@ -8,28 +8,16 @@ import unittest
>  import testenv
>  from hawkmoth import hawkmoth
>  
> -class ParserTest(unittest.TestCase):
> -def _run_test(self, input_filename):
> -# sanity check
> -self.assertTrue(os.path.isfile(input_filename))
> -
> -options = testenv.get_testcase_options(input_filename)
> -output = hawkmoth.parse_to_string(input_filename, False, **options)
> -expected = testenv.read_file(input_filename, ext='stdout')
> +def _get_output(input_filename, **options):
> +return hawkmoth.parse_to_string(input_filename, False, **options)
>  
> -self.assertEqual(expected, output)
> +def _get_expected(input_filename, **options):
> +return testenv.read_file(input_filename, ext='stdout')
>  
> -def _run_dir(self, path):
> -# sanity check
> -self.assertTrue(os.path.isdir(path))
> -
> -with self.subTest(path=path):
> -for f in testenv.get_testcases(path):
> -with self.subTest(source=os.path.basename(f)):
> -self._run_test(f)
> +class ParserTest(unittest.TestCase):
> +pass
>  
> -def test_parser(self):
> -self._run_dir(testenv.testdir)
> +testenv.assign_test_methods(ParserTest, _get_output, _get_expected)
>  
>  if __name__ == '__main__':
>  unittest.main()
> diff --git a/test/testenv.py b/test/testenv.py
> index f026aead8c07..cc80ef2218ed 100644
> --- a/test/testenv.py
> +++ b/test/testenv.py
> @@ -3,6 +3,7 @@
>  
>  import sys
>  import os
> +import unittest
>  
>  testext = '.c'
>  testdir = os.path.dirname(os.path.abspath(__file__))
> @@ -10,6 +11,16 @@ rootdir = os.path.dirname(testdir)
>  
>  sys.path.insert(0, rootdir)
>  
> +def _testcase_name(testcase):
> +"""Convert a testcase filename into a test case identifier."""
> +name = os.path.splitext(os.path.basename(testcase))[0]
> +name = name.replace('-', '_')
> +name = 'test_{name}'.format(name=name)
> +
> +assert name.isidentifier()
> +
> +return name
> +
>  def get_testcases(path):
>  for f in sorted(os.listdir(path)):
>  if f.endswith(testext):
> @@ -52,3 +63,21 @@ def read_file(filename, **kwargs):
>  expected = file.read()
>  
>  return expected
> +
> +def _test_generator(get_output, get_expected, input_filename, **options):
> +"""Return a function that compares output/expected results on 
> input_filename."""
> +def test(self):
> +output = get_output(input_filename, **options)
> +expected = get_expected(input_filename, **options)
> +
> +self.assertEqual(expected, output)
> +
> +return test
> +
> +def assign_test_methods(cls, get_output, get_expected):
> +"""Assign test case functions to the given class."""
> +for f in get_testcases(testdir):
> +options = get_testcase_options(f)
> +method = _test_generator(get_output, get_expected, f, **options)
> +
> +setattr(cls, _testcase_name(f), method)
> -- 
> 2.20.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 4/4] test: use html builder for directive tests

[Jani Nikula]

Slower but does not lose information as much, providing more accurate
results. Switch to the basic template for speed.
---
 test/conf.py  | 2 +-
 test/test_cautodoc.py | 8 
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/test/conf.py b/test/conf.py
index 6d36327df085..d4c3d34c8b3b 100644
--- a/test/conf.py
+++ b/test/conf.py
@@ -75,7 +75,7 @@ pygments_style = None
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = 'basic'
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
diff --git a/test/test_cautodoc.py b/test/test_cautodoc.py
index 0a8e5bb2815f..ad54cae2e98e 100755
--- a/test/test_cautodoc.py
+++ b/test/test_cautodoc.py
@@ -9,7 +9,7 @@ import unittest
 import testenv
 from sphinx_testing import with_app
 
-@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='text')
+@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='html')
 def _get_output(input_filename, app, status, warning, **options):
 shutil.copyfile(input_filename,
 testenv.modify_filename(input_filename, dir=app.srcdir))
@@ -23,16 +23,16 @@ def _get_output(input_filename, app, status, warning, 
**options):
 
 app.build()
 
-return testenv.read_file(os.path.join(app.outdir, 'index.txt'))
+return testenv.read_file(os.path.join(app.outdir, 'index.html'))
 
-@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='text')
+@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='html')
 def _get_expected(input_filename, app, status, warning, **options):
 shutil.copyfile(testenv.modify_filename(input_filename, ext='stdout'),
 os.path.join(app.srcdir, 'index.rst'))
 
 app.build()
 
-return testenv.read_file(os.path.join(app.outdir, 'index.txt'))
+return testenv.read_file(os.path.join(app.outdir, 'index.html'))
 
 class DirectiveTest(unittest.TestCase):
 pass
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 3/4] test: add support for flagging expected failures in testcase options

[Jani Nikula]

Since our tests are dynamically created, we also need to decorate
expected failures dynamically. Use the testcase options file as the
source. Only pass known directive options to the directive in the
cautodoc test.

Add a meta test to verify this works, with result "OK (expected
failures=2)".
---
 test/meta-expected-failure.c   | 3 +++
 test/meta-expected-failure.options | 1 +
 test/meta-expected-failure.stdout  | 3 +++
 test/test_cautodoc.py  | 2 +-
 test/testenv.py| 8 
 5 files changed, 16 insertions(+), 1 deletion(-)
 create mode 100644 test/meta-expected-failure.c
 create mode 100644 test/meta-expected-failure.options
 create mode 100644 test/meta-expected-failure.stdout

diff --git a/test/meta-expected-failure.c b/test/meta-expected-failure.c
new file mode 100644
index ..18065136572f
--- /dev/null
+++ b/test/meta-expected-failure.c
@@ -0,0 +1,3 @@
+/**
+ * Meta test: This fails. Always.
+ */
diff --git a/test/meta-expected-failure.options 
b/test/meta-expected-failure.options
new file mode 100644
index ..68c905946fce
--- /dev/null
+++ b/test/meta-expected-failure.options
@@ -0,0 +1 @@
+test-expected-failure
diff --git a/test/meta-expected-failure.stdout 
b/test/meta-expected-failure.stdout
new file mode 100644
index ..dbd71dcb308f
--- /dev/null
+++ b/test/meta-expected-failure.stdout
@@ -0,0 +1,3 @@
+
+Meta test: This fails.
+
diff --git a/test/test_cautodoc.py b/test/test_cautodoc.py
index 848f2105a1da..0a8e5bb2815f 100755
--- a/test/test_cautodoc.py
+++ b/test/test_cautodoc.py
@@ -17,7 +17,7 @@ def _get_output(input_filename, app, status, warning, 
**options):
 with open(os.path.join(app.srcdir, 'index.rst'), 'w') as file:
 fmt = '.. c:autodoc:: {source}\n'
 file.write(fmt.format(source=os.path.basename(input_filename)))
-for key in options.keys():
+for key in [k for k in options.keys() if k in 
testenv.directive_options]:
 fmt = '   :{key}: {value}\n'
 file.write(fmt.format(key=key, value=options[key]))
 
diff --git a/test/testenv.py b/test/testenv.py
index cc80ef2218ed..b6842a81b375 100644
--- a/test/testenv.py
+++ b/test/testenv.py
@@ -26,6 +26,11 @@ def get_testcases(path):
 if f.endswith(testext):
 yield os.path.join(path, f)
 
+directive_options = [
+'compat',
+'clang',
+]
+
 def get_testcase_options(testcase):
 options_filename = modify_filename(testcase, ext='options')
 
@@ -80,4 +85,7 @@ def assign_test_methods(cls, get_output, get_expected):
 options = get_testcase_options(f)
 method = _test_generator(get_output, get_expected, f, **options)
 
+if options.get('test-expected-failure') is not None:
+method = unittest.expectedFailure(method)
+
 setattr(cls, _testcase_name(f), method)
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 2/4] test: dynamically generate directive tests

[Jani Nikula]

Similar to the parser test transition to dynamically generate test
methods, but with more test separation added. After this, the Sphinx
build is done independently for each test case, and separately for the
actual directive output and the expected output. This removes any
potential C domain interactions between test cases and expected/actual
inputs.

The change does mean the Sphinx build is run roughly 50x times per full
test run, at the current number of test cases. This is somewhat offset
by the ability to run individual directive test cases:

$ test/test_cautodoc.py DirectiveTest.test_example_10_macro
---
 test/{sphinx => }/conf.py |  0
 test/sphinx/index.rst | 20 -
 test/test_cautodoc.py | 61 ++-
 3 files changed, 21 insertions(+), 60 deletions(-)
 rename test/{sphinx => }/conf.py (100%)
 delete mode 100644 test/sphinx/index.rst

diff --git a/test/sphinx/conf.py b/test/conf.py
similarity index 100%
rename from test/sphinx/conf.py
rename to test/conf.py
diff --git a/test/sphinx/index.rst b/test/sphinx/index.rst
deleted file mode 100644
index 20404ce8ee5e..
--- a/test/sphinx/index.rst
+++ /dev/null
@@ -1,20 +0,0 @@
-.. Hawkmoth Test documentation master file, created by
-   sphinx-quickstart on Wed Dec 12 13:11:26 2018.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
-Welcome to Hawkmoth Test's documentation!
-=
-
-.. toctree::
-   :maxdepth: 2
-   :caption: Contents:
-
-
-
-Indices and tables
-==
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
diff --git a/test/test_cautodoc.py b/test/test_cautodoc.py
index 3fb98af8629d..848f2105a1da 100755
--- a/test/test_cautodoc.py
+++ b/test/test_cautodoc.py
@@ -9,54 +9,35 @@ import unittest
 import testenv
 from sphinx_testing import with_app
 
-class DirectiveTest(unittest.TestCase):
-
-def _setup_src(self, srcdir, testcase_in):
-testcase_out = testenv.modify_filename(testcase_in, dir=srcdir)
-
-# use the pre-generated rst as comparison data
-shutil.copyfile(testenv.modify_filename(testcase_in, ext='stdout'),
-testenv.modify_filename(testcase_out, 
ext='expected.rst'))
-
-# set up an rst file to run the extension
-shutil.copyfile(testcase_in, testcase_out)
-options = testenv.get_testcase_options(testcase_in)
+@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='text')
+def _get_output(input_filename, app, status, warning, **options):
+shutil.copyfile(input_filename,
+testenv.modify_filename(input_filename, dir=app.srcdir))
 
-with open(testenv.modify_filename(testcase_out, ext='output.rst'), 
'w') as file:
-fmt = '.. c:autodoc:: {source}\n'
-file.write(fmt.format(source=os.path.basename(testcase_out)))
-for key in options.keys():
-fmt = '   :{key}: {value}\n'
-file.write(fmt.format(key=key, value=options[key]))
+with open(os.path.join(app.srcdir, 'index.rst'), 'w') as file:
+fmt = '.. c:autodoc:: {source}\n'
+file.write(fmt.format(source=os.path.basename(input_filename)))
+for key in options.keys():
+fmt = '   :{key}: {value}\n'
+file.write(fmt.format(key=key, value=options[key]))
 
-def _check_out(self, outdir, testcase_in):
-testcase_out = testenv.modify_filename(testcase_in, dir=outdir)
+app.build()
 
-# compare output from the pre-generated rst against the output 
generated
-# by the extension
+return testenv.read_file(os.path.join(app.outdir, 'index.txt'))
 
-output = testenv.read_file(testenv.modify_filename(testcase_out,
-   ext='output.txt'))
+@with_app(confdir=testenv.testdir, create_new_srcdir=True, buildername='text')
+def _get_expected(input_filename, app, status, warning, **options):
+shutil.copyfile(testenv.modify_filename(input_filename, ext='stdout'),
+os.path.join(app.srcdir, 'index.rst'))
 
-expected = testenv.read_file(testenv.modify_filename(testcase_out,
- 
ext='expected.txt'))
+app.build()
 
-self.assertEqual(expected, output)
+return testenv.read_file(os.path.join(app.outdir, 'index.txt'))
 
-# Use copy_srcdir_to_tmpdir=False and outdir='some-dir' for debugging
-@with_app(srcdir=os.path.join(testenv.testdir, 'sphinx'),
-  buildername='text', copy_srcdir_to_tmpdir=True)
-def test_directive(self, app, status, warning):
-testcases = list(testenv.get_testcases(testenv.testdir))
-
-for f in testcases:
-self._setup_src(app.srcdir, f)
-
-app.build()
+class DirectiveTest(unittest.TestCase):
+pass
 
-for f in testcases:
- 

[PATCH v2 1/4] test: dynamically generate parser tests

[Jani Nikula]

It's impossible to have expected failures or other unittest decorators
at subtest granularity. They only work at the test method level. On the
other hand we don't want to be manually adding test methods when all of
the tests are defined in terms of input files and expected results.

Generate test methods dynamically from the input files, and assign to
the test class. Running code at import time to do this is less than
stellar, but it needs to be done early to have unittest test discovery
find the methods.

The alternative would be to add a load_tests protocol function [1], but
that seems like more boilerplate. Can be added later as needed.

Finally, one massive upside to this is the ability to run individual
named testcases. For example, to test enum.c and typedef-enum.c, use:

$ test/test_hawkmoth.py ParserTest.test_enum ParserTest.test_typedef_enum

[1] https://docs.python.org/3/library/unittest.html#load-tests-protocol
---
 test/test_hawkmoth.py | 26 +++---
 test/testenv.py   | 29 +
 2 files changed, 36 insertions(+), 19 deletions(-)

diff --git a/test/test_hawkmoth.py b/test/test_hawkmoth.py
index 1fe02efc004d..75eebbe35eef 100755
--- a/test/test_hawkmoth.py
+++ b/test/test_hawkmoth.py
@@ -8,28 +8,16 @@ import unittest
 import testenv
 from hawkmoth import hawkmoth
 
-class ParserTest(unittest.TestCase):
-def _run_test(self, input_filename):
-# sanity check
-self.assertTrue(os.path.isfile(input_filename))
-
-options = testenv.get_testcase_options(input_filename)
-output = hawkmoth.parse_to_string(input_filename, False, **options)
-expected = testenv.read_file(input_filename, ext='stdout')
+def _get_output(input_filename, **options):
+return hawkmoth.parse_to_string(input_filename, False, **options)
 
-self.assertEqual(expected, output)
+def _get_expected(input_filename, **options):
+return testenv.read_file(input_filename, ext='stdout')
 
-def _run_dir(self, path):
-# sanity check
-self.assertTrue(os.path.isdir(path))
-
-with self.subTest(path=path):
-for f in testenv.get_testcases(path):
-with self.subTest(source=os.path.basename(f)):
-self._run_test(f)
+class ParserTest(unittest.TestCase):
+pass
 
-def test_parser(self):
-self._run_dir(testenv.testdir)
+testenv.assign_test_methods(ParserTest, _get_output, _get_expected)
 
 if __name__ == '__main__':
 unittest.main()
diff --git a/test/testenv.py b/test/testenv.py
index f026aead8c07..cc80ef2218ed 100644
--- a/test/testenv.py
+++ b/test/testenv.py
@@ -3,6 +3,7 @@
 
 import sys
 import os
+import unittest
 
 testext = '.c'
 testdir = os.path.dirname(os.path.abspath(__file__))
@@ -10,6 +11,16 @@ rootdir = os.path.dirname(testdir)
 
 sys.path.insert(0, rootdir)
 
+def _testcase_name(testcase):
+"""Convert a testcase filename into a test case identifier."""
+name = os.path.splitext(os.path.basename(testcase))[0]
+name = name.replace('-', '_')
+name = 'test_{name}'.format(name=name)
+
+assert name.isidentifier()
+
+return name
+
 def get_testcases(path):
 for f in sorted(os.listdir(path)):
 if f.endswith(testext):
@@ -52,3 +63,21 @@ def read_file(filename, **kwargs):
 expected = file.read()
 
 return expected
+
+def _test_generator(get_output, get_expected, input_filename, **options):
+"""Return a function that compares output/expected results on 
input_filename."""
+def test(self):
+output = get_output(input_filename, **options)
+expected = get_expected(input_filename, **options)
+
+self.assertEqual(expected, output)
+
+return test
+
+def assign_test_methods(cls, get_output, get_expected):
+"""Assign test case functions to the given class."""
+for f in get_testcases(testdir):
+options = get_testcase_options(f)
+method = _test_generator(get_output, get_expected, f, **options)
+
+setattr(cls, _testcase_name(f), method)
-- 
2.20.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli: Support --include-html and --body=false for --format=text

[Jani Nikula]

On Thu, 25 Oct 2018, ma...@kakoune.org wrote:
> text format is convenient for shell based parsing of notmuch output,
> adding support for --include-html and --body=false improves its
> usefulness to write complex shell based scripts.

It's debatable whether parsing the --format=text output correctly is
convenient or not. Particularly for notmuch show I think the text format
output is basically legacy, and I don't think we're all that fond of
adding new features to it.

In any case, this patch has two independent changes in one, and should
be split.

BR,
Jani.


> ---
>   NEWS  |  9 +
>   doc/man1/notmuch-show.rst | 15 ---
>   notmuch-show.c| 20 +++-
>   test/T190-multipart.sh| 24 
>   4 files changed, 56 insertions(+), 12 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index ca3ba99e..6d7e7162 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -1,3 +1,12 @@
> +Notmuch 0.29 (UNRELEASED)
> +=
> +
> +Command Line Interface
> +--
> +
> +`notmuch show` now supports --include-html and --body=false with
> +--format=text
> +
>   Notmuch 0.28 (2018-10-12)
>   =
>
> diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
> index 8bfa87c6..a2708a04 100644
> --- a/doc/man1/notmuch-show.rst
> +++ b/doc/man1/notmuch-show.rst
> @@ -176,18 +176,19 @@ Supported options for **show** include
>   ``--body=(true|false)``
>   If true (the default) **notmuch show** includes the bodies of the
>   messages in the output; if false, bodies are omitted.
> -``--body=false`` is only implemented for the json and sexp formats
> -and it is incompatible with ``--part > 0.``
> +``--body=false`` is only implemented for the text, json and sexp
> +formats and it is incompatible with ``--part > 0.``
>
>   This is useful if the caller only needs the headers as body-less
>   output is much faster and substantially smaller.
>
>   ``--include-html``
> -Include "text/html" parts as part of the output (currently only
> -supported with ``--format=json`` and ``--format=sexp``). By 
> default,
> -unless ``--part=N`` is used to select a specific part or
> -``--include-html`` is used to include all "text/html" parts, no
> -part with content type "text/html" is included in the output.
> +Include "text/html" parts as part of the output (currently
> +only supported with ``--format=text``, ``--format=json`` and
> +``--format=sexp``). By default, unless ``--part=N`` is used to
> +select a specific part or ``--include-html`` is used to include all
> +"text/html" parts, no part with content type "text/html" is 
> included
> +in the output.
>
>   A common use of **notmuch show** is to display a single thread of email
>   messages. For this, use a search term of "thread:" as can be
> diff --git a/notmuch-show.c b/notmuch-show.c
> index c3a3783a..07e9a5db 100644
> --- a/notmuch-show.c
> +++ b/notmuch-show.c
> @@ -574,12 +574,18 @@ format_part_text (const void *ctx, sprinter_t *sp, 
> mime_node_t *node,
>   g_mime_stream_printf (stream, "Date: %s\n", date_string);
>   g_mime_stream_printf (stream, "\fheader}\n");
>
> + if (!params->output_body)
> + {
> + g_mime_stream_printf (stream, "\f%s}\n", part_type);
> + return NOTMUCH_STATUS_SUCCESS;
> + }
>   g_mime_stream_printf (stream, "\fbody{\n");
>   }
>
>   if (leaf) {
>   if (g_mime_content_type_is_type (content_type, "text", "*") &&
> - !g_mime_content_type_is_type (content_type, "text", "html"))
> + (params->include_html ||
> +  ! g_mime_content_type_is_type (content_type, "text", "html")))
>   {
>   show_text_part_content (node->part, stream, 0);
>   } else {
> @@ -1204,15 +1210,19 @@ notmuch_show_command (notmuch_config_t *config, 
> int argc, char *argv[])
>   fprintf (stderr, "Warning: --body=false is incompatible with 
> --part > 0. Disabling.\n");
>   params.output_body = true;
>   } else {
> - if (format != NOTMUCH_FORMAT_JSON && format != 
> NOTMUCH_FORMAT_SEXP)
> + if (format != NOTMUCH_FORMAT_TEXT &&
> + format != NOTMUCH_FORMAT_JSON &&
> + format != NOTMUCH_FORMAT_SEXP)
>   fprintf (stderr,
> -  "Warning: --body=false only implemented for 
> format=json and 
> format=sexp\n");
> +  "Warning: --body=false only implemented for 
> format=text, 
> format=json and format=sexp\n");
>   }
>   }
>
>   if (params.include_html &&
> -(format != NOTMUCH_FORMAT_JSON && format != 
> NOTMUCH_FORMAT_SEXP)) {
> - fprintf (stderr, "Warning: --include-html only implemented for 
> format=json and format=sexp\n");
> +(format != NOTMUCH_FORMAT_TEXT &&
> +  format != NOTMUCH_FORMAT_JSON &&
> +  format != NOTMUCH_FORMAT_SEXP)) {
> + fprintf (stderr, 

Re: Notmuch DB Problems

[Jani Nikula]

On Tue, 04 Sep 2018, mu...@nawaz.org wrote:
> Hi,
>
> A few days ago I noticed notmuch new was no longer working (I have it
> as a cron job so it took a while to figure it out).
>
> It just freezes. I do have a Python hook, and it was freezing on the
> line that opens the database.
>
> I tried a notmuch dump. Same problem - freezes
>
> Based on some earlier threads, I tried a notmuch compact. Same problem
> - freezes.
>
> All these freezes seem to use no memory/CPU. 
>
> Interestingly, queries work fine - from both the command line and the
> Emacs interface. So I can read old stuff just fine. But all the
> commands above cause a freeze. 
>
> Currently using notmuch-0.24.2. I tried notmuch-0.27 - same problem.
>
> Results of a xapian check:
>
> docdata:
> blocksize=8K items=6 firstunused=3 revision=6442 levels=0 root=0
> B-tree checked okay
> docdata table structure checked OK
>
> termlist:
> blocksize=8K items=178562 firstunused=53441 revision=6442 levels=2
> root=46086
> /usr/bin/xapian-check: DatabaseError: 1 unused block(s) missing from
> the free list, first is 0
>
> What are my options? Unfortunately the last dump I have is many months
> old, so I'm a bit wary of deleting the database and rebuilding. Given
> that the show and search commands work, I was wondering if I can write
> a script to get all the message/thread ID's for all the tags and store
> them, and then rebuild the database and use that stored information to
> retag all my messages (all without using the dump command)?

It might be interesting to see an strace log to possibly get an idea
where it gets stuck.

Is the filesystem writable and working okay?

If search and show work, I'm guessing it gets stuck in trying to open
the database writable. One hackish idea is to patch notmuch dump to open
the database in read-only mode, and dump the tags. See below. The dump
command opens the database writable to prevent changes while
dumping. (Arguably this could be a command line option for cases like
yours.)

BR,
Jani.

diff --git a/notmuch-dump.c b/notmuch-dump.c
index ef2f02dfeb5c..d06dbcf50224 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -364,7 +364,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, 
char *argv[])
 int ret;
 
 if (notmuch_database_open (notmuch_config_get_database_path (config),
-  NOTMUCH_DATABASE_MODE_READ_WRITE, ))
+  NOTMUCH_DATABASE_MODE_READ_ONLY, ))
return EXIT_FAILURE;
 
 notmuch_exit_if_unmatched_db_uuid (notmuch);
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch shows tag which is not on any email

[Jani Nikula]

On Mon, 04 Jun 2018, Gregor Zattler  wrote:
> Dear notmuch developers,
>
> does notmuch remember tags even if at time of query there is no
> message tagged with the respective tag?:
>
> $ notmuch search --output=tags '*' | grep telegraph
> EA%3Dtelegraph%40gmx%2Enet
> EA=telegr...@gmx.net
>
>
> $ notmuch count  -- is:EA%3Dtelegraph%40gmx%2Enet
> 0
>
>
> $ notmuch count  -- is:/EA.*telegraph/
> 4498
>
>
> There is no invisible character or some such:
> $ notmuch search --output=tags '*' | grep telegraph | head -n 1 | wc -c
> 27

Try all of the above with --exclude=false parameter and see if it makes
a difference. If it does, each of the messages tagged with
EA%3Dtelegraph%40gmx%2Enet is probably also tagged with one of the tags
in 'notmuch config get search.exclude_tags'.

> I might have done something wrong while experimenting with this
> tags and  'EA%3Dtelegraph%40gmx%2Enet' may be a leftover.  I
> would like to remove it but since no message matches it's not
> possible to remove it from messages and therefore the tag remains:
>
> $ notmuch tag +EA=telegr...@gmx.net  -- is:/EA.*telegraph/

Try that with

$ notmuch tag +EA=telegr...@gmx.net -EA%3Dtelegraph%40gmx%2Enet -- 
is:/EA.*telegraph/

to remedy the situation.

BR,
Jani.

> $ notmuch search --output=tags '*' | grep telegraph
> EA%3Dtelegraph%40gmx%2Enet
> EA=telegr...@gmx.net
>
>
> I tried to utilise emacs to remove the tag, but notmuch-emacs
> does not show 'EA%3Dtelegraph%40gmx%2Enet' in notmuch-hello's
> 'All tags' section.
>
>
>
> Why is said tag in the tags listing, how to get rid of it?
>
>
> Thanks, Gregor
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/4] lib: define specialized get_thread_id for use in thread subquery

[Jani Nikula]

On Sat, 05 May 2018, David Bremner  wrote:
> The observation is that we are only using the messages to get there
> thread_id, which is kindof a pessimal access pattern for the current
> notmuch_message_get_thread_id

LGTM.

> ---
>  lib/message.cc| 17 +
>  lib/notmuch-private.h |  4 
>  lib/thread-fp.cc  |  2 +-
>  3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/lib/message.cc b/lib/message.cc
> index d5db89b6..b2067076 100644
> --- a/lib/message.cc
> +++ b/lib/message.cc
> @@ -318,6 +318,23 @@ _notmuch_message_get_term (notmuch_message_t *message,
>  return value;
>  }
>  
> +/*
> + * For special applications where we only want the thread id, reading
> + * in all metadata is a heavy I/O penalty.
> + */
> +const char *
> +_notmuch_message_get_thread_id_only (notmuch_message_t *message)
> +{
> +
> +Xapian::TermIterator i = message->doc.termlist_begin ();
> +Xapian::TermIterator end = message->doc.termlist_end ();
> +
> +message->thread_id = _notmuch_message_get_term (message, i, end,
> + _find_prefix ("thread"));
> +return message->thread_id;
> +}
> +
> +
>  static void
>  _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field)
>  {
> diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
> index 1093429c..4598577f 100644
> --- a/lib/notmuch-private.h
> +++ b/lib/notmuch-private.h
> @@ -537,6 +537,10 @@ _notmuch_message_database (notmuch_message_t *message);
>  
>  void
>  _notmuch_message_remove_unprefixed_terms (notmuch_message_t *message);
> +
> +const char *
> +_notmuch_message_get_thread_id_only(notmuch_message_t *message);
> +
>  /* sha1.c */
>  
>  char *
> diff --git a/lib/thread-fp.cc b/lib/thread-fp.cc
> index dd292bf6..661d00dd 100644
> --- a/lib/thread-fp.cc
> +++ b/lib/thread-fp.cc
> @@ -50,7 +50,7 @@ ThreadFieldProcessor::operator() (const std::string & str)
>   std::string term = thread_prefix;
>   notmuch_message_t *message;
>   message = notmuch_messages_get (messages);
> - term += notmuch_message_get_thread_id (message);
> + term += _notmuch_message_get_thread_id_only (message);
>   terms.insert (term);
>   }
>   return Xapian::Query (Xapian::Query::OP_OR, terms.begin (), 
> terms.end ());
> -- 
> 2.17.0
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/4] lib: add thread subqueries.

[Jani Nikula]

On Sat, 05 May 2018, David Bremner  wrote:
> This change allows queries of the form
>
>  thread:{from:me} and thread:{from:jian} and not thread:{from:dave}
>
> This is still somewhat brute-force, but it's a big improvement over
> both the shell script solution and the previous proposal [1], because it
> does not build the whole thread structure just generate a
> query. A further potential optimization is to replace the calls to
> notmuch with more specialized Xapian code; in particular it's not
> likely that reading all of the message metadata is a win here.
>
> [1]: id:20170820213240.20526-1-da...@tethera.net
> ---
>  lib/Makefile.local   |  3 +-
>  lib/database.cc  |  6 +++-
>  lib/thread-fp.cc | 67 
>  lib/thread-fp.h  | 42 ++
>  test/T585-thread-subquery.sh | 46 +
>  5 files changed, 162 insertions(+), 2 deletions(-)
>  create mode 100644 lib/thread-fp.cc
>  create mode 100644 lib/thread-fp.h
>  create mode 100755 test/T585-thread-subquery.sh
>
> diff --git a/lib/Makefile.local b/lib/Makefile.local
> index 8aa03891..5dc057c0 100644
> --- a/lib/Makefile.local
> +++ b/lib/Makefile.local
> @@ -58,7 +58,8 @@ libnotmuch_cxx_srcs =   \
>   $(dir)/query-fp.cc  \
>   $(dir)/config.cc\
>   $(dir)/regexp-fields.cc \
> - $(dir)/thread.cc
> + $(dir)/thread.cc \
> + $(dir)/thread-fp.cc
>  
>  libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) 
> $(libnotmuch_cxx_srcs:.cc=.o)
>  
> diff --git a/lib/database.cc b/lib/database.cc
> index 02444e09..9cf8062c 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -21,6 +21,7 @@
>  #include "database-private.h"
>  #include "parse-time-vrp.h"
>  #include "query-fp.h"
> +#include "thread-fp.h"
>  #include "regexp-fields.h"
>  #include "string-util.h"
>  
> @@ -258,7 +259,8 @@ prefix_t prefix_table[] = {
>  { "directory",   "XDIRECTORY",   NOTMUCH_FIELD_NO_FLAGS },
>  { "file-direntry",   "XFDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS 
> },
>  { "directory-direntry",  "XDDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
> -{ "thread",  "G",NOTMUCH_FIELD_EXTERNAL 
> },
> +{ "thread",  "G",NOTMUCH_FIELD_EXTERNAL |
> + NOTMUCH_FIELD_PROCESSOR },
>  { "tag", "K",NOTMUCH_FIELD_EXTERNAL |
>   NOTMUCH_FIELD_PROCESSOR },
>  { "is",  "K",NOTMUCH_FIELD_EXTERNAL |
> @@ -317,6 +319,8 @@ _setup_query_field (const prefix_t *prefix, 
> notmuch_database_t *notmuch)
>   fp = (new DateFieldProcessor())->release ();
>   else if (STRNCMP_LITERAL(prefix->name, "query") == 0)
>   fp = (new QueryFieldProcessor (*notmuch->query_parser, 
> notmuch))->release ();
> + else if (STRNCMP_LITERAL(prefix->name, "thread") == 0)
> + fp = (new ThreadFieldProcessor (*notmuch->query_parser, 
> notmuch))->release ();
>   else
>   fp = (new RegexpFieldProcessor (prefix->name, prefix->flags,
>   *notmuch->query_parser, 
> notmuch))->release ();
> diff --git a/lib/thread-fp.cc b/lib/thread-fp.cc
> new file mode 100644
> index ..dd292bf6
> --- /dev/null
> +++ b/lib/thread-fp.cc
> @@ -0,0 +1,67 @@
> +/* thread-fp.cc - "thread:" field processor glue
> + *
> + * This file is part of notmuch.
> + *
> + * Copyright © 2018 David Bremner
> + *
> + * This program is free software: you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation, either version 3 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program.  If not, see https://www.gnu.org/licenses/ .
> + *
> + * Author: David Bremner 
> + */
> +
> +#include "database-private.h"
> +#include "thread-fp.h"
> +#include 
> +
> +#if HAVE_XAPIAN_FIELD_PROCESSOR
> +
> +Xapian::Query
> +ThreadFieldProcessor::operator() (const std::string & str)
> +{
> +notmuch_status_t status;
> +const char *thread_prefix = _find_prefix ("thread");
> +
> +if (str.at (0) == '{') {
> + if (str.length () > 1 && str.at (str.size () - 1) == '}') {

IIUC .length() and .size() are the same thing, but it's confusing to see
them both used on the same line.

Nitpick, I always favor dealing with error cases first, so you can do
the happy day scenario with less indent. So I'd check the opposite,

Re: [PATCH 2/2] test: use --no-mtime-opt in T050-new.sh

[Jani Nikula]

On Sun, 29 Apr 2018, David Bremner  wrote:
> Wherever the test relies on directories being scanned, this option
> should be used to avoid skipping them due to mtimes on directories
> matching the database.

I think you could additionally remove a few touch calls in the
test. Some of them do actually create empty files, but some of them just
touch directories to force rescans.

Otherwise, LGTM.

BR,
Jani.

> ---
>  test/T050-new.sh | 14 +++---
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/test/T050-new.sh b/test/T050-new.sh
> index 9025fa7a..12dba471 100755
> --- a/test/T050-new.sh
> +++ b/test/T050-new.sh
> @@ -87,7 +87,7 @@ notmuch new > /dev/null
>  
>  mv "${MAIL_DIR}"/dir "${MAIL_DIR}"/dir-renamed
>  
> -output=$(NOTMUCH_NEW --debug)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt)
>  test_expect_equal "$output" "(D) add_files, pass 2: queuing passed directory 
> ${MAIL_DIR}/dir for deletion from database
>  No new mail. Detected 3 file renames."
>  
> @@ -95,7 +95,7 @@ No new mail. Detected 3 file renames."
>  test_begin_subtest "Deleted directory"
>  rm -rf "${MAIL_DIR}"/dir-renamed
>  
> -output=$(NOTMUCH_NEW --debug)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt)
>  test_expect_equal "$output" "(D) add_files, pass 2: queuing passed directory 
> ${MAIL_DIR}/dir-renamed for deletion from database
>  No new mail. Removed 3 messages."
>  
> @@ -114,7 +114,7 @@ test_begin_subtest "Deleted directory (end of list)"
>  
>  rm -rf "${MAIL_DIR}"/zzz
>  
> -output=$(NOTMUCH_NEW --debug)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt)
>  test_expect_equal "$output" "(D) add_files, pass 3: queuing leftover 
> directory ${MAIL_DIR}/zzz for deletion from database
>  No new mail. Removed 3 messages."
>  
> @@ -165,7 +165,7 @@ test_begin_subtest "Deleted two-level directory"
>  
>  rm -rf "${MAIL_DIR}"/two
>  
> -output=$(NOTMUCH_NEW --debug)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt)
>  test_expect_equal "$output" "(D) add_files, pass 3: queuing leftover 
> directory ${MAIL_DIR}/two for deletion from database
>  No new mail. Removed 3 messages."
>  
> @@ -211,7 +211,7 @@ Subject: Test mbox message 2
>  
>  Body 2.
>  EOF
> -output=$(NOTMUCH_NEW --debug 2>&1)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt 2>&1)
>  test_expect_equal "$output" \
>  "Note: Ignoring non-mail file: ${MAIL_DIR}/.git/config
>  Note: Ignoring non-mail file: ${MAIL_DIR}/.ignored_hidden_file
> @@ -234,7 +234,7 @@ touch "${MAIL_DIR}"/.git # change .git's mtime for 
> notmuch new to rescan.
>  touch "${MAIL_DIR}"  # likewise for MAIL_DIR
>  mkdir -p "${MAIL_DIR}"/one/two/three/.git
>  touch "${MAIL_DIR}"/{one,one/two,one/two/three}/ignored_file
> -output=$(NOTMUCH_NEW --debug 2>&1 | sort)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt 2>&1 | sort)
>  test_expect_equal "$output" \
>  "(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.git
>  (D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
> @@ -261,7 +261,7 @@ test_expect_equal "$output" "No new mail."
>  
>  test_begin_subtest "Ignore files and directories specified in new.ignore 
> (regexp)"
>  notmuch config set new.ignore ".git" "/^bro.*ink\$/" "/ignored.*file/"
> -output=$(NOTMUCH_NEW --debug 2>&1 | sort)
> +output=$(NOTMUCH_NEW --debug --no-mtime-opt 2>&1 | sort)
>  test_expect_equal "$output" \
>  "(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.git
>  (D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
> -- 
> 2.17.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] CLI/new: add mtime-opt option

[Jani Nikula]

On Sun, 29 Apr 2018, David Bremner  wrote:
> This option, on by default, controls whether mtimes are used to
> optimize the scanning of directories. The intent is to turn this
> optimization off for certain tests.
> ---
>  notmuch-new.c | 9 ++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/notmuch-new.c b/notmuch-new.c
> index c4345705..099bbbae 100644
> --- a/notmuch-new.c
> +++ b/notmuch-new.c
> @@ -47,6 +47,7 @@ typedef struct {
>  int output_is_a_tty;
>  enum verbosity verbosity;
>  bool debug;
> +bool mtime_opt;
>  const char **new_tags;
>  size_t new_tags_length;
>  const char **ignore_verbatim;
> @@ -527,7 +528,7 @@ add_files (notmuch_database_t *notmuch,
>   * mistakenly return the total number of directory entries, since
>   * that only inflates the count beyond 2.
>   */
> -if (directory && fs_mtime == db_mtime && st.st_nlink == 2) {
> +if (directory && state->mtime_opt && fs_mtime == db_mtime && st.st_nlink 
> == 2) {
>   /* There's one catch: pass 1 below considers symlinks to
>* directories to be directories, but these don't increase the
>* file system link count.  So, only bail early if the
> @@ -618,7 +619,7 @@ add_files (notmuch_database_t *notmuch,
>   * being discovered until the clock catches up and the directory
>   * is modified again).
>   */
> -if (directory && fs_mtime == db_mtime)
> +if (directory && state->mtime_opt && fs_mtime == db_mtime)
>   goto DONE;
>  
>  /* If the database has never seen this directory before, we can
> @@ -771,7 +772,7 @@ add_files (notmuch_database_t *notmuch,
>   * the database because a message could be delivered later in this
>   * same second.  This may lead to unnecessary re-scans, but it
>   * avoids overlooking messages. */
> -if (fs_mtime != stat_time)
> +if (state->mtime_opt && fs_mtime != stat_time)
>   _filename_list_add (state->directory_mtimes, path)->mtime = fs_mtime;

I don't think we should skip this part. We've done a full scan now, so
we should record that in the database so a subsequent scan doesn't have
to.

>  
>DONE:
> @@ -1053,6 +1054,7 @@ notmuch_new_command (notmuch_config_t *config, int 
> argc, char *argv[])
>  add_files_state_t add_files_state = {
>   .verbosity = VERBOSITY_NORMAL,
>   .debug = false,
> + .mtime_opt = true,
>   .output_is_a_tty = isatty (fileno (stdout)),
>  };
>  struct timeval tv_start;
> @@ -1073,6 +1075,7 @@ notmuch_new_command (notmuch_config_t *config, int 
> argc, char *argv[])
>   { .opt_bool = , .name = "quiet" },
>   { .opt_bool = , .name = "verbose" },
>   { .opt_bool = _files_state.debug, .name = "debug" },
> + { .opt_bool = _files_state.mtime_opt, .name = "mtime-opt" },

--full-scan? --force? --force-scan?

I think we've had people ask how they can have notmuch scan some
directories again, for some reason or another. Maybe to test their
ignores, and they can't have notmuch rescan the directory without
touching it. IMHO --no-mtime-opt doesn't sound very intuitive for that
purpose.

Otherwise, seems like a thing we should add.

BR,
Jani.

>   { .opt_bool = , .name = "hooks" },
>   { .opt_inherit = notmuch_shared_indexing_options },
>   { .opt_inherit = notmuch_shared_options },
> -- 
> 2.17.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: non-deterministic behaviour of new.ignore (regexp) test

[Jani Nikula]

On Sat, 28 Apr 2018, David Bremner  wrote:
> For me the following seems to consistently fail after between 30 and 500
> attempts
>
> export NOTMUCH_TEST_QUIET=yes; count=0; while ./T050-new.sh; do (( 
> count++ )); echo $count; done

I believe this happens because the directory mtime is unchanged from the
previous scan in the test, and we skip the directory before we could
ignore the files. Quoting add_files():

/* If the directory's modification time in the filesystem is the
 * same as what we recorded in the database the last time we
 * scanned it, then we can skip the second pass entirely.
 *
 * We test for strict equality here to avoid a bug that can happen
 * if the system clock jumps backward, (preventing new mail from
 * being discovered until the clock catches up and the directory
 * is modified again).
 */

I can't reproduce if I add this to the test:

diff --git a/test/T050-new.sh b/test/T050-new.sh
index 9025fa7aa63e..0db76f47130b 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -260,6 +260,7 @@ output=$(NOTMUCH_NEW 2>&1)
 test_expect_equal "$output" "No new mail."
 
 test_begin_subtest "Ignore files and directories specified in new.ignore 
(regexp)"
+touch "${MAIL_DIR}" # force rescan of the top level directory
 notmuch config set new.ignore ".git" "/^bro.*ink\$/" "/ignored.*file/"
 output=$(NOTMUCH_NEW --debug 2>&1 | sort)
 test_expect_equal "$output" \

---

However, I'm not sure even that is enough if all this happens in the
same second. I think the way notmuch new is written, it may skip as long
as it ensures a subsequent scan will catch the modified files:

/* If the directory's mtime is the same as the wall-clock time
 * when we stat'ed the directory, we skip updating the mtime in
 * the database because a message could be delivered later in this
 * same second.  This may lead to unnecessary re-scans, but it
 * avoids overlooking messages. */

I think we can make the problem less likely with the touch, but as
everything gets faster, we might hit this more and more. One approach
might be a notmuch new --force option that would rescan all directories
regardless of mtimes. We could use this for testing (except when we're
testing the optimization).

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] cli: looking for config file in $XDG_CONFIG_HOME

[Jani Nikula]

On Thu, 15 Mar 2018, Daniel Kahn Gillmor  wrote:
> On Wed 2018-03-14 22:54:06 -0300, David Bremner wrote:
>> There doesn't seem to be a good reason to drop ~/.notmuch-config
>> completely here. As Tomi notes, that would break notmuch for all current
>> users. I suppose I could live with $XDG_CONFIG_HOME/notmuch/config (or
>> whatever) taking precedence if it exists.
>
> If we ant to keep the config file, then i agree that
> $XDG_CONFIG_HOME/notmuch/config is probably the a better place for it
> than ~/.notmuch-config.  But:
>
> It seemed to me that there was a growing consensus that the configfile
> could be phased out in favor of storing configuration details in the
> database itself.  (this is dependent on someone™ actually doing the
> implementation work, of course)

What about configuration specific to the CLI that is irrelevant for the
library/database? Do you propose to store those in the database too? For
example user.name or crypto.gpg_path. And then there's the database
location, which obviously can't be in the database.

For database specific configuration that currently lies in the CLI
config, there's also a bunch of rework required. For example, if we
store new.tags in the database, what would be the point of the CLI
reading those out of the database, and then applying them to the
message? Shouldn't the new.tags be applied to the messages at the
library level directly? Ditto for search.exclude_tags and
maildir.synchronize_flags.

Where's the point in moving all this stuff to the database, unless we
also use them directly in the database?

> if we do this, it seems likely that we'll need to keep around handling
> the config file for backward compatibility, via something like:
>
>  * when we observe a config file, we could walk each option present in
>it.  For each option:
>
> a) if that option is not present in the database, copy it into the
>database.
>
> b) if that option is present in the database, and it matches the
>option in the config file, ignore
>
> c) if that option is present in the database but does not match the
>config file, use the version in the config file but warn the user
>that they have a conflict they should probably resolve soon.

Wouldn't this require having an in-memory copy of the database config,
where the items originating from the config file would override but not
get saved into the on-disk version of the database config?

> after some number of versions has elapsed and we are ready to explicitly
> deprecate the config file, perhaps we convert situations (a) and (b)
> into warning messages and situation (c) into a hard error.
>
> This is all a bit confusing, but it is the price we pay for having a
> smooth transition that allows users to upgrade knowing they can roll
> back if a new version isn't working for them.

I think it's more confusing than you think. ;)

I also kind of like the CLI config file that I can just save in git
along with my other dotfiles. The database config is much more annoying
in this regard.

> So i think it would be a shame to have an additional layer of confusion
> added by having two different deprecated configuration files.  So i lean
> against adopting this change. I'd much rather see work on phasing out
> the configfile.

Agreed on not increasing the confusion, at least not before we figure
out what we want to do.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch-search-get-tags unique candidates

[Jani Nikula]

On Wed, Mar 21, 2018 at 11:21 PM, Nicolò Balzarotti
 wrote:
> Hi, I'm using notmuch with emacs and I'm loving it. One thing I don't
> like is that in `notmuch-search`, when applying a tag to a selection the
> tab completition return multiple copies of the same tag. Example:
>
>> -inbox
>> -lists/aaa
>> -lists
>> -inbox
>> -lists/aaa
>> -lists
>
> I changed `notmuch-search-get-tags-region` to:
> ```(defun notmuch-search-get-tags-region (beg end)
>   (let (output)
> (notmuch-search-foreach-result beg end
>   (lambda (pos)
> (setq output (append output (notmuch-search-get-tags pos)
> (delete-dups output)));; add delete-dups before returning
> ```
>
> Why is the current one the default behavior?

FWIW, I can't reproduce the behaviour you describe.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: multilingual notmuch (and Content-Language)

[Jani Nikula]

On Sun, 18 Mar 2018, Daniel Kahn Gillmor  wrote:
>  * if we know our index expects english, and we have a message part that
>*is not* english (e.g. Content-Language: es), we could avoid indexing
>that part.

Why would we do that? Search mostly works just fine for non-English
languages, it's just that the *stemming* is not right.

> what do you think?  what ideas are missing from the branstorm above?  I'd
> love to hear from people with multilingual mailboxes about how we might
> be able to make notmuch work better for them.

With my limited understanding of this, stemming happens both at indexing
and searching. Basically at indexing, the term generator indexes both
the full and the stemmed version of words. I'm wondering if we could
look at Content-Language (and missing that, heuristics), and (if the
user so desires) use multiple term generators with different stemmers on
a per document basis. Or, use non-stemming indexing for unidentified or
unsupported languages. How far would that take us? Then, perhaps, we
could also perform language specific queries?

I don't know how feasible that is, or if it would require Xapian
changes.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] notmuch-mutt: use --format=text0 and xargs -0

[Jani Nikula]

On Tue, 27 Feb 2018, Jani Nikula <j...@nikula.org> wrote:
> notmuch-mutt fails for message files with special characters such as
> single quote in their filename. Use notmuch search --format=text0 and
> xargs -0 combo to handle them.
>
> Reported and tested by "dob1" on IRC.
> ---
>  contrib/notmuch-mutt/notmuch-mutt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/contrib/notmuch-mutt/notmuch-mutt 
> b/contrib/notmuch-mutt/notmuch-mutt
> index 0e46a8c1b95e..57f13075aa22 100755
> --- a/contrib/notmuch-mutt/notmuch-mutt
> +++ b/contrib/notmuch-mutt/notmuch-mutt
> @@ -48,9 +48,9 @@ sub search($$$) {
>  }
>  
>  empty_maildir($maildir);
> -system("notmuch search --output=files $dup_option $query"
> +system("notmuch search --format=text0 --output=files $dup_option $query"
>  . " | sed -e 's: : :g'"

Come to think of it, does this need sed -z too?

> -. " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/");
> +. " | xargs -0 -r -I searchoutput ln -s searchoutput $maildir/cur/");
>  }
>  
>  sub prompt($$) {
> -- 
> 2.11.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] notmuch-mutt: use --format=text0 and xargs -0

[Jani Nikula]

notmuch-mutt fails for message files with special characters such as
single quote in their filename. Use notmuch search --format=text0 and
xargs -0 combo to handle them.

Reported and tested by "dob1" on IRC.
---
 contrib/notmuch-mutt/notmuch-mutt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/notmuch-mutt/notmuch-mutt 
b/contrib/notmuch-mutt/notmuch-mutt
index 0e46a8c1b95e..57f13075aa22 100755
--- a/contrib/notmuch-mutt/notmuch-mutt
+++ b/contrib/notmuch-mutt/notmuch-mutt
@@ -48,9 +48,9 @@ sub search($$$) {
 }
 
 empty_maildir($maildir);
-system("notmuch search --output=files $dup_option $query"
+system("notmuch search --format=text0 --output=files $dup_option $query"
   . " | sed -e 's: : :g'"
-  . " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/");
+  . " | xargs -0 -r -I searchoutput ln -s searchoutput $maildir/cur/");
 }
 
 sub prompt($$) {
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re:

[Jani Nikula]

On Thu, 01 Feb 2018, Matthew Lear  wrote:
> From: Matthew Lear 
> To: notmuch@notmuchmail.org
> Cc: Matthew Lear 
> Subject: [PATCH] Update date search syntax.
> Date: Thu,  1 Feb 2018 20:52:18 +
> Message-Id: <20180201205218.4368-1-m...@bubblegen.co.uk>
> X-Mailer: git-send-email 2.14.1
>
> If searching using the date prefix and timestamps, each timestamp
> is required to be prefixed with an @
> Legacy syntax of .. without the
> date prefix is still honoured, only without the @ specifiers.
> ---
>  doc/man7/notmuch-search-terms.rst | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/doc/man7/notmuch-search-terms.rst 
> b/doc/man7/notmuch-search-terms.rst
> index 6d2bf62a..b6e7079a 100644
> --- a/doc/man7/notmuch-search-terms.rst
> +++ b/doc/man7/notmuch-search-terms.rst
> @@ -124,10 +124,13 @@ date:.. or date:
>  The time range can also be specified using timestamps with a
>  syntax of:
>  
> -..
> +@..@

So I think I'd add the @ syntax in the DATE AND TIME SEARCH section,
maybe under a separate new heading, and just emphasize this here is
about the non-date prefixed thing.

BR,
Jani.
  
>  Each timestamp is a number representing the number of seconds
> -since 1970-01-01 00:00:00 UTC.
> +since 1970-01-01 00:00:00 UTC. A date range search using
> +timestamps is also permitted without using the date prefix and
> +@ specifiers, although this is considered legacy and pre-dates
> +the date prefix.
>  
>  lastmod:..
>  The **lastmod:** prefix can be used to restrict the result by the
> -- 
> 2.14.1
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: trouble searching with unix timestamps

[Jani Nikula]

On Mon, 29 Jan 2018, Matthew Lear  wrote:
> Hi. I've noticed that I'm unable to notmuch search using date with unix
> timestamps (ie number of seconds since Jan 1st 1970 UTC). I get a xapian
> error. The notmuch man pages state that searching using
> 'date:..'
> where "each timestamp is a number representing the number of seconds since
> 1970-01-01 00:00:00 UTC" is supported, so I'm wondering what's wrong with
> my installation.

The man page could use some clarification.

The .. syntax predates the date:
prefix, and needs to be given as-is:

$ notmuch search 1517152333..1517238733

Alternatively, you can use @ in date: queries, although this
seems to be completely undocumented:

$ notmuch search date:@1517152333..@1517238733

HTH,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] doc: unify definition list usage across man pages

[Jani Nikula]

On Sat, 30 Dec 2017, David Bremner <da...@tethera.net> wrote:
> Jani Nikula <j...@nikula.org> writes:
>
>> Make all parameter descriptions etc. use reStructuredText definition
>> lists with uniform style and indentation. Remove redundant indentation
>> from around the lists. Remove blank lines between term lines and
>> definition blocks. Use four spaces for indentation.
>>
>> This is almost completely whitespace and paragraph reflow changes.
>
> My inclination is to apply this to the master branch, but not to include
> it in the 0.26 release. Any comments?

Make it so.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli: add support for only printing the addresses in notmuch address

[Jani Nikula]

On Tue, 19 Dec 2017, Jameson Graef Rollins <jroll...@finestructure.net> wrote:
> On Wed, Dec 20 2017, Jani Nikula <j...@nikula.org> wrote:
>> ~$ notmuch address --output=sender --output=recipients --output=address 
>> --output=count id:878tdy8a2q@ligo.caltech.edu
>> 1notmuch@notmuchmail.org
>> 1jroll...@finestructure.net
>> 1d...@fifthhorseman.net
>> 1j...@nikula.org
>>
>> I prefer this to separate options.
>
> Really?  If each is just a switch then why not:
>
> ~$ notmuch address --sender --recipients --address --count 
> id:878tdy8a2q@ligo.caltech.edu
>
> ?  It's just shorter, right?
>
> Also, this behavior is quite different than for search, in which only
> the last --output applies.
>
>> notmuch search uses separate --entire-thread, --body, and --include-html
>> options, and I think those are getting messy.

That was supposed to be notmuch show, not search.

> The seem less messy than a "--output=" prefixed version of the same.

Matter of taste. I like the self-documenting aspect of *what* these
options control. --output=body is obvious, --body is less
so. --include-html includes html somewhere, I think --output=html would
be better.

In the spirit of worse is better, I'll note that using --output= stores
all the possible values in a single bit mask, and it's easy to define
the defaults for *all* possible outputs in one variable, and it's easy
to check *all* possible combinations with bit masks. Not so with
independent parameters. Again, matter of taste how much you appreciate
implementation simplicity. With notmuch show, I think that guideline
would have made the interface better too. YMMV.

As to notmuch address --output=address, it fulfills all the feature
needs that popped up in this thread. If there's a strong desired to
change the interface, we'll need patches as this one's already merged.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli: add support for only printing the addresses in notmuch address

[Jani Nikula]

On Tue, 19 Dec 2017, Jameson Graef Rollins <jroll...@finestructure.net> wrote:
> On Tue, Dec 19 2017, Daniel Kahn Gillmor <d...@fifthhorseman.net> wrote:
>> On Tue 2017-12-19 13:23:55 -0800, Jameson Graef Rollins wrote:
>>> On Thu, Nov 02 2017, Jani Nikula <j...@nikula.org> wrote:
>>>> The notmuch address output is much more useful for scripts with just
>>>> the addresses printed. Support this using the --output=address option.
>>>
>>> Isn't "address" kind of orthogonal to "sender" and "recipient"?  Isn't
>>> this more like the --output/--format distinction in search?
>>
>> i think i agree with Jamie here -- it'd be nice to be able to ask for
>> the addresses of the senders, or the addresses of the recipients.  how
>> would that be done here?
>
> Sorry, I see now that address already has the --format option with the
> expected values.  So I think either address-only or sender/recipient
> should be a separate option.

Note that you can give the notmuch address --output option multiple
times to control the output. For example,

~$ notmuch address --output=sender --output=recipients 
id:878tdy8a2q@ligo.caltech.edu
Jameson Graef Rollins <jroll...@finestructure.net>
Daniel Kahn Gillmor <d...@fifthhorseman.net>
Jani Nikula <j...@nikula.org>
notmuch@notmuchmail.org

~$ notmuch address --output=recipients --output=address 
id:878tdy8a2q@ligo.caltech.edu
d...@fifthhorseman.net
j...@nikula.org
notmuch@notmuchmail.org

~$ notmuch address --output=sender --output=recipients --output=address 
--output=count id:878tdy8a2q@ligo.caltech.edu
1   notmuch@notmuchmail.org
1   jroll...@finestructure.net
1   d...@fifthhorseman.net
1   j...@nikula.org

I prefer this to separate options.

notmuch search uses separate --entire-thread, --body, and --include-html
options, and I think those are getting messy.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] doc: unify definition list usage across man pages

[Jani Nikula]

Make all parameter descriptions etc. use reStructuredText definition
lists with uniform style and indentation. Remove redundant indentation
from around the lists. Remove blank lines between term lines and
definition blocks. Use four spaces for indentation.

This is almost completely whitespace and paragraph reflow changes.
---
 doc/man1/notmuch-address.rst| 160 -
 doc/man1/notmuch-compact.rst|  16 +--
 doc/man1/notmuch-config.rst | 256 +++-
 doc/man1/notmuch-count.rst  |  65 +-
 doc/man1/notmuch-dump.rst   | 116 +-
 doc/man1/notmuch-emacs-mua.rst  |  64 +-
 doc/man1/notmuch-insert.rst |  64 +-
 doc/man1/notmuch-new.rst|  33 +++---
 doc/man1/notmuch-reindex.rst|  21 ++--
 doc/man1/notmuch-reply.rst  |  94 ---
 doc/man1/notmuch-restore.rst| 118 +-
 doc/man1/notmuch-search.rst | 220 +-
 doc/man1/notmuch-show.rst   | 256 
 doc/man1/notmuch-tag.rst|  33 +++---
 doc/man1/notmuch.rst|  33 +++---
 doc/man5/notmuch-hooks.rst  |  55 +
 doc/man7/notmuch-properties.rst |   1 -
 17 files changed, 779 insertions(+), 826 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index dbac20f7b012..00cb554523e7 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -18,88 +18,84 @@ See **notmuch-search-terms(7)** for details of the 
supported syntax for
 
 Supported options for **address** include
 
-``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**)
-Presents the results in either JSON, S-Expressions, newline
-character separated plain-text (default), or null character
-separated plain-text (compatible with **xargs(1)** -0 option
-where available).
-
-``--format-version=N``
-Use the specified structured output format version. This is
-intended for programs that invoke **notmuch(1)** internally. If
-omitted, the latest supported version will be used.
-
-``--output=(sender|recipients|count)``
-
-Controls which information appears in the output. This option
-can be given multiple times to combine different outputs.
-When neither --output=sender nor --output=recipients is
-given, --output=sender is implied.
-
-**sender**
-Output all addresses from the *From* header.
-
-Note: Searching for **sender** should be much faster than
-searching for **recipients**, because sender addresses are
-cached directly in the database whereas other addresses
-need to be fetched from message files.
-
-**recipients**
-Output all addresses from the *To*, *Cc* and *Bcc*
-headers.
-
-**count**
-Print the count of how many times was the address
-encountered during search.
-
-Note: With this option, addresses are printed only after
-the whole search is finished. This may take long time.
-
-``--deduplicate=(no|mailbox|address)``
-
-Control the deduplication of results.
-
-**no**
-Output all occurrences of addresses in the matching
-messages. This is not applicable with --output=count.
-
-**mailbox**
-Deduplicate addresses based on the full, case sensitive
-name and email address, or mailbox. This is effectively
-the same as piping the --deduplicate=no output to **sort |
-uniq**, except for the order of results. This is the
-default.
-
-**address**
-Deduplicate addresses based on the case insensitive
-address part of the mailbox. Of all the variants (with
-different name or case), print the one occurring most
-frequently among the matching messages. If --output=count
-is specified, include all variants in the count.
-
-``--sort=``\ (**newest-first**\ \|\ **oldest-first**)
-This option can be used to present results in either
-chronological order (**oldest-first**) or reverse chronological
-order (**newest-first**).
-
-By default, results will be displayed in reverse chronological
-order, (that is, the newest results will be displayed first).
-
-However, if either --output=count or --deduplicate=address is
-specified, this option is ignored and the order of the results
-is unspecified.
-
-``--exclude=(true|false)``
-A message is called "excluded" if it matches at least one tag in
-search.tag\_exclude that does not appear explicitly in the
-search terms. This option specifies whether to omit excluded
-messages in the search process.
-
-The default value, **true**, 

[PATCH] test: expand argument parsing sanity checks

[Jani Nikula]

Test the boolean --arg=true and --arg=false formats.

---

Extracted from id:20171014131608.17587-1-j...@nikula.org as this makes
sense independent of that series.
---
 test/T410-argument-parsing.sh | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/test/T410-argument-parsing.sh b/test/T410-argument-parsing.sh
index 71ed7e38553b..910843d684c6 100755
--- a/test/T410-argument-parsing.sh
+++ b/test/T410-argument-parsing.sh
@@ -37,4 +37,18 @@ positional arg 1 false
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "--boolean=true"
+$TEST_DIRECTORY/arg-test --boolean=true > OUTPUT
+cat < EXPECTED
+boolean 1
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "--boolean=false"
+$TEST_DIRECTORY/arg-test --boolean=false > OUTPUT
+cat < EXPECTED
+boolean 0
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: what is the concept of archiving?

[Jani Nikula]

On Thu, Nov 9, 2017 at 11:56 AM, Linus Arver  wrote:
> Unfortunately, I cannot find enough information online about what any of
> these functions do, or what the concept of "archiving" means. From
> reading some of the sources in notmuch-tree.el I found out that
> archiving a message results in "applying the tag changes in
> `notmuch-archive-tags' to it" but I am not sure what this means.

Archiving, by default, simply means removing the inbox tag, either
from a single message or from all messages in a thread. You can use
notmuch-archive-tags to configure the tag change actions if you like.

> I found out that the key bindings are in that same file, which assigns
> "a" to notmuch-tree-archive-message-then-next; when I press "a" in my
> inbox view (all messages tagged "inbox"), it appears to just remove the
> "inbox" tag. I have some questions:
>
> (1) What is the point of archiving (what problem does it solve)?

Remove the messages from your inbox search. (At least I prefer to keep
this distinct from removing the unread tag; I archive tons of messages
without reading, and I have some messages with inbox tag that I've
already read.)

> (2) Do any other things happen to such archived messages, apart from the
> tag being removed?

No. Optionally, you can configure notmuch-archive-tags to remove more
or other tags, or add tags.

> (3) How do I un-archive these messages, if I press the "a" key by
> accident?

Find the message, hit "k k a". ("k" enters a submenu for doing
customizable tagging operations quickly, "k k" does the reverse
operation.)

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: buggy tags

[Jani Nikula]

On Thu, Nov 9, 2017 at 12:11 PM, Tom Hirschowitz
 wrote:
> I sometimes mistype some tags, e.g., collaegues instead of colleagues,
> etc. Of course, I then replace them with the right version.
>
> But notmuch keeps track of these wrong tags (as of all others) and
> proposes them when autocompleting, which after a few years of using
> notmuch, starts being significantly annoying.

Notmuch only completes to tags that are actually present in the
database. So you've missed some.

> Is there a way of getting rid of them?

This will list all the tags you have:

$ notmuch search --output=tags "*"

And this will replace a wrong tag with the right tag:

$ notmuch tag +right-tag -wrong-tag tag:wrong-tag

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Segfault with pkcs7 message

[Jani Nikula]

On Fri, Nov 3, 2017 at 11:05 AM, Holger Schurig  wrote:
> The attached e-mail makes notmuch from git HEAD segfault:
>
> $ notmuch --version
> notmuch 0.25.1+161~g1b91884
> $ notmuch show --decrypt id:86595dab-d486-536b-fe7a-ae3e1dca1...@posteo.de
> Warning: Crypto engine initialization failure (application/pkcs7-signature).
> Segmentation fault

See http://mid.mail-archive.com/20171016154044.14564-1-jani%40nikula.org

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] doc: unify definition list usage across man pages

[Jani Nikula]

Make all parameter descriptions etc. use reStructuredText definition
lists with uniform style and indentation. Remove redundant indentation
from around the lists. Remove blank lines between term lines and
definition blocks. Use four spaces for indentation.

This is almost completely whitespace and paragraph reflow changes.
---
 doc/man1/notmuch-address.rst| 160 -
 doc/man1/notmuch-compact.rst|  16 +--
 doc/man1/notmuch-config.rst | 256 +++-
 doc/man1/notmuch-count.rst  |  65 +-
 doc/man1/notmuch-dump.rst   | 116 +-
 doc/man1/notmuch-emacs-mua.rst  |  64 +-
 doc/man1/notmuch-insert.rst |  62 +-
 doc/man1/notmuch-new.rst|  33 +++---
 doc/man1/notmuch-reindex.rst|  21 ++--
 doc/man1/notmuch-reply.rst  |  94 ---
 doc/man1/notmuch-restore.rst| 118 +-
 doc/man1/notmuch-search.rst | 220 +-
 doc/man1/notmuch-show.rst   | 256 
 doc/man1/notmuch-tag.rst|  33 +++---
 doc/man1/notmuch.rst|  33 +++---
 doc/man5/notmuch-hooks.rst  |  55 +
 doc/man7/notmuch-properties.rst |   1 -
 17 files changed, 778 insertions(+), 825 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index dbac20f7b012..00cb554523e7 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -18,88 +18,84 @@ See **notmuch-search-terms(7)** for details of the 
supported syntax for
 
 Supported options for **address** include
 
-``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**)
-Presents the results in either JSON, S-Expressions, newline
-character separated plain-text (default), or null character
-separated plain-text (compatible with **xargs(1)** -0 option
-where available).
-
-``--format-version=N``
-Use the specified structured output format version. This is
-intended for programs that invoke **notmuch(1)** internally. If
-omitted, the latest supported version will be used.
-
-``--output=(sender|recipients|count)``
-
-Controls which information appears in the output. This option
-can be given multiple times to combine different outputs.
-When neither --output=sender nor --output=recipients is
-given, --output=sender is implied.
-
-**sender**
-Output all addresses from the *From* header.
-
-Note: Searching for **sender** should be much faster than
-searching for **recipients**, because sender addresses are
-cached directly in the database whereas other addresses
-need to be fetched from message files.
-
-**recipients**
-Output all addresses from the *To*, *Cc* and *Bcc*
-headers.
-
-**count**
-Print the count of how many times was the address
-encountered during search.
-
-Note: With this option, addresses are printed only after
-the whole search is finished. This may take long time.
-
-``--deduplicate=(no|mailbox|address)``
-
-Control the deduplication of results.
-
-**no**
-Output all occurrences of addresses in the matching
-messages. This is not applicable with --output=count.
-
-**mailbox**
-Deduplicate addresses based on the full, case sensitive
-name and email address, or mailbox. This is effectively
-the same as piping the --deduplicate=no output to **sort |
-uniq**, except for the order of results. This is the
-default.
-
-**address**
-Deduplicate addresses based on the case insensitive
-address part of the mailbox. Of all the variants (with
-different name or case), print the one occurring most
-frequently among the matching messages. If --output=count
-is specified, include all variants in the count.
-
-``--sort=``\ (**newest-first**\ \|\ **oldest-first**)
-This option can be used to present results in either
-chronological order (**oldest-first**) or reverse chronological
-order (**newest-first**).
-
-By default, results will be displayed in reverse chronological
-order, (that is, the newest results will be displayed first).
-
-However, if either --output=count or --deduplicate=address is
-specified, this option is ignored and the order of the results
-is unspecified.
-
-``--exclude=(true|false)``
-A message is called "excluded" if it matches at least one tag in
-search.tag\_exclude that does not appear explicitly in the
-search terms. This option specifies whether to omit excluded
-messages in the search process.
-
-The default value, **true**, 

[PATCH] test: test notmuch insert --folder=""

[Jani Nikula]

Test insert into top level folder.
---
 test/T070-insert.sh | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index f1650e623e35..40519bb2f217 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -132,6 +132,13 @@ output=$(notmuch search --output=files path:Drafts/new)
 dirname=$(dirname "$output")
 test_expect_equal "$dirname" "$MAIL_DIR/Drafts/new"
 
+test_begin_subtest "Insert message into top level folder"
+gen_insert_msg
+notmuch insert --folder="" < "$gen_msg_filename"
+output=$(notmuch search --output=files id:${gen_msg_id})
+dirname=$(dirname "$output")
+test_expect_equal "$dirname" "$MAIL_DIR/new"
+
 test_begin_subtest "Insert message into folder with trailing /"
 gen_insert_msg
 notmuch insert --folder=Drafts/ < "$gen_msg_filename"
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] cli: add support for only printing the addresses in notmuch address

[Jani Nikula]

The notmuch address output is much more useful for scripts with just
the addresses printed. Support this using the --output=address option.
---
 completion/notmuch-completion.bash |  2 +-
 doc/man1/notmuch-address.rst   |  7 ++-
 notmuch-search.c   |  7 ++-
 test/T095-address.sh   | 36 
 4 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 7aae4297ae0e..4e94275e3193 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -468,7 +468,7 @@ _notmuch_address()
return
;;
--output)
-   COMPREPLY=( $( compgen -W "sender recipients count" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "sender recipients count address" -- 
"${cur}" ) )
return
;;
--sort)
diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index dbac20f7b012..68415d13c5b6 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -29,7 +29,7 @@ Supported options for **address** include
 intended for programs that invoke **notmuch(1)** internally. If
 omitted, the latest supported version will be used.
 
-``--output=(sender|recipients|count)``
+``--output=(sender|recipients|count|address)``
 
 Controls which information appears in the output. This option
 can be given multiple times to combine different outputs.
@@ -55,6 +55,11 @@ Supported options for **address** include
 Note: With this option, addresses are printed only after
 the whole search is finished. This may take long time.
 
+**address**
+Output only the email addresses instead of the full
+mailboxes with names and email addresses. This option has
+no effect on the JSON or S-Expression output formats.
+
 ``--deduplicate=(no|mailbox|address)``
 
 Control the deduplication of results.
diff --git a/notmuch-search.c b/notmuch-search.c
index 0abac08eb7ab..8f467db43cf0 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -34,6 +34,7 @@ typedef enum {
 OUTPUT_SENDER  = 1 << 5,
 OUTPUT_RECIPIENTS  = 1 << 6,
 OUTPUT_COUNT   = 1 << 7,
+OUTPUT_ADDRESS = 1 << 8,
 } output_t;
 
 typedef enum {
@@ -370,7 +371,10 @@ print_mailbox (const search_context_t *ctx, const 
mailbox_t *mailbox)
format->integer (format, count);
format->string (format, "\t");
}
-   format->string (format, name_addr);
+   if (ctx->output & OUTPUT_ADDRESS)
+   format->string (format, addr);
+   else
+   format->string (format, name_addr);
format->separator (format);
 } else {
format->begin_map (format);
@@ -877,6 +881,7 @@ notmuch_address_command (notmuch_config_t *config, int 
argc, char *argv[])
  (notmuch_keyword_t []){ { "sender", OUTPUT_SENDER },
  { "recipients", OUTPUT_RECIPIENTS },
  { "count", OUTPUT_COUNT },
+ { "address", OUTPUT_ADDRESS },
  { 0, 0 } } },
{ .opt_keyword = >exclude, .name = "exclude", .keywords =
  (notmuch_keyword_t []){ { "true", NOTMUCH_EXCLUDE_TRUE },
diff --git a/test/T095-address.sh b/test/T095-address.sh
index f0291d29ec43..817be5380a45 100755
--- a/test/T095-address.sh
+++ b/test/T095-address.sh
@@ -119,6 +119,42 @@ cat OUTPUT
+cat OUTPUT
+cat 

[PATCH] test: fix test database backup/restore location

[Jani Nikula]

backup_database() and restore_database() used to store the backups in
the test specific temporary directory, through the current working
directory being there. Commit 8e7fb88237ae ("test: use source and
build paths in test-lib-common.sh") started using a test specific
backup directories under the build tree test directory. This was in
error. Switch back to the old location, but using paths to the
location instead of relying on current working directory.

Reported by Daniel Kahn Gillmor .
---
 test/test-lib-common.sh | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 4300eb65418f..6c3571d4c560 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -31,14 +31,14 @@ fi
 
 backup_database () {
 test_name=$(basename $0 .sh)
-rm -rf $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."$test_name"
-cp -pR ${MAIL_DIR}/.notmuch 
$NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}"
+rm -rf $TMP_DIRECTORY/notmuch-dir-backup."$test_name"
+cp -pR ${MAIL_DIR}/.notmuch 
$TMP_DIRECTORY/notmuch-dir-backup."${test_name}"
 }
 
 restore_database () {
 test_name=$(basename $0 .sh)
 rm -rf ${MAIL_DIR}/.notmuch
-cp -pR $NOTMUCH_BUILDDIR/test/notmuch-dir-backup."${test_name}" 
${MAIL_DIR}/.notmuch
+cp -pR $TMP_DIRECTORY/notmuch-dir-backup."${test_name}" 
${MAIL_DIR}/.notmuch
 }
 
 # Test the binaries we have just built.  The tests are kept in
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: nmbug backtrace due to .mailmap

[Jani Nikula]

On Mon, 16 Oct 2017, "W. Trevor King"  wrote:
> b. Comment out the ‘_insist_committed()’ line in nmbug's ‘pull’
>definition and try again.  Make sure you restore the check after
>the successful pull.

FWIW this fixed it for me, thanks.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Jani Nikula]

On Thu, 19 Oct 2017, Daniel Kahn Gillmor  wrote:
> On Thu 2017-10-19 11:01:53 -0400, Brian Sniffen wrote:
>> I put together something like this, visible at
>> https://github.com/briansniffen/notmuch/tree/nmweb/contrib/notmuch-web
>>
>> It's not much of a service.  I am pretty sure it is exploitable---that
>> content in text/html parts of messages can do Bad Things to your
>> session.
>
> I think this is the crux of the problem, right?  I was noticing the
> other day that notmuch's own mail archives are published in pipermail,
> which is *absolutely terrible* compared to dealing with a mailstore with
> notmuch as a frontend.  I'd love to be able to expose the archive to the
> public this way.

For the list archive, we could restrict to displaying text/plain only.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] NEWS: test suite out-of-tree builds

[Jani Nikula]

Out-of-tree builds now work and supersede --root option.
---
 NEWS | 13 +
 1 file changed, 13 insertions(+)

diff --git a/NEWS b/NEWS
index 3c8198d3ef12..c2ebb6447bd0 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,16 @@
+Notmuch 0.26 (UNRELEASED)
+=
+
+Test Suite
+--
+
+Out-of-tree builds
+
+  The test suite now works properly with out-of-tree builds, i.e. with
+  separate source and build directories. The --root option to tests
+  has been dropped. The same can now be achieved more reliably using
+  out-of-tree builds.
+
 Notmuch 0.25.1 (2017-09-11)
 ===
 
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

On Sat, 21 Oct 2017, David Bremner <da...@tethera.net> wrote:
> Jani Nikula <j...@nikula.org> writes:
>
>> The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
>> running each test when the current working directory is not the test
>> subdirectory. Tests with failures would leave their TMP_DIRECTORY
>> directory behind for debugging, and repeated out-of-tree test runs
>> would have old temp directories. (This lead to e.g. T310-emacs.sh
>> hanging because emacs would prompt for overwriting files.)
>>
>> We remove the likely anyway defunct --root test option while at it,
>> just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
>
> This seems to fix the problem I reported, however applying
>
>  
> id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.j...@nikula.org
>
> on top breaks
>
>   ./devel/check-out-of-tree-build.sh test
>
> I get lots of errors along the lines of
>
>   Error opening database at
>   /tmp/tmp.GD3HPpejbL/test/tmp.T350-crypto/mail/.notmuch: No such
>   file or directory
>
> So this patch seems to unbreak in-tree builds (and out of tree builds
> where we copy the whole test hierarchy), and is probably worth applying
> as is, but I wanted to give you a chance to respond before I proceed.

So these two should be applied first, in either order:

id:20171021192141.-1-j...@nikula.org
id:20171021115802.31197-1-j...@nikula.org

And then this should work on top:

id:cd03efa9c93ee54f7e7f7e166079062984ddd658.1506370901.git.j...@nikula.org

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] test: use source path in add_gnupg_home

[Jani Nikula]

Make a distinction between source and build directories.

---

v2 rebase, add_gnupg_home is now in test-lib.sh
---
 test/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 7926d2787710..5274e6e1cf2c 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -109,7 +109,7 @@ add_gnupg_home ()
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
 mkdir -m 0700 "$GNUPGHOME"
-gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
+gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
echo quick-random >> "$GNUPGHOME"/gpg.conf
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: add support for thread: queries

[Jani Nikula]

On Sat, 21 Oct 2017, David Bremner <da...@tethera.net> wrote:
> Jani Nikula <j...@nikula.org> writes:
>
>> On Tue, 17 Oct 2017, David Bremner <da...@tethera.net> wrote:
>>> Jani Nikula <j...@nikula.org> writes:
>>>
>>> The other field processors throw Xapian::QueryParserErrors in case bad
>>> things happen (e.g. failure to read from the database). This seems to be
>>> better than nothing as it allows transmitting some detail to the
>>> user. See e.g. "regex error reporting" in test/T650-regexp-query.sh. And
>>> of course, speaking of tests, this should have one.
>>
>> I'm not sure what you're suggesting. Do you mean I should open-code
>> notmuch_database_find_message() or notmuch_message_get_thread_id() to
>> get at the exceptions thrown within them?
>>
>> BR,
>> Jani.
>
> nothing so fancy. Consider the following bit of query-fp.cc
>
> status = notmuch_database_get_config (notmuch, key.c_str (), );
> if (status) {
>   throw Xapian::QueryParserError ("error looking up key" + name);
> }
>
>
> That could no doubt be improved to report the specifics of `status`, but
> it's already better than silently ignoring the error, as this will
> eventually be reported back to library clients. I agree the exception ->
> error-code -> exception -> error-code  path is pretty gross, but thats
> the price we pay for mostly pretending we're not writing C++.

Okay. Seems like we are in agreement we want this feature, so I'll fix
that up and write some tests when I have a moment.

Thanks,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

On Sat, 21 Oct 2017, Jani Nikula <j...@nikula.org> wrote:
> The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
> running each test when the current working directory is not the test
> subdirectory. Tests with failures would leave their TMP_DIRECTORY
> directory behind for debugging, and repeated out-of-tree test runs
> would have old temp directories. (This lead to e.g. T310-emacs.sh
> hanging because emacs would prompt for overwriting files.)

To clarify, plain 'make test' no longer has its CWD in the test
subdirectory either, leading to the same problem as out-of-tree runs
here.

The problem noticed by David could also be reproduced by running
'test/T310-emacs.sh; test/T310-emacs.sh' but not 'cd test;
./T310-emacs.sh; ./T310-emacs.sh'.

BR,
Jani.

>
> We remove the likely anyway defunct --root test option while at it,
> just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
> ---
>  test/README | 9 -
>  test/test-lib-common.sh | 8 ++--
>  test/test-lib.sh| 3 ---
>  3 files changed, 2 insertions(+), 18 deletions(-)
>
> diff --git a/test/README b/test/README
> index 8e06f44241a6..b378c3ff3c5f 100644
> --- a/test/README
> +++ b/test/README
> @@ -80,15 +80,6 @@ The following command-line options are available when 
> running tests:
>   As the names depend on the tests' file names, it is safe to
>   run the tests with this option in parallel.
>  
> ---root=::
> - This runs the testsuites specified under a separate directory.
> - However, caution is advised, as not all tests are maintained
> - with this relocation in mind, so some tests may behave
> - differently.
> -
> - Pointing this argument at a tmpfs filesystem can improve the
> - speed of the test suite for some users.
> -
>  Certain tests require precomputed databases to complete. You can fetch these
>  databases with
>  
> diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
> index 5e53348a9438..4300eb65418f 100644
> --- a/test/test-lib-common.sh
> +++ b/test/test-lib-common.sh
> @@ -307,13 +307,9 @@ export PATH MANPATH
>  
>  # Test repository
>  test="tmp.$(basename "$0" .sh)"
> -test -n "$root" && test="$root/$test"
> -case "$test" in
> -/*) TMP_DIRECTORY="$test" ;;
> - *) TMP_DIRECTORY="$TEST_DIRECTORY/$test" ;;
> -esac
> +TMP_DIRECTORY="$TEST_DIRECTORY/$test"
>  test ! -z "$debug" || remove_tmp=$TMP_DIRECTORY
> -rm -fr "$test" || {
> +rm -rf "$TMP_DIRECTORY" || {
>   GIT_EXIT_OK=t
>   echo >&6 "FATAL: Cannot prepare test area"
>   exit 1
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 4619c327dd02..7926d2787710 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -160,9 +160,6 @@ do
>   valgrind=t; verbose=t; shift ;;
>   --tee)
>   shift ;; # was handled already
> - --root=*)
> - root=$(expr "z$1" : 'z[^=]*=\(.*\)')
> - shift ;;
>   *)
>   echo "error: unknown test option '$1'" >&2; exit 1 ;;
>   esac
> -- 
> 2.11.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test: remove --root option and fix TMP_DIRECTORY cleanup

[Jani Nikula]

The primary motivation here is to fix TMP_DIRECTORY cleanup prior to
running each test when the current working directory is not the test
subdirectory. Tests with failures would leave their TMP_DIRECTORY
directory behind for debugging, and repeated out-of-tree test runs
would have old temp directories. (This lead to e.g. T310-emacs.sh
hanging because emacs would prompt for overwriting files.)

We remove the likely anyway defunct --root test option while at it,
just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
---
 test/README | 9 -
 test/test-lib-common.sh | 8 ++--
 test/test-lib.sh| 3 ---
 3 files changed, 2 insertions(+), 18 deletions(-)

diff --git a/test/README b/test/README
index 8e06f44241a6..b378c3ff3c5f 100644
--- a/test/README
+++ b/test/README
@@ -80,15 +80,6 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
 
---root=::
-   This runs the testsuites specified under a separate directory.
-   However, caution is advised, as not all tests are maintained
-   with this relocation in mind, so some tests may behave
-   differently.
-
-   Pointing this argument at a tmpfs filesystem can improve the
-   speed of the test suite for some users.
-
 Certain tests require precomputed databases to complete. You can fetch these
 databases with
 
diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh
index 5e53348a9438..4300eb65418f 100644
--- a/test/test-lib-common.sh
+++ b/test/test-lib-common.sh
@@ -307,13 +307,9 @@ export PATH MANPATH
 
 # Test repository
 test="tmp.$(basename "$0" .sh)"
-test -n "$root" && test="$root/$test"
-case "$test" in
-/*) TMP_DIRECTORY="$test" ;;
- *) TMP_DIRECTORY="$TEST_DIRECTORY/$test" ;;
-esac
+TMP_DIRECTORY="$TEST_DIRECTORY/$test"
 test ! -z "$debug" || remove_tmp=$TMP_DIRECTORY
-rm -fr "$test" || {
+rm -rf "$TMP_DIRECTORY" || {
GIT_EXIT_OK=t
echo >&6 "FATAL: Cannot prepare test area"
exit 1
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 4619c327dd02..7926d2787710 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -160,9 +160,6 @@ do
valgrind=t; verbose=t; shift ;;
--tee)
shift ;; # was handled already
-   --root=*)
-   root=$(expr "z$1" : 'z[^=]*=\(.*\)')
-   shift ;;
*)
echo "error: unknown test option '$1'" >&2; exit 1 ;;
esac
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: add support for thread: queries

[Jani Nikula]

On Tue, 17 Oct 2017, David Bremner <da...@tethera.net> wrote:
> Jani Nikula <j...@nikula.org> writes:
>
>
>> +if (strchr (thread.c_str (), '@'))
>> +notmuch_database_find_message (notmuch, thread.c_str (), );
>> +
>> +if (message) {
>> +thread_id = notmuch_message_get_thread_id (message);
>> +notmuch_message_destroy (message);
>> +} else {
>> +thread_id = thread;
>> +}
>> +
>> +return Xapian::Query (term_prefix + thread_id);
>> +}
>
> The other field processors throw Xapian::QueryParserErrors in case bad
> things happen (e.g. failure to read from the database). This seems to be
> better than nothing as it allows transmitting some detail to the
> user. See e.g. "regex error reporting" in test/T650-regexp-query.sh. And
> of course, speaking of tests, this should have one.

I'm not sure what you're suggesting. Do you mean I should open-code
notmuch_database_find_message() or notmuch_message_get_thread_id() to
get at the exceptions thrown within them?

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

nmbug backtrace due to .mailmap

[Jani Nikula]

Presumably because of the .mailmap added in nmbug repository:

commit 5ea99ebcb7d188003bf4854b566f6d21721f5ea1
Author: W. Trevor King 
Date:   Mon Oct 16 10:46:27 2017 -0700

.mailmap: Consolidate David and Tomi's author names

The format is documented in git-shortlog(1).  In both cases, I've
prefered the email used for their most-recent commit.

A   .mailmap

I now get backtrace trying to nmbug pull:

Traceback (most recent call last):
  File "/home/jani/bin/nmbug", line 834, in 
args.func(**kwargs)
  File "/home/jani/bin/nmbug", line 433, in pull
_insist_committed()
  File "/home/jani/bin/nmbug", line 412, in _insist_committed
status = get_status()
  File "/home/jani/bin/nmbug", line 580, in get_status
maybe_deleted = _diff_index(index=index, filter='D')
  File "/home/jani/bin/nmbug", line 658, in _diff_index
for id, tag in _unpack_diff_lines(stream=p.stdout):
  File "/home/jani/bin/nmbug", line 678, in _unpack_diff_lines
'Invalid line in diff: {!r}'.format(line.strip()))
ValueError: Invalid line in diff: u'.mailmap'

I also didn't see any discussion about adding a .mailmap prior to it
being pushed.


BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] lib: add support for thread: queries

[Jani Nikula]

Add support for querying threads using message-ids in addition to
thread-ids. The main benefit is that thread queries via message-ids
are portable across databases, re-indexing, and thread joining, while
thread ids can be somewhat transient. A thread: query can
be shared between notmuch users, while a thread: query may
cease to work after the regular delivery of a message that joins
threads.

What previously required:

$ notmuch search $(notmuch search --output=threads id:)

can now be reduced to:

$ notmuch search thread:

We limit the query to message-ids that have @ to optimize regular
thread-id queries and to avoid collisions with thread-ids which are
guaranteed (or which we can guarantee) to not contain @. This seems
reasonable, as valid message-ids will have @.

The performance penalty for regular thread-id queries seems to be
neglible, and thread queries via message-ids seem to be about 10%
slower than the direct thread-id query counterpart.
---
 doc/man7/notmuch-search-terms.rst |  9 ---
 lib/Makefile.local|  1 +
 lib/database.cc   |  6 -
 lib/thread-fp.cc  | 53 +++
 lib/thread-fp.h   | 39 
 5 files changed, 104 insertions(+), 4 deletions(-)
 create mode 100644 lib/thread-fp.cc
 create mode 100644 lib/thread-fp.h

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index b27f31f7545c..f3723ceb959b 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -52,7 +52,7 @@ indicate user-supplied values):
 
 -  id:
 
--  thread:
+-  thread: or thread:
 
 -  folder:
 
@@ -101,10 +101,13 @@ For **id:**, message ID values are the literal contents 
of the
 Message-ID: header of email messages, but without the '<', '>'
 delimiters.
 
-The **thread:** prefix can be used with the thread ID values that are
+The **thread:** prefix can be used with either the message ID of any
+of the messages in the thread or the thread ID values that are
 generated internally by notmuch (and do not appear in email messages).
 These thread ID values can be seen in the first column of output from
-**notmuch search**
+**notmuch search**. The thread ID values are transient. Thread queries
+by message ID are only available if notmuch is built with **Xapian
+Field Processors** (see below).
 
 The **path:** prefix searches for email messages that are in
 particular directories within the mail store. The directory must be
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 8aa03891d775..123ef04c64a9 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -58,6 +58,7 @@ libnotmuch_cxx_srcs = \
$(dir)/query-fp.cc  \
$(dir)/config.cc\
$(dir)/regexp-fields.cc \
+   $(dir)/thread-fp.cc \
$(dir)/thread.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database.cc b/lib/database.cc
index 35c66939bdcf..82d2dcf22aa0 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -21,6 +21,7 @@
 #include "database-private.h"
 #include "parse-time-vrp.h"
 #include "query-fp.h"
+#include "thread-fp.h"
 #include "regexp-fields.h"
 #include "string-util.h"
 
@@ -258,7 +259,8 @@ prefix_t prefix_table[] = {
 { "directory", "XDIRECTORY",   NOTMUCH_FIELD_NO_FLAGS },
 { "file-direntry", "XFDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
 { "directory-direntry","XDDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "thread","G",NOTMUCH_FIELD_EXTERNAL 
},
+{ "thread","G",NOTMUCH_FIELD_EXTERNAL |
+   NOTMUCH_FIELD_PROCESSOR },
 { "tag",   "K",NOTMUCH_FIELD_EXTERNAL |
NOTMUCH_FIELD_PROCESSOR },
 { "is","K",NOTMUCH_FIELD_EXTERNAL |
@@ -317,6 +319,8 @@ _setup_query_field (const prefix_t *prefix, 
notmuch_database_t *notmuch)
fp = (new DateFieldProcessor())->release ();
else if (STRNCMP_LITERAL(prefix->name, "query") == 0)
fp = (new QueryFieldProcessor (*notmuch->query_parser, 
notmuch))->release ();
+   else if (STRNCMP_LITERAL(prefix->name, "thread") == 0)
+   fp = (new ThreadFieldProcessor (notmuch))->release ();
else
fp = (new RegexpFieldProcessor (prefix->name, prefix->flags,
*notmuch->query_parser, 
notmuch))->release ();
diff --git a/lib/thread-fp.cc b/lib/thread-fp.cc
new file mode 100644
index ..f15aabba18c0
--- /dev/nul

[PATCH] cli/crypto: fix segfault on failed gmime2 crypto context creation

[Jani Nikula]

Commit 1fdc08d0ffab ("cli/crypto: treat failure to create a crypto
context as fatal.") started treating crypto context creation failures
"as fatal", returning NULL from _mime_node_create().

Unfortunately, we do not have NULL checks for _mime_node_create()
failures. The only caller, mime_node_child(), could check and return
NULL (as it's documented to do on errors) but none of the several call
sites have NULL checks either. And none of them really have a trivial
but feasible and graceful way of recovery.

So while the right thing to do would be to handle NULL returns
properly all over the place, and we have other scenarios that do
return NULL from above mentioned functions, the crypto context
creation failure is something that does seem to show up regularly in
some scenarios, revert back to the functionality before commit
1fdc08d0ffab as an interim fix.
---
 mime-node.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mime-node.c b/mime-node.c
index d48be4c46695..27de72fa2f84 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -271,7 +271,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
const char *protocol = g_mime_content_type_get_parameter (content_type, 
"protocol");
cryptoctx = _notmuch_crypto_get_gmime_context (node->ctx->crypto, 
protocol);
if (!cryptoctx)
-   return NULL;
+   return node;
 }
 #endif
 
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/3] cli: add support for --no- prefixed boolean and keyword flag arguments

[Jani Nikula]

On Sat, 14 Oct 2017, William Casarin <j...@jb55.com> wrote:
> Hey Jani,
>
> Patches look good so far, concept ack for sure.
>
>
> Jani Nikula <j...@nikula.org> writes:
>
>> For example, you can use --no-exclude instead of --exclude=false in
>> notmuch show. If we had keyword flag arguments with some flags
>> defaulting to on, say --include=tags in notmuch dump/restore, this
>> would allow --no-include=tags to switch that off while not affecting
>> other flags.
>
> I've been testing it a bit, I can't seem to make this work in this example:
>
> ./notmuch count --no-exclude
>
> After some brief investigation it might be because count is using
> EXCLUDE_true(1) and EXCLUDE_false(0) which are not equal to
> NOTMUCH_EXCLUDE_TRUE(1) and NOTMUCH_EXCLUDE_FALSE(2), but I'm not sure.

*blush* I screwed those enums up. Here's a patch that takes care of both
issues id:20171014201836.4486-1-j...@nikula.org. It's independent of
this series.

BR,
Jani.


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] cli: make notmuch count --exclude a boolean argument

[Jani Nikula]

Commit 0f314c0c99be ("cli: convert notmuch_bool_t to stdbool")
over-eagerly converted EXCLUDE_TRUE and EXCLUDE_FALSE to EXCLUDE_true
and EXCLUDE_false in notmuch-count.c. We could just fix the case back,
but convert the option to an actual boolean argument instead.

We've used a keyword argument rather than a boolean argument for
notmuch count --exclude for five years, since commit 785c1e497f05
("cli: move count to the new --exclude=(true|false|flag) naming
scheme."), "to allow future options to be added more easily". I think
we can conclude future options aren't coming any time soon.
---
 notmuch-count.c | 15 +++
 1 file changed, 3 insertions(+), 12 deletions(-)

diff --git a/notmuch-count.c b/notmuch-count.c
index 1ae7d5146d92..ca05c9793b70 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -27,12 +27,6 @@ enum {
 OUTPUT_FILES,
 };
 
-/* The following is to allow future options to be added more easily */
-enum {
-EXCLUDE_true,
-EXCLUDE_false,
-};
-
 /* Return the number of files matching the query, or -1 for an error */
 static int
 count_files (notmuch_query_t *query)
@@ -160,7 +154,7 @@ notmuch_count_command (notmuch_config_t *config, int argc, 
char *argv[])
 char *query_str;
 int opt_index;
 int output = OUTPUT_MESSAGES;
-int exclude = EXCLUDE_true;
+bool exclude = true;
 const char **search_exclude_tags = NULL;
 size_t search_exclude_tags_length = 0;
 bool batch = false;
@@ -175,10 +169,7 @@ notmuch_count_command (notmuch_config_t *config, int argc, 
char *argv[])
  { "messages", OUTPUT_MESSAGES },
  { "files", OUTPUT_FILES },
  { 0, 0 } } },
-   { .opt_keyword = , .name = "exclude", .keywords =
- (notmuch_keyword_t []){ { "true", EXCLUDE_true },
- { "false", EXCLUDE_false },
- { 0, 0 } } },
+   { .opt_bool = , .name = "exclude" },
{ .opt_bool = _lastmod, .name = "lastmod" },
{ .opt_bool = , .name = "batch" },
{ .opt_string = _file_name, .name = "input" },
@@ -221,7 +212,7 @@ notmuch_count_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 }
 
-if (exclude == EXCLUDE_true) {
+if (exclude) {
search_exclude_tags = notmuch_config_get_search_exclude_tags
(config, _exclude_tags_length);
 }
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] cli: use the negating boolean support for new and insert --no-hooks

[Jani Nikula]

This lets us use the positive hooks variable in code, increasing
clarity.
---
 notmuch-insert.c | 6 +++---
 notmuch-new.c| 8 
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/notmuch-insert.c b/notmuch-insert.c
index 32be74193472..6878313e188f 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -455,7 +455,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 const char *folder = "";
 bool create_folder = false;
 bool keep = false;
-bool no_hooks = false;
+bool hooks = true;
 bool synchronize_flags;
 char *maildir;
 char *newpath;
@@ -466,7 +466,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
{ .opt_string = , .name = "folder" },
{ .opt_bool = _folder, .name = "create-folder" },
{ .opt_bool = , .name = "keep" },
-   { .opt_bool =  _hooks, .name = "no-hooks" },
+   { .opt_bool = , .name = "hooks" },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -573,7 +573,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
}
 }
 
-if (! no_hooks && status == NOTMUCH_STATUS_SUCCESS) {
+if (hooks && status == NOTMUCH_STATUS_SUCCESS) {
/* Ignore hook failures. */
notmuch_run_hook (db_path, "post-insert");
 }
diff --git a/notmuch-new.c b/notmuch-new.c
index 0f50457eb894..a6ca484101ce 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -954,7 +954,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, 
char *argv[])
 int opt_index;
 unsigned int i;
 bool timer_is_active = false;
-bool no_hooks = false;
+bool hooks = true;
 bool quiet = false, verbose = false;
 notmuch_status_t status;
 
@@ -962,7 +962,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, 
char *argv[])
{ .opt_bool = , .name = "quiet" },
{ .opt_bool = , .name = "verbose" },
{ .opt_bool = _files_state.debug, .name = "debug" },
-   { .opt_bool = _hooks, .name = "no-hooks" },
+   { .opt_bool = , .name = "hooks" },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -995,7 +995,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, 
char *argv[])
}
 }
 
-if (!no_hooks) {
+if (hooks) {
ret = notmuch_run_hook (db_path, "pre-new");
if (ret)
return EXIT_FAILURE;
@@ -1160,7 +1160,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, 
char *argv[])
 
 notmuch_database_destroy (notmuch);
 
-if (!no_hooks && !ret && !interrupted)
+if (hooks && !ret && !interrupted)
ret = notmuch_run_hook (db_path, "post-new");
 
 if (ret || interrupted)
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] test: test regexp based new.ignore

[Jani Nikula]

Just some basics.
---
 test/T050-new.sh | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/test/T050-new.sh b/test/T050-new.sh
index 272ed417aa2e..ee9ce08c8e86 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -259,6 +259,28 @@ ln -s i_do_not_exist "${MAIL_DIR}"/broken_link
 output=$(NOTMUCH_NEW 2>&1)
 test_expect_equal "$output" "No new mail."
 
+test_begin_subtest "Ignore files and directories specified in new.ignore 
(regexp)"
+notmuch config set new.ignore ".git" "/^bro.*ink\$/" "/ignored.*file/"
+output=$(NOTMUCH_NEW --debug 2>&1 | sort)
+test_expect_equal "$output" \
+"(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.git
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/broken_link
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/two/three/.git
+(D) add_files, pass 1: explicitly ignoring 
${MAIL_DIR}/one/two/three/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/.git
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/broken_link
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/.git
+(D) add_files, pass 2: explicitly ignoring 
${MAIL_DIR}/one/two/three/ignored_file
+No new mail."
+
 test_begin_subtest "Quiet: No new mail."
 output=$(NOTMUCH_NEW --quiet)
 test_expect_equal "$output" ""
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/3] test: expand argument parsing sanity checks

[Jani Nikula]

Test the various boolean formats and --no- prefixed boolean and
keyword flag arguments.
---
 test/T410-argument-parsing.sh | 28 
 1 file changed, 28 insertions(+)

diff --git a/test/T410-argument-parsing.sh b/test/T410-argument-parsing.sh
index 4a2b25c6486d..243d0241b9b6 100755
--- a/test/T410-argument-parsing.sh
+++ b/test/T410-argument-parsing.sh
@@ -37,4 +37,32 @@ positional arg 1 false
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "--boolean=true"
+$TEST_DIRECTORY/arg-test --boolean=true > OUTPUT
+cat < EXPECTED
+boolean 1
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "--boolean=false"
+$TEST_DIRECTORY/arg-test --boolean=false > OUTPUT
+cat < EXPECTED
+boolean 0
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "--no-boolean"
+$TEST_DIRECTORY/arg-test --no-boolean > OUTPUT
+cat < EXPECTED
+boolean 0
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "--no-flag"
+$TEST_DIRECTORY/arg-test --flag=one --flag=three --no-flag=three > OUTPUT
+cat < EXPECTED
+flags 1
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/2] cli/new: support // in new.ignore

[Jani Nikula]

Add support for using // style regular expressions in
new.ignore, mixed with the old style verbatim file and directory
basenames. The regex is matched against the relative path from the
database path.
---
 doc/man1/notmuch-config.rst |  21 +--
 notmuch-new.c   | 134 
 2 files changed, 140 insertions(+), 15 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 6a51e64f1517..0af86f9beee4 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -75,11 +75,22 @@ The available configuration items are described below.
 Default: ``unread;inbox``.
 
 **new.ignore**
-A list of file and directory names, without path, that will not
-be searched for messages by **notmuch new**. All the files and
-directories matching any of the names specified here will be
-ignored, regardless of the location in the mail store directory
-hierarchy.
+A list to specify files and directories that will not be
+searched for messages by **notmuch new**. Each entry in the
+list is either:
+
+  A file or a directory name, without path, that will be
+  ignored, regardless of the location in the mail store
+  directory hierarchy.
+
+Or:
+
+  A regular expression delimited with // that will be matched
+  against the path of the file or directory relative to the
+  database path. Matching files and directories will be
+  ignored. The beginning and end of string must be explictly
+  anchored. For example, /.*/foo$/ would match "bar/foo" and
+  "bar/baz/foo", but not "foo" or "bar/foobar".
 
 Default: empty list.
 
diff --git a/notmuch-new.c b/notmuch-new.c
index 0f50457eb894..5a262e419b44 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -42,13 +42,17 @@ enum verbosity {
 };
 
 typedef struct {
+const char *db_path;
+
 int output_is_a_tty;
 enum verbosity verbosity;
 bool debug;
 const char **new_tags;
 size_t new_tags_length;
-const char **new_ignore;
-size_t new_ignore_length;
+const char **ignore_verbatim;
+size_t ignore_verbatim_length;
+regex_t *ignore_regex;
+size_t ignore_regex_length;
 
 int total_files;
 int processed_files;
@@ -240,18 +244,125 @@ _special_directory (const char *entry)
 return strcmp (entry, ".") == 0 || strcmp (entry, "..") == 0;
 }
 
+static bool
+_setup_ignore (notmuch_config_t *config, add_files_state_t *state)
+{
+const char **ignore_list, **ignore;
+int nregex = 0, nverbatim = 0;
+const char **verbatim = NULL;
+regex_t *regex = NULL;
+
+ignore_list = notmuch_config_get_new_ignore (config, NULL);
+if (! ignore_list)
+   return true;
+
+for (ignore = ignore_list; *ignore; ignore++) {
+   const char *s = *ignore;
+   size_t len = strlen (s);
+
+   if (len == 0) {
+   fprintf (stderr, "Error: Empty string in new.ignore list\n");
+   return false;
+   }
+
+   if (s[0] == '/') {
+   regex_t *preg;
+   char *r;
+   int rerr;
+
+   if (len < 3 || s[len - 1] != '/') {
+   fprintf (stderr, "Error: Malformed pattern '%s' in 
new.ignore\n",
+s);
+   return false;
+   }
+
+   r = talloc_strndup (config, s + 1, len - 2);
+   regex = talloc_realloc (config, regex, regex_t, nregex + 1);
+   preg = [nregex];
+
+   rerr = regcomp (preg, r, REG_EXTENDED | REG_NOSUB);
+   if (rerr) {
+   size_t error_size = regerror (rerr, preg, NULL, 0);
+   char *error = talloc_size (r, error_size);
+
+   regerror (rerr, preg, error, error_size);
+
+   fprintf (stderr, "Error: Invalid regex '%s' in new.ignore: 
%s\n",
+r, error);
+   return false;
+   }
+   nregex++;
+
+   talloc_free (r);
+   } else {
+   verbatim = talloc_realloc (config, verbatim, const char *,
+  nverbatim + 1);
+   verbatim[nverbatim++] = s;
+   }
+}
+
+state->ignore_regex = regex;
+state->ignore_regex_length = nregex;
+state->ignore_verbatim = verbatim;
+state->ignore_verbatim_length = nverbatim;
+
+return true;
+}
+
+static char *
+_get_relative_path (const char *db_path, const char *dirpath, const char 
*entry)
+{
+size_t db_path_len = strlen (db_path);
+
+/* paranoia? */
+if (strncmp (dirpath, db_path, db_path_len) != 0) {
+   fprintf (stderr, "Warning: '%s' is not a subdirectory of '%s'\n",
+dirpath, db_path);
+   return NULL;
+}
+
+dirpath += db_path_len;
+while (*dirpath == '/')
+   dirpath++;
+
+if (*dirpath)
+   return talloc_asprintf (NULL, "%s/%s", dirpath, entry);
+else
+   return 

[PATCH 1/3] cli: add support for --no- prefixed boolean and keyword flag arguments

[Jani Nikula]

Add transparent support for negating boolean and keyword flag
arguments using --no-argument style on the command line. That is, if
the option description contains a boolean or a keyword flag argument
named "argument", --no-argument will match and negate it.

For boolean arguments this obviously means the logical NOT. For
keyword flag arguments this means bitwise AND of the bitwise NOT,
i.e. masking out the specified bits instead of OR'ing them in.

For example, you can use --no-exclude instead of --exclude=false in
notmuch show. If we had keyword flag arguments with some flags
defaulting to on, say --include=tags in notmuch dump/restore, this
would allow --no-include=tags to switch that off while not affecting
other flags.

As a curiosity, you should be able to warp your brain using
--no-exclude=true meaning false and --no-exclude=false meaning true if
you wish.

Specifying both "argument" and "no-argument" style arguments in the
same option description should be avoided. In this case, --no-argument
would match whichever is specified first, and --argument would only
match "argument".
---
 command-line-arguments.c | 48 +---
 1 file changed, 37 insertions(+), 11 deletions(-)

diff --git a/command-line-arguments.c b/command-line-arguments.c
index 1ff5aae578c6..69ee1cb07f47 100644
--- a/command-line-arguments.c
+++ b/command-line-arguments.c
@@ -11,8 +11,9 @@
 */
 
 static bool
-_process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
-
+_process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next,
+ const char *arg_str, bool negate)
+{
 const notmuch_keyword_t *keywords;
 
 if (next == '\0') {
@@ -24,7 +25,9 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
if (strcmp (arg_str, keywords->name) != 0)
continue;
 
-   if (arg_desc->opt_flags)
+   if (arg_desc->opt_flags && negate)
+   *arg_desc->opt_flags &= ~keywords->value;
+   else if (arg_desc->opt_flags)
*arg_desc->opt_flags |= keywords->value;
else
*arg_desc->opt_keyword = keywords->value;
@@ -39,7 +42,9 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
 }
 
 static bool
-_process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
+_process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next,
+ const char *arg_str, bool negate)
+{
 bool value;
 
 if (next == '\0' || strcmp (arg_str, "true") == 0) {
@@ -51,7 +56,7 @@ _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
return false;
 }
 
-*arg_desc->opt_bool = value;
+*arg_desc->opt_bool = negate ? !value : value;
 
 return true;
 }
@@ -139,6 +144,8 @@ parse_position_arg (const char *arg_str, int pos_arg_index,
 return false;
 }
 
+#define NEGATIVE_PREFIX "no-"
+
 /*
  * Search for a non-positional (i.e. starting with --) argument matching arg,
  * parse a possible value, and assign to *output_var
@@ -155,6 +162,14 @@ parse_option (int argc, char **argv, const 
notmuch_opt_desc_t *options, int opt_
 assert(options);
 
 const char *arg = _arg + 2; /* _arg starts with -- */
+const char *negative_arg = NULL;
+
+/* See if this is a --no-argument */
+if (strlen (arg) > strlen (NEGATIVE_PREFIX) &&
+   strncmp (arg, NEGATIVE_PREFIX, strlen (NEGATIVE_PREFIX)) == 0) {
+   negative_arg = arg + strlen (NEGATIVE_PREFIX);
+}
+
 const notmuch_opt_desc_t *try;
 
 const char *next_arg = NULL;
@@ -171,11 +186,22 @@ parse_option (int argc, char **argv, const 
notmuch_opt_desc_t *options, int opt_
if (! try->name)
continue;
 
-   if (strncmp (arg, try->name, strlen (try->name)) != 0)
+   char next;
+   const char *value;
+   bool negate = false;
+
+   if (strncmp (arg, try->name, strlen (try->name)) == 0) {
+   next = arg[strlen (try->name)];
+   value = arg + strlen (try->name) + 1;
+   } else if (negative_arg && (try->opt_bool || try->opt_flags) &&
+  strncmp (negative_arg, try->name, strlen (try->name)) == 0) {
+   next = negative_arg[strlen (try->name)];
+   value = negative_arg + strlen (try->name) + 1;
+   /* The argument part of --no-argument matches, negate the result. */
+   negate = true;
+   } else {
continue;
-
-   char next = arg[strlen (try->name)];
-   const char *value = arg + strlen(try->name) + 1;
+   }
 
/*
 * If we have not reached the end of the argument (i.e. the
@@ -194,9 +220,9 @@ parse_option (int argc, char **argv, const 
notmuch_opt_desc_t *options, int opt_
 
bool opt_status = false;
if (try->opt_keyword || try->opt_flags)
-   opt_status = _process_keyword_arg (try, next, value);
+   opt_status = 

[PATCH] cli: allow empty strings for notmuch insert --folder argument

[Jani Nikula]

Now that it's easy to add argument specific modifiers in opt
descriptions, add a new .allow_empty field to allow empty strings for
individual string arguments while retaining strict checks
elsewhere. Use this for notmuch insert --folder, where the empty
string means top level folder.

---

This patch addresses id:87y3owr22c@nikula.org
---
 command-line-arguments.c| 2 +-
 command-line-arguments.h| 3 +++
 doc/man1/notmuch-insert.rst | 3 ++-
 notmuch-insert.c| 2 +-
 4 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/command-line-arguments.c b/command-line-arguments.c
index 1ff5aae578c6..db73ca5efb89 100644
--- a/command-line-arguments.c
+++ b/command-line-arguments.c
@@ -81,7 +81,7 @@ _process_string_arg (const notmuch_opt_desc_t *arg_desc, char 
next, const char *
fprintf (stderr, "Option \"%s\" needs a string argument.\n", 
arg_desc->name);
return false;
 }
-if (arg_str[0] == '\0') {
+if (arg_str[0] == '\0' && ! arg_desc->allow_empty) {
fprintf (stderr, "String argument for option \"%s\" must be 
non-empty.\n", arg_desc->name);
return false;
 }
diff --git a/command-line-arguments.h b/command-line-arguments.h
index 76ca4dcbb276..c0228f7cb634 100644
--- a/command-line-arguments.h
+++ b/command-line-arguments.h
@@ -32,6 +32,9 @@ typedef struct notmuch_opt_desc {
 /* Optional, if non-NULL, set to true if the option is present. */
 bool *present;
 
+/* Optional, allow empty strings for opt_string. */
+bool allow_empty;
+
 /* Must be set for opt_keyword and opt_flags. */
 const struct notmuch_keyword *keywords;
 } notmuch_opt_desc_t;
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index f79600d6571f..2f2466a6588b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -34,7 +34,8 @@ Supported options for **insert** include
 ``--folder=<``\ folder\ **>**
 Deliver the message to the specified folder, relative to the
 top-level directory given by the value of **database.path**. The
-default is to deliver to the top-level directory.
+default is the empty string, which means delivering to the
+top-level directory.
 
 ``--create-folder``
 Try to create the folder named by the ``--folder`` option, if it
diff --git a/notmuch-insert.c b/notmuch-insert.c
index 32be74193472..cff74731 100644
--- a/notmuch-insert.c
+++ b/notmuch-insert.c
@@ -463,7 +463,7 @@ notmuch_insert_command (notmuch_config_t *config, int argc, 
char *argv[])
 unsigned int i;
 
 notmuch_opt_desc_t options[] = {
-   { .opt_string = , .name = "folder" },
+   { .opt_string = , .name = "folder", .allow_empty = true },
{ .opt_bool = _folder, .name = "create-folder" },
{ .opt_bool = , .name = "keep" },
{ .opt_bool =  _hooks, .name = "no-hooks" },
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 0/3] nmbug:

[Jani Nikula]

On Tue, 10 Oct 2017, "W. Trevor King"  wrote:
> I expect the best time to make that bump is just before we cut our
> next notmuch release.  Once we do, we can update the wiki page [1] to
> suggest nmbug 0.3+ and remove the checkout step from “Getting
> started”.

Could we add a "nmbug required version" metadata to the repository, and
add a version check in nmbug? Obviously this doesn't help with 0.2, but,
if we added this at 0.3, we could require a new nmbug version on the
clients.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [bug] [emacs] notmuch-show: names not shown on some mailing lists

[Jani Nikula]

On Tue, 10 Oct 2017, William Casarin  wrote:
> Hey there,
>
> Here's something I've noticed in some mailing list threads in notmuch-show:
>
>   https://jb55.com/s/81d7c740ef60984d.png
>
> It doesn't look like it is showing the correct name.
> Looking at the raw message, the From line looks like this:
>
>   From: Some Person via Some-mailinglist
> 
>   Reply-To: Some Person 
>
> Any way to handle these situations gracefully? Somehow show
> `Some Person` instead of `somemailingl...@lists.linuxfoundation.org`?

The information comes from the cli. Can you reproduce this using some
notmuch show --format=sexp --body=false query, perhaps on just one of
the messages?

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 04/10] index: implement notmuch_indexopts_t with try_decrypt

[Jani Nikula]

On Mon, 09 Oct 2017, Daniel Kahn Gillmor <d...@fifthhorseman.net> wrote:
> On Sat 2017-09-23 19:10:18 +0300, Jani Nikula wrote:
>>> --- a/lib/indexopts.c
>>> +++ b/lib/indexopts.c
>>> @@ -21,9 +21,27 @@
>>>  #include "notmuch-private.h"
>>>  
>>>  notmuch_indexopts_t *
>>> -notmuch_database_get_default_indexopts (notmuch_database_t unused (*db))
>>> +notmuch_database_get_default_indexopts (notmuch_database_t *db)
>>>  {
>>> -return NULL;
>>> +return talloc_zero (db, notmuch_indexopts_t);
>>
>> I wonder about the lifetime of indexopts. Should default indexopts be
>> part of the db object, so that your caller above doesn't have to
>> alloc/destroy it for every file?
>
> The caller doesn't have to alloc/destroy it for every file, they can
> alloc it once and pass it in for every file.

My point was, if the caller doesn't do it, it'll get alloced and
destroyed per file.

BR,
Jani.


>
> I'd rather not have the indexopts be part of the db object itself
> explicitly, because i can imagine a longer-running program wanting to
> create two indexopts objects, configuring them differently, and re-using
> one or the other without wanting to modify the database itself.
>
>> Our library interface has a leaky abstraction of the talloc hierarchical
>> refcounting. We don't talk about it in any of the docs, some of it is
>> implied, most of it is completely surprising if the library interface
>> user assumes a traditional C memory allocation model without
>> refcounting.
>
> right, probably the most surprising thing would be if the user got a
> default indexopts from one database object, and then tried to apply it
> to another database object, after having deleted the first database
> object.
>
> However, we *do* talk about it in the docs now, so i think we've given
> fair warning:
>
> -
> /**
>  * get the current default indexing options for a given database.
>  *
>  * This object will survive until the database itself is destroyed,
>  * but the caller may also release it earlier with
>  * notmuch_indexopts_destroy.
>  *
>  * This object represents a set of options on how a message can be
>  * added to the index.  At the moment it is a featureless stub.
>  *
>  * @since libnotmuch 5.1 (notmuch 0.26)
>  */
> notmuch_indexopts_t *
> notmuch_database_get_default_indexopts (notmuch_database_t *db);
> -
>
> --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] lib: convert notmuch_bool_t to stdbool internally

[Jani Nikula]

C99 stdbool turned 18 this year. There really is no reason to use our
own, except in the library interface for backward
compatibility. Convert the lib internally to stdbool.
---
 lib/add-message.cc  | 12 +-
 lib/built-with.c|  2 +-
 lib/config.cc   | 10 
 lib/database-private.h  |  6 ++---
 lib/database.cc | 30 
 lib/directory.cc|  8 +++
 lib/filenames.c |  2 +-
 lib/index.cc|  6 ++---
 lib/message-file.c  | 10 
 lib/message-private.h   |  2 +-
 lib/message-property.cc |  6 ++---
 lib/message.cc  | 62 -
 lib/messages.c  |  4 ++--
 lib/notmuch-private.h   | 13 ++-
 lib/query.cc| 42 -
 lib/string-map.c| 26 ++---
 lib/thread.cc   | 14 +--
 17 files changed, 128 insertions(+), 127 deletions(-)

diff --git a/lib/add-message.cc b/lib/add-message.cc
index 73bde7faf049..bce10a0f614c 100644
--- a/lib/add-message.cc
+++ b/lib/add-message.cc
@@ -404,7 +404,7 @@ static notmuch_status_t
 _notmuch_database_link_message (notmuch_database_t *notmuch,
notmuch_message_t *message,
notmuch_message_file_t *message_file,
-   notmuch_bool_t is_ghost)
+   bool is_ghost)
 {
 void *local = talloc_new (NULL);
 notmuch_status_t status;
@@ -467,7 +467,7 @@ notmuch_database_index_file (notmuch_database_t *notmuch,
 notmuch_message_t *message = NULL;
 notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS, ret2;
 notmuch_private_status_t private_status;
-notmuch_bool_t is_ghost = FALSE, is_new = FALSE;
+bool is_ghost = false, is_new = false;
 
 const char *date;
 const char *from, *to, *subject;
@@ -510,12 +510,12 @@ notmuch_database_index_file (notmuch_database_t *notmuch,
/* We cannot call notmuch_message_get_flag for a new message */
switch (private_status) {
case NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND:
-   is_ghost = FALSE;
-   is_new = TRUE;
+   is_ghost = false;
+   is_new = true;
break;
case NOTMUCH_PRIVATE_STATUS_SUCCESS:
is_ghost = notmuch_message_get_flag (message, 
NOTMUCH_MESSAGE_FLAG_GHOST);
-   is_new = FALSE;
+   is_new = false;
break;
default:
ret = COERCE_STATUS (private_status,
@@ -551,7 +551,7 @@ notmuch_database_index_file (notmuch_database_t *notmuch,
 } catch (const Xapian::Error ) {
_notmuch_database_log (notmuch, "A Xapian exception occurred adding 
message: %s.\n",
 error.get_msg().c_str());
-   notmuch->exception_reported = TRUE;
+   notmuch->exception_reported = true;
ret = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
goto DONE;
 }
diff --git a/lib/built-with.c b/lib/built-with.c
index 2f1f0b5c1bf3..27384bd044bc 100644
--- a/lib/built-with.c
+++ b/lib/built-with.c
@@ -31,6 +31,6 @@ notmuch_built_with (const char *name)
 } else if (STRNCMP_LITERAL (name, "retry_lock") == 0) {
return HAVE_XAPIAN_DB_RETRY_LOCK;
 } else {
-   return FALSE;
+   return false;
 }
 }
diff --git a/lib/config.cc b/lib/config.cc
index 0703b9bb8fde..da71c16e5adc 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -56,7 +56,7 @@ notmuch_database_set_config (notmuch_database_t *notmuch,
db->set_metadata (CONFIG_PREFIX + key, value);
 } catch (const Xapian::Error ) {
status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
-   notmuch->exception_reported = TRUE;
+   notmuch->exception_reported = true;
_notmuch_database_log (notmuch, "Error: A Xapian exception occurred 
setting metadata: %s\n",
   error.get_msg().c_str());
 }
@@ -74,7 +74,7 @@ _metadata_value (notmuch_database_t *notmuch,
value = notmuch->xapian_db->get_metadata (CONFIG_PREFIX + key);
 } catch (const Xapian::Error ) {
status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
-   notmuch->exception_reported = TRUE;
+   notmuch->exception_reported = true;
_notmuch_database_log (notmuch, "Error: A Xapian exception occurred 
getting metadata: %s\n",
   error.get_msg().c_str());
 }
@@ -128,7 +128,7 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
 } catch (const Xapian::Error ) {
_notmuch_database_log (notmuch, "A Xapian exception occurred getting 
metadata iterator: %s.\n",
   error.get_msg().c_str());
-   notmuch->exception_reported = TRUE;
+   notmuch->exception_reported = true;
status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
 }
 
@@ -145,9 +145,9 @@ notmuch_bool_t
 notmuch_config_list_valid (notmuch_config_list_t *metadata)
 {
 if (metadata->iterator == 

[PATCH 1/2] cli: convert notmuch_bool_t to stdbool

[Jani Nikula]

C99 stdbool turned 18 this year. There really is no reason to use our
own, except in the library interface for backward
compatibility. Convert the cli and test binaries to stdbool.
---
 command-line-arguments.c | 58 +++
 command-line-arguments.h | 10 ---
 crypto.c |  6 ++---
 debugger.c   |  8 +++---
 gmime-filter-reply.c | 32 +++---
 mime-node.c  | 10 +++
 notmuch-client.h | 35 
 notmuch-compact.c|  2 +-
 notmuch-config.c | 28 +--
 notmuch-count.c  | 18 ++---
 notmuch-dump.c   | 10 +++
 notmuch-insert.c | 70 
 notmuch-new.c| 30 ++---
 notmuch-reply.c  | 30 ++---
 notmuch-restore.c|  4 +--
 notmuch-search.c | 24 -
 notmuch-setup.c  |  6 ++---
 notmuch-show.c   | 52 +--
 notmuch-tag.c|  6 ++---
 notmuch.c| 12 -
 sprinter-json.c  | 18 ++---
 sprinter-sexp.c  | 16 +--
 sprinter-text.c  |  6 ++---
 sprinter.h   |  6 ++---
 tag-util.c   | 24 -
 tag-util.h   |  8 +++---
 test/arg-test.c  |  6 ++---
 test/hex-xcode.c |  8 +++---
 test/random-corpus.c |  2 +-
 29 files changed, 275 insertions(+), 270 deletions(-)

diff --git a/command-line-arguments.c b/command-line-arguments.c
index 3fa8d9044966..1ff5aae578c6 100644
--- a/command-line-arguments.c
+++ b/command-line-arguments.c
@@ -6,11 +6,11 @@
 
 /*
   Search the array of keywords for a given argument, assigning the
-  output variable to the corresponding value.  Return FALSE if nothing
+  output variable to the corresponding value.  Return false if nothing
   matches.
 */
 
-static notmuch_bool_t
+static bool
 _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
 
 const notmuch_keyword_t *keywords;
@@ -29,64 +29,64 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
else
*arg_desc->opt_keyword = keywords->value;
 
-   return TRUE;
+   return true;
 }
 if (next != '\0')
fprintf (stderr, "Unknown keyword argument \"%s\" for option 
\"%s\".\n", arg_str, arg_desc->name);
 else
fprintf (stderr, "Option \"%s\" needs a keyword argument.\n", 
arg_desc->name);
-return FALSE;
+return false;
 }
 
-static notmuch_bool_t
+static bool
 _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
-notmuch_bool_t value;
+bool value;
 
 if (next == '\0' || strcmp (arg_str, "true") == 0) {
-   value = TRUE;
+   value = true;
 } else if (strcmp (arg_str, "false") == 0) {
-   value = FALSE;
+   value = false;
 } else {
fprintf (stderr, "Unknown argument \"%s\" for (boolean) option 
\"%s\".\n", arg_str, arg_desc->name);
-   return FALSE;
+   return false;
 }
 
 *arg_desc->opt_bool = value;
 
-return TRUE;
+return true;
 }
 
-static notmuch_bool_t
+static bool
 _process_int_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 char *endptr;
 if (next == '\0' || arg_str[0] == '\0') {
fprintf (stderr, "Option \"%s\" needs an integer argument.\n", 
arg_desc->name);
-   return FALSE;
+   return false;
 }
 
 *arg_desc->opt_int = strtol (arg_str, , 10);
 if (*endptr == '\0')
-   return TRUE;
+   return true;
 
 fprintf (stderr, "Unable to parse argument \"%s\" for option \"%s\" as an 
integer.\n",
 arg_str, arg_desc->name);
-return FALSE;
+return false;
 }
 
-static notmuch_bool_t
+static bool
 _process_string_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 if (next == '\0') {
fprintf (stderr, "Option \"%s\" needs a string argument.\n", 
arg_desc->name);
-   return FALSE;
+   return false;
 }
 if (arg_str[0] == '\0') {
fprintf (stderr, "String argument for option \"%s\" must be 
non-empty.\n", arg_desc->name);
-   return FALSE;
+   return false;
 }
 *arg_desc->opt_string = arg_str;
-return TRUE;
+return true;
 }
 
 /* Return number of non-NULL opt_* fields in opt_desc. */
@@ -102,8 +102,8 @@ static int _opt_set_count (const notmuch_opt_desc_t 
*opt_desc)
!!opt_desc->opt_position;
 }
 
-/* Return TRUE if opt_desc is valid. */
-static notmuch_bool_t _opt_valid (const notmuch_opt_desc_t *opt_desc)
+/* Return true if opt_desc is valid. */
+static bool _opt_valid (const notmuch_opt_desc_t *opt_desc)
 {
 int n = _opt_set_count (opt_desc);
 
@@ -115,11 +115,11 @@ static notmuch_bool_t _opt_valid (const 
notmuch_opt_desc_t *opt_desc)
 }
 
 /*
-  

[PATCH v2 0/2] cli: support regex in new.ignore

[Jani Nikula]

The rest of the patches from id:cover.1504280923.git.j...@nikula.org
with review addressed. Be more strict about parsing new.ignore, and give
sensible error messages on failures. Also improve the text in man page a
bit.

BR,
Jani.

Jani Nikula (2):
  cli/new: support // in new.ignore
  test: test regexp based new.ignore

 doc/man1/notmuch-config.rst |  21 +--
 notmuch-new.c   | 134 
 test/T050-new.sh|  22 
 3 files changed, 162 insertions(+), 15 deletions(-)

-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 1/2] cli/new: support // in new.ignore

[Jani Nikula]

Add support for using // style regular expressions in
new.ignore, mixed with the old style verbatim file and directory
basenames. The regex is matched against the relative path from the
database path.
---
 doc/man1/notmuch-config.rst |  21 +--
 notmuch-new.c   | 134 
 2 files changed, 140 insertions(+), 15 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 6a51e64f1517..0af86f9beee4 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -75,11 +75,22 @@ The available configuration items are described below.
 Default: ``unread;inbox``.
 
 **new.ignore**
-A list of file and directory names, without path, that will not
-be searched for messages by **notmuch new**. All the files and
-directories matching any of the names specified here will be
-ignored, regardless of the location in the mail store directory
-hierarchy.
+A list to specify files and directories that will not be
+searched for messages by **notmuch new**. Each entry in the
+list is either:
+
+  A file or a directory name, without path, that will be
+  ignored, regardless of the location in the mail store
+  directory hierarchy.
+
+Or:
+
+  A regular expression delimited with // that will be matched
+  against the path of the file or directory relative to the
+  database path. Matching files and directories will be
+  ignored. The beginning and end of string must be explictly
+  anchored. For example, /.*/foo$/ would match "bar/foo" and
+  "bar/baz/foo", but not "foo" or "bar/foobar".
 
 Default: empty list.
 
diff --git a/notmuch-new.c b/notmuch-new.c
index 50597b75c07e..7e12b73a7883 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -42,13 +42,17 @@ enum verbosity {
 };
 
 typedef struct {
+const char *db_path;
+
 int output_is_a_tty;
 enum verbosity verbosity;
 notmuch_bool_t debug;
 const char **new_tags;
 size_t new_tags_length;
-const char **new_ignore;
-size_t new_ignore_length;
+const char **ignore_verbatim;
+size_t ignore_verbatim_length;
+regex_t *ignore_regex;
+size_t ignore_regex_length;
 
 int total_files;
 int processed_files;
@@ -240,18 +244,125 @@ _special_directory (const char *entry)
 return strcmp (entry, ".") == 0 || strcmp (entry, "..") == 0;
 }
 
+static notmuch_bool_t
+_setup_ignore (notmuch_config_t *config, add_files_state_t *state)
+{
+const char **ignore_list, **ignore;
+int nregex = 0, nverbatim = 0;
+const char **verbatim = NULL;
+regex_t *regex = NULL;
+
+ignore_list = notmuch_config_get_new_ignore (config, NULL);
+if (! ignore_list)
+   return TRUE;
+
+for (ignore = ignore_list; *ignore; ignore++) {
+   const char *s = *ignore;
+   size_t len = strlen (s);
+
+   if (len == 0) {
+   fprintf (stderr, "Error: Empty string in new.ignore list\n");
+   return FALSE;
+   }
+
+   if (s[0] == '/') {
+   regex_t *preg;
+   char *r;
+   int rerr;
+
+   if (len < 3 || s[len - 1] != '/') {
+   fprintf (stderr, "Error: Malformed pattern '%s' in 
new.ignore\n",
+s);
+   return FALSE;
+   }
+
+   r = talloc_strndup (config, s + 1, len - 2);
+   regex = talloc_realloc (config, regex, regex_t, nregex + 1);
+   preg = [nregex];
+
+   rerr = regcomp (preg, r, REG_EXTENDED | REG_NOSUB);
+   if (rerr) {
+   size_t error_size = regerror (rerr, preg, NULL, 0);
+   char *error = talloc_size (r, error_size);
+
+   regerror (rerr, preg, error, error_size);
+
+   fprintf (stderr, "Error: Invalid regex '%s' in new.ignore: 
%s\n",
+r, error);
+   return FALSE;
+   }
+   nregex++;
+
+   talloc_free (r);
+   } else {
+   verbatim = talloc_realloc (config, verbatim, const char *,
+  nverbatim + 1);
+   verbatim[nverbatim++] = s;
+   }
+}
+
+state->ignore_regex = regex;
+state->ignore_regex_length = nregex;
+state->ignore_verbatim = verbatim;
+state->ignore_verbatim_length = nverbatim;
+
+return TRUE;
+}
+
+static char *
+_get_relative_path (const char *db_path, const char *dirpath, const char 
*entry)
+{
+size_t db_path_len = strlen (db_path);
+
+/* paranoia? */
+if (strncmp (dirpath, db_path, db_path_len) != 0) {
+   fprintf (stderr, "Warning: '%s' is not a subdirectory of '%s'\n",
+dirpath, db_path);
+   return NULL;
+}
+
+dirpath += db_path_len;
+while (*dirpath == '/')
+   dirpath++;
+
+if (*dirpath)
+   return talloc_asprintf (NULL, "%s/%s", dirpath, entry);
+

[PATCH v2 2/2] test: test regexp based new.ignore

[Jani Nikula]

Just some basics.
---
 test/T050-new.sh | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/test/T050-new.sh b/test/T050-new.sh
index 272ed417aa2e..ee9ce08c8e86 100755
--- a/test/T050-new.sh
+++ b/test/T050-new.sh
@@ -259,6 +259,28 @@ ln -s i_do_not_exist "${MAIL_DIR}"/broken_link
 output=$(NOTMUCH_NEW 2>&1)
 test_expect_equal "$output" "No new mail."
 
+test_begin_subtest "Ignore files and directories specified in new.ignore 
(regexp)"
+notmuch config set new.ignore ".git" "/^bro.*ink\$/" "/ignored.*file/"
+output=$(NOTMUCH_NEW --debug 2>&1 | sort)
+test_expect_equal "$output" \
+"(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.git
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/broken_link
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file
+(D) add_files, pass 1: explicitly ignoring ${MAIL_DIR}/one/two/three/.git
+(D) add_files, pass 1: explicitly ignoring 
${MAIL_DIR}/one/two/three/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/.git
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/.ignored_hidden_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/broken_link
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/ignored_file
+(D) add_files, pass 2: explicitly ignoring ${MAIL_DIR}/one/two/three/.git
+(D) add_files, pass 2: explicitly ignoring 
${MAIL_DIR}/one/two/three/ignored_file
+No new mail."
+
 test_begin_subtest "Quiet: No new mail."
 output=$(NOTMUCH_NEW --quiet)
 test_expect_equal "$output" ""
-- 
2.11.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch-emacs: Fcc to top-level directory given by database.path

[Jani Nikula]

On Sat, 30 Sep 2017, Jani Nikula <j...@nikula.org> wrote:
> On Sat, 23 Sep 2017, David Bremner <da...@tethera.net> wrote:
>> Do you happen to know if it calls with an empty string as the folder
>> name? It would be consistent with searching for that to insert at the
>> top level.
>
> notmuch insert --folder= or --folder="" does not work:
>
> String argument for option "folder" must be non-empty.
> Unrecognized option: --folder=
>
> It seems that our argument parser doesn't accept empty strings at all,
> which is a bit of a limitation. It would be logical for the empty string
> to work here, and AFAICT the notmuch insert code would handle this if
> the argument was let through the parsing.

Turns out this is trivial to address on top of the arg parsing series:
id:20171002162552.4827-1-j...@nikula.org.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch