Re: Adding config cache to notmuch_database_t breaks python-cffi test
David Bremner writes: > David Bremner writes: > >> I hope Floris (or someone) can tell me what is going on here, I don't >> understand the memory management in the CFFI bindings very well. >> >> As part of my attempt to remodel config handling, I've added a cache >> for the configuration information stored in the database. This doesn't >> seem to break any other tests, and valgrind doesn't spot any memory >> errors. >> >> Here's the output from pytest > > Oh, I bet I know what it is. I need to update the cache when calling > notmuch_database_set_config. It's just a surprise that nothing in our > test suite does a read after write for configuration information in the > same database session. > Yep, that was it, sorry for the noise. I guess I kindof panicked at not understanding what was going on in the bindings tests. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Adding config cache to notmuch_database_t breaks python-cffi test
David Bremner writes: > I hope Floris (or someone) can tell me what is going on here, I don't > understand the memory management in the CFFI bindings very well. > > As part of my attempt to remodel config handling, I've added a cache > for the configuration information stored in the database. This doesn't > seem to break any other tests, and valgrind doesn't spot any memory > errors. > > Here's the output from pytest Oh, I bet I know what it is. I need to update the cache when calling notmuch_database_set_config. It's just a surprise that nothing in our test suite does a read after write for configuration information in the same database session. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] nmbug: notmuch-report: set both background and foreground colors
Whenever setting background color, set also corresponding
foreground color. Don't expect default foreground color to
be #000 (or something close); user may have changed it.
---
I have set default background color #000 and default foreground
color #ddd.
Just used firefox style editor to add those lines to
nmbug.tethera.net output and it fixed the problem I had with
it (black, instead of bright grey on bright backrounds). Body
background on that site is black for me, which works very well.
devel/nmbug/notmuch-report | 2 ++
1 file changed, 2 insertions(+)
diff --git a/devel/nmbug/notmuch-report b/devel/nmbug/notmuch-report
index 18a0bc70..9a6a31cc 100755
--- a/devel/nmbug/notmuch-report
+++ b/devel/nmbug/notmuch-report
@@ -370,9 +370,11 @@ header_template = config['meta'].get('header',
'''
border-bottom-right-radius: {border_radius};
}}
tbody:nth-child(4n+1) tr td {{
+ color: #000;
background-color: #ffd96e;
}}
tbody:nth-child(4n+3) tr td {{
+ color: #000;
background-color: #bce;
}}
hr {{
--
2.29.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 3/6] lib: factor out prefix related code to its own file
Tomi Ollila writes: > > Good stuff! is this extra line above intentional ? > > (otherwise I did not find anything else, but I may have > got tired looking through the series ;/) > > Tomi Nope, good catch. Dropped the extra whitespace in git. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] lib: cache configuration information from database
The main goal is to allow configuration information to be temporarily
overridden by a separate config file. That will require further
changes not in this commit.
The performance impact is unclear, and will depend on the balance
between number of queries and number of distinct metadata items read
on the first call to n_d_get_config.
---
lib/config.cc | 45 +--
lib/database-private.h| 3 +++
lib/notmuch-private.h | 3 +++
lib/open.cc | 2 +-
test/T562-lib-database.sh | 2 +-
5 files changed, 47 insertions(+), 8 deletions(-)
diff --git a/lib/config.cc b/lib/config.cc
index 0b760dbc..7f97e528 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -84,17 +84,25 @@ notmuch_database_get_config (notmuch_database_t *notmuch,
const char *key,
char **value)
{
-std::string strval;
+const char* stored_val;
notmuch_status_t status;
+if (! notmuch->config) {
+ if ((status = _notmuch_config_load_from_database (notmuch)))
+ return status;
+}
+
if (! value)
return NOTMUCH_STATUS_NULL_POINTER;
-status = _metadata_value (notmuch, key, strval);
-if (status)
- return status;
-
-*value = strdup (strval.c_str ());
+stored_val = _notmuch_string_map_get (notmuch->config, key);
+if (! stored_val) {
+ /* XXX in principle this API should be fixed so empty string
+* is distinguished from not found */
+ *value = strdup("");
+} else {
+ *value = strdup (stored_val);
+}
return NOTMUCH_STATUS_SUCCESS;
}
@@ -201,3 +209,28 @@ notmuch_config_list_destroy (notmuch_config_list_t *list)
{
talloc_free (list);
}
+
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t *notmuch)
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+notmuch_config_list_t *list;
+
+if (notmuch->config == NULL)
+ notmuch->config = _notmuch_string_map_create (notmuch);
+
+if (unlikely(notmuch->config == NULL))
+ return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
+status = notmuch_database_get_config_list (notmuch, "", );
+if (status)
+ return status;
+
+for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next
(list)) {
+ _notmuch_string_map_append (notmuch->config,
+ notmuch_config_list_key (list),
+ notmuch_config_list_value (list));
+}
+
+return status;
+}
diff --git a/lib/database-private.h b/lib/database-private.h
index c9bc712b..d83cf0d0 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -228,6 +228,9 @@ struct _notmuch_database {
* here, but at least they are small */
notmuch_string_map_t *user_prefix;
notmuch_string_map_t *user_header;
+
+/* Cached and possibly overridden configuration */
+notmuch_string_map_t *config;
};
/* Prior to database version 3, features were implied by the database
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 57ec7f72..0f26b371 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -699,6 +699,9 @@ struct _notmuch_indexopts {
#define EMPTY_STRING(s) ((s)[0] == '\0')
+/* config.cc */
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t * db);
NOTMUCH_END_DECLS
#ifdef __cplusplus
diff --git a/lib/open.cc b/lib/open.cc
index 1b17e63a..c04e6f77 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -90,7 +90,7 @@ notmuch_database_open_verbose (const char *path,
notmuch->exception_reported = false;
notmuch->status_string = NULL;
notmuch->path = talloc_strdup (notmuch, path);
-
+notmuch->config = NULL;
strip_trailing (notmuch->path, '/');
notmuch->writable_xapian_db = NULL;
diff --git a/test/T562-lib-database.sh b/test/T562-lib-database.sh
index dd4f2566..37ca16ab 100755
--- a/test/T562-lib-database.sh
+++ b/test/T562-lib-database.sh
@@ -350,7 +350,7 @@ cat < EXPECTED
== stdout ==
1
== stderr ==
-Error: A Xapian exception occurred getting metadata: Database has been closed
+A Xapian exception occurred getting metadata iterator: Database has been
closed.
EOF
test_expect_equal_file EXPECTED OUTPUT
--
2.29.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Adding config cache to notmuch_database_t breaks python-cffi test
I hope Floris (or someone) can tell me what is going on here, I don't
understand the memory management in the CFFI bindings very well.
As part of my attempt to remodel config handling, I've added a cache
for the configuration information stored in the database. This doesn't
seem to break any other tests, and valgrind doesn't spot any memory
errors.
Here's the output from pytest
__ TestIter.test_del ___
self =
db = Database(path=/tmp/pytest-of-bremner/pytest-50/test_del0,
mode=Mode.READ_WRITE)
def test_del(self, db):
db.config['spam'] = 'ham'
assert db.config.get('spam') == 'ham'
del db.config['spam']
> assert db.config.get('spam') is None
E AssertionError: assert 'ham' is None
E+ where 'ham' = >('spam')
E+where > =
.get
E+ where
= Database(path=/tmp/pytest-of-bremner/pytest-50/test_del0,
mode=Mode.READ_WRITE).config
tests/test_config.py:56: AssertionError
=== short test summary info
FAILED tests/test_config.py::TestIter::test_del - AssertionError: assert 'ham...
1 failed, 155 passed in 29.25 seconds =
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 3/6] lib: factor out prefix related code to its own file
On Sun, Dec 20 2020, David Bremner wrote:
> Reduce the size of database.cc, and limit the scope of prefix_table,
> make sure it's accessed via a well-defined internal API.
> ---
> lib/Makefile.local | 4 +-
> lib/database-private.h | 7 ++
> lib/database.cc| 203 ++-
> lib/prefix.cc | 210 +
> 4 files changed, 229 insertions(+), 195 deletions(-)
> create mode 100644 lib/prefix.cc
>
> diff --git a/lib/Makefile.local b/lib/Makefile.local
> index 04418fa8..3aa9e80f 100644
> --- a/lib/Makefile.local
> +++ b/lib/Makefile.local
> @@ -60,7 +60,9 @@ libnotmuch_cxx_srcs = \
> $(dir)/regexp-fields.cc \
> $(dir)/thread.cc \
> $(dir)/thread-fp.cc \
> - $(dir)/features.cc
> + $(dir)/features.cc \
> + $(dir)/prefix.cc
> +
Good stuff! is this extra line above intentional ?
(otherwise I did not find anything else, but I may have
got tired looking through the series ;/)
Tomi
>
> libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o)
> $(libnotmuch_cxx_srcs:.cc=.o)
>
> diff --git a/lib/database-private.h b/lib/database-private.h
> index 2d220811..c9bc712b 100644
> --- a/lib/database-private.h
> +++ b/lib/database-private.h
> @@ -277,4 +277,11 @@ _notmuch_database_parse_features (const void *ctx, const
> char *features, unsigne
> char *
> _notmuch_database_print_features (const void *ctx, unsigned int features);
>
> +/* prefix.cc */
> +notmuch_status_t
> +_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch);
> +
> +notmuch_status_t
> +_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch);
> +
> #endif
> diff --git a/lib/database.cc b/lib/database.cc
> index 4a477bd7..defa3062 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -263,80 +263,6 @@ _notmuch_database_mode (notmuch_database_t *notmuch)
> * same thread.
> */
>
> -/* With these prefix values we follow the conventions published here:
> - *
> - * https://xapian.org/docs/omega/termprefixes.html
> - *
> - * as much as makes sense. Note that I took some liberty in matching
> - * the reserved prefix values to notmuch concepts, (for example, 'G'
> - * is documented as "newsGroup (or similar entity - e.g. a web forum
> - * name)", for which I think the thread is the closest analogue in
> - * notmuch. This in spite of the fact that we will eventually be
> - * storing mailing-list messages where 'G' for "mailing list name"
> - * might be even a closer analogue. I'm treating the single-character
> - * prefixes preferentially for core notmuch concepts (which will be
> - * nearly universal to all mail messages).
> - */
> -
> -static const
> -prefix_t prefix_table[] = {
> -/* name term prefix flags */
> -{ "type", "T",NOTMUCH_FIELD_NO_FLAGS },
> -{ "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS },
> -{ "replyto","XREPLYTO", NOTMUCH_FIELD_NO_FLAGS },
> -{ "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS },
> -{ "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS },
> -{ "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS },
> -{ "body", "", NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROBABILISTIC },
> -{ "thread", "G",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "tag","K",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "is", "K",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "id", "Q",NOTMUCH_FIELD_EXTERNAL },
> -{ "mid","Q",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "path", "P",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "property", "XPROPERTY",NOTMUCH_FIELD_EXTERNAL },
> -/*
> - * Unconditionally add ':' to reduce potential ambiguity with
> - * overlapping prefixes and/or terms that start with capital
> - * letters. See Xapian document termprefixes.html for related
> - * discussion.
> - */
> -{ "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "date", NULL, NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "query", NULL, NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "from", "XFROM",NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROBABILISTIC |
> - NOTMUCH_FIELD_PROCESSOR },
> -{ "to", "XTO", NOTMUCH_FIELD_EXTERNAL |
> - NOTMUCH_FIELD_PROBABILISTIC },
> -{
[PATCH 6/6] lib: factor out notmuch_database_open* related code to own file
Reduce the size of database.cc, and prepare for extending the database
opening API
---
lib/Makefile.local | 3 +-
lib/database.cc| 219 -
lib/open.cc| 218
3 files changed, 220 insertions(+), 220 deletions(-)
create mode 100644 lib/open.cc
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 3aa9e80f..ddd169dc 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -61,7 +61,8 @@ libnotmuch_cxx_srcs = \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
$(dir)/features.cc \
- $(dir)/prefix.cc
+ $(dir)/prefix.cc\
+ $(dir)/open.cc
libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database.cc b/lib/database.cc
index defa3062..0f4e2ff9 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,10 +19,6 @@
*/
#include "database-private.h"
-#include "parse-time-vrp.h"
-#include "query-fp.h"
-#include "thread-fp.h"
-#include "regexp-fields.h"
#include "string-util.h"
#include
@@ -50,12 +46,6 @@ typedef struct {
#define STRINGIFY(s) _SUB_STRINGIFY (s)
#define _SUB_STRINGIFY(s) #s
-#if HAVE_XAPIAN_DB_RETRY_LOCK
-#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK)
-#else
-#define DB_ACTION Xapian::DB_CREATE_OR_OPEN
-#endif
-
#define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception
(__location__, message, error)
static void
@@ -594,215 +584,6 @@ _notmuch_database_new_revision (notmuch_database_t
*notmuch)
return new_revision;
}
-notmuch_status_t
-notmuch_database_open (const char *path,
- notmuch_database_mode_t mode,
- notmuch_database_t **database)
-{
-char *status_string = NULL;
-notmuch_status_t status;
-
-status = notmuch_database_open_verbose (path, mode, database,
- _string);
-
-if (status_string) {
- fputs (status_string, stderr);
- free (status_string);
-}
-
-return status;
-}
-
-notmuch_status_t
-notmuch_database_open_verbose (const char *path,
- notmuch_database_mode_t mode,
- notmuch_database_t **database,
- char **status_string)
-{
-notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
-void *local = talloc_new (NULL);
-notmuch_database_t *notmuch = NULL;
-char *notmuch_path, *xapian_path, *incompat_features;
-char *message = NULL;
-struct stat st;
-int err;
-unsigned int i, version;
-static int initialized = 0;
-
-if (path == NULL) {
- message = strdup ("Error: Cannot open a database for a NULL path.\n");
- status = NOTMUCH_STATUS_NULL_POINTER;
- goto DONE;
-}
-
-if (path[0] != '/') {
- message = strdup ("Error: Database path must be absolute.\n");
- status = NOTMUCH_STATUS_PATH_ERROR;
- goto DONE;
-}
-
-if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch")))
{
- message = strdup ("Out of memory\n");
- status = NOTMUCH_STATUS_OUT_OF_MEMORY;
- goto DONE;
-}
-
-err = stat (notmuch_path, );
-if (err) {
- IGNORE_RESULT (asprintf (, "Error opening database at %s: %s\n",
-notmuch_path, strerror (errno)));
- status = NOTMUCH_STATUS_FILE_ERROR;
- goto DONE;
-}
-
-if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path,
"xapian"))) {
- message = strdup ("Out of memory\n");
- status = NOTMUCH_STATUS_OUT_OF_MEMORY;
- goto DONE;
-}
-
-/* Initialize the GLib type system and threads */
-#if ! GLIB_CHECK_VERSION (2, 35, 1)
-g_type_init ();
-#endif
-
-/* Initialize gmime */
-if (! initialized) {
- g_mime_init ();
- initialized = 1;
-}
-
-notmuch = talloc_zero (NULL, notmuch_database_t);
-notmuch->exception_reported = false;
-notmuch->status_string = NULL;
-notmuch->path = talloc_strdup (notmuch, path);
-
-strip_trailing (notmuch->path, '/');
-
-notmuch->writable_xapian_db = NULL;
-notmuch->atomic_nesting = 0;
-notmuch->view = 1;
-try {
- string last_thread_id;
- string last_mod;
-
- if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) {
- notmuch->writable_xapian_db = new Xapian::WritableDatabase
(xapian_path,
-
DB_ACTION);
- notmuch->xapian_db = notmuch->writable_xapian_db;
- } else {
- notmuch->xapian_db = new Xapian::Database (xapian_path);
- }
-
- /* Check version. As of database version 3, we represent
-* changes in terms of features, so assume a version bump
-* means a dramatically incompatible change. */
- version = notmuch_database_get_version (notmuch);
- if (version >
[PATCH 4/6] lib/config: delay setting talloc destructor
If Xapian has thrown an exception, it is not safe to invoke the
destructor when freeing the list struct.
---
lib/config.cc | 11 +--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/lib/config.cc b/lib/config.cc
index efab01e4..0b760dbc 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -133,8 +133,15 @@ notmuch_database_get_config_list (notmuch_database_t
*notmuch,
*out = list;
DONE:
-if (status && list)
- talloc_free (list);
+if (status) {
+ if (list) {
+ talloc_free (list);
+ if (status != NOTMUCH_STATUS_XAPIAN_EXCEPTION)
+ _notmuch_config_list_destroy (list);
+ }
+} else {
+ talloc_set_destructor (list, _notmuch_config_list_destroy);
+}
return status;
}
--
2.29.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH 5/6] test: add regression test for searching with alternate config
Make sure upcoming changes to config handling do not break command
line specification.
---
test/T140-excludes.sh | 10 ++
1 file changed, 10 insertions(+)
diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index 0cf69975..cef07095 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -39,6 +39,16 @@ deleted_id=$gen_msg_id
output=$(notmuch search subject:deleted | notmuch_search_sanitize)
test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite;
Not deleted (inbox unread)"
+test_begin_subtest "Search, exclude \"deleted\" messages; alternate config
file"
+cp ${NOTMUCH_CONFIG} alt-config
+notmuch config set search.exclude_tags
+notmuch --config=alt-config search subject:deleted | notmuch_search_sanitize >
OUTPUT
+cp alt-config ${NOTMUCH_CONFIG}
+cat < EXPECTED
+thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
test_begin_subtest "Search, exclude \"deleted\" messages from message search"
output=$(notmuch search --output=messages subject:deleted |
notmuch_search_sanitize)
test_expect_equal "$output" "id:$not_deleted_id"
--
2.29.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH 1/6] test: use keys with group 'test' in T590-libconfig
In a future commit we want to interoperate better with glib KeyFiles,
which need groups for all keys.
---
test/T590-libconfig.sh | 36 ++--
1 file changed, 18 insertions(+), 18 deletions(-)
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 360e45b0..8c34acf9 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -28,18 +28,18 @@ EOF
test_begin_subtest "notmuch_database_{set,get}_config"
cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
{
- EXPECT0(notmuch_database_set_config (db, "testkey1", "testvalue1"));
- EXPECT0(notmuch_database_set_config (db, "testkey2", "testvalue2"));
- EXPECT0(notmuch_database_get_config (db, "testkey1", ));
- printf("testkey1 = %s\n", val);
- EXPECT0(notmuch_database_get_config (db, "testkey2", ));
- printf("testkey2 = %s\n", val);
+ EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1"));
+ EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2"));
+ EXPECT0(notmuch_database_get_config (db, "test.key1", ));
+ printf("test.key1 = %s\n", val);
+ EXPECT0(notmuch_database_get_config (db, "test.key2", ));
+ printf("test.key2 = %s\n", val);
}
EOF
cat <<'EOF' >EXPECTED
== stdout ==
-testkey1 = testvalue1
-testkey2 = testvalue2
+test.key1 = testvalue1
+test.key2 = testvalue2
== stderr ==
EOF
test_expect_equal_file EXPECTED OUTPUT
@@ -93,8 +93,8 @@ EOF
cat <<'EOF' >EXPECTED
== stdout ==
aaabefore beforeval
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
zzzafter afterval
== stderr ==
EOF
@@ -115,8 +115,8 @@ EOF
cat <<'EOF' >EXPECTED
== stdout ==
aaabefore 1
-testkey1 1
-testkey2 1
+test.key1 1
+test.key2 1
zzzafter 1
== stderr ==
EOF
@@ -126,7 +126,7 @@ test_begin_subtest "notmuch_database_get_config_list: one
prefix"
cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
{
notmuch_config_list_t *list;
- EXPECT0(notmuch_database_get_config_list (db, "testkey", ));
+ EXPECT0(notmuch_database_get_config_list (db, "test.key", ));
for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next
(list)) {
printf("%s %s\n", notmuch_config_list_key (list),
notmuch_config_list_value(list));
}
@@ -135,8 +135,8 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
EOF
cat <<'EOF' >EXPECTED
== stdout ==
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
== stderr ==
EOF
test_expect_equal_file EXPECTED OUTPUT
@@ -152,8 +152,8 @@ cat <<'EOF' >EXPECTED
#notmuch-dump batch-tag:3 config
#@ aaabefore beforeval
#@ key%20with%20spaces value,%20with,%20spaces%21
-#@ testkey1 testvalue1
-#@ testkey2 testvalue2
+#@ test.key1 testvalue1
+#@ test.key2 testvalue2
#@ zzzafter afterval
EOF
test_expect_equal_file EXPECTED OUTPUT
@@ -162,7 +162,7 @@ test_begin_subtest "restore config"
notmuch dump --include=config >EXPECTED
cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
{
-EXPECT0(notmuch_database_set_config (db, "testkey1", "mutatedvalue"));
+EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
}
EOF
notmuch restore --include=config
[PATCH 3/6] lib: factor out prefix related code to its own file
Reduce the size of database.cc, and limit the scope of prefix_table,
make sure it's accessed via a well-defined internal API.
---
lib/Makefile.local | 4 +-
lib/database-private.h | 7 ++
lib/database.cc| 203 ++-
lib/prefix.cc | 210 +
4 files changed, 229 insertions(+), 195 deletions(-)
create mode 100644 lib/prefix.cc
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 04418fa8..3aa9e80f 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -60,7 +60,9 @@ libnotmuch_cxx_srcs = \
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
- $(dir)/features.cc
+ $(dir)/features.cc \
+ $(dir)/prefix.cc
+
libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database-private.h b/lib/database-private.h
index 2d220811..c9bc712b 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -277,4 +277,11 @@ _notmuch_database_parse_features (const void *ctx, const
char *features, unsigne
char *
_notmuch_database_print_features (const void *ctx, unsigned int features);
+/* prefix.cc */
+notmuch_status_t
+_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch);
+
+notmuch_status_t
+_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch);
+
#endif
diff --git a/lib/database.cc b/lib/database.cc
index 4a477bd7..defa3062 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -263,80 +263,6 @@ _notmuch_database_mode (notmuch_database_t *notmuch)
* same thread.
*/
-/* With these prefix values we follow the conventions published here:
- *
- * https://xapian.org/docs/omega/termprefixes.html
- *
- * as much as makes sense. Note that I took some liberty in matching
- * the reserved prefix values to notmuch concepts, (for example, 'G'
- * is documented as "newsGroup (or similar entity - e.g. a web forum
- * name)", for which I think the thread is the closest analogue in
- * notmuch. This in spite of the fact that we will eventually be
- * storing mailing-list messages where 'G' for "mailing list name"
- * might be even a closer analogue. I'm treating the single-character
- * prefixes preferentially for core notmuch concepts (which will be
- * nearly universal to all mail messages).
- */
-
-static const
-prefix_t prefix_table[] = {
-/* nameterm prefix flags */
-{ "type", "T",NOTMUCH_FIELD_NO_FLAGS },
-{ "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS },
-{ "replyto","XREPLYTO", NOTMUCH_FIELD_NO_FLAGS },
-{ "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS },
-{ "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS },
-{ "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS },
-{ "body", "", NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC },
-{ "thread", "G",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "tag","K",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "is", "K",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "id", "Q",NOTMUCH_FIELD_EXTERNAL },
-{ "mid","Q",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "path", "P",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "property", "XPROPERTY",NOTMUCH_FIELD_EXTERNAL },
-/*
- * Unconditionally add ':' to reduce potential ambiguity with
- * overlapping prefixes and/or terms that start with capital
- * letters. See Xapian document termprefixes.html for related
- * discussion.
- */
-{ "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "date", NULL, NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "query", NULL, NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROCESSOR },
-{ "from", "XFROM",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC |
- NOTMUCH_FIELD_PROCESSOR },
-{ "to", "XTO", NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC },
-{ "attachment", "XATTACHMENT", NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC },
-{ "mimetype", "XMIMETYPE",NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC },
-{ "subject","XSUBJECT", NOTMUCH_FIELD_EXTERNAL |
- NOTMUCH_FIELD_PROBABILISTIC |
- NOTMUCH_FIELD_PROCESSOR },
-};
-
-static void
-_setup_query_field_default
v2 preliminaries for merged config
This is a respin of id:20200812224931.3611801-1-da...@tethera.net, rebased against master and with one typo in a commit message fixed. I'm going to mark these as ready to merge since they are primarily code movement, and recieved no feedback in the previous 4 months. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH 2/6] lib: factor out feature name related code.
database.cc is uncomfortably large, and some of the static data
structures do not need to be shared as much as they are.
This is a somewhat small piece to factor out, but it will turn out to
be helpful to further refactoring.
---
lib/Makefile.local | 3 +-
lib/database-private.h | 14 +
lib/database.cc| 116 +
lib/features.cc| 114
4 files changed, 131 insertions(+), 116 deletions(-)
create mode 100644 lib/features.cc
diff --git a/lib/Makefile.local b/lib/Makefile.local
index a6400126..04418fa8 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -59,7 +59,8 @@ libnotmuch_cxx_srcs = \
$(dir)/config.cc\
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
- $(dir)/thread-fp.cc
+ $(dir)/thread-fp.cc \
+ $(dir)/features.cc
libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database-private.h b/lib/database-private.h
index 041602cd..2d220811 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -32,6 +32,8 @@
#include "notmuch-private.h"
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
#ifdef SILENCE_XAPIAN_DEPRECATION_WARNINGS
#define XAPIAN_DEPRECATED(D) D
#endif
@@ -263,4 +265,16 @@ _notmuch_database_find_doc_ids (notmuch_database_t
*notmuch,
const char *value,
Xapian::PostingIterator *begin,
Xapian::PostingIterator *end);
+
+#define NOTMUCH_DATABASE_VERSION 3
+
+/* features.cc */
+
+_notmuch_features
+_notmuch_database_parse_features (const void *ctx, const char *features,
unsigned int version,
+ char mode, char **incompat_out);
+
+char *
+_notmuch_database_print_features (const void *ctx, unsigned int features);
+
#endif
diff --git a/lib/database.cc b/lib/database.cc
index 75189685..4a477bd7 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -39,8 +39,6 @@
using namespace std;
-#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
-
typedef struct {
const char *name;
const char *prefix;
@@ -458,41 +456,6 @@ _notmuch_database_prefix (notmuch_database_t *notmuch,
const char *name)
return NULL;
}
-static const struct {
-/* NOTMUCH_FEATURE_* value. */
-_notmuch_features value;
-/* Feature name as it appears in the database. This name should
- * be appropriate for displaying to the user if an older version
- * of notmuch doesn't support this feature. */
-const char *name;
-/* Compatibility flags when this feature is declared. */
-const char *flags;
-} feature_names[] = {
-{ NOTMUCH_FEATURE_FILE_TERMS,
- "multiple paths per message", "rw" },
-{ NOTMUCH_FEATURE_DIRECTORY_DOCS,
- "relative directory paths", "rw" },
-/* Header values are not required for reading a database because a
- * reader can just refer to the message file. */
-{ NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES,
- "from/subject/message-ID in database", "w" },
-{ NOTMUCH_FEATURE_BOOL_FOLDER,
- "exact folder:/path: search", "rw" },
-{ NOTMUCH_FEATURE_GHOSTS,
- "mail documents for missing messages", "w" },
-/* Knowledge of the index mime-types are not required for reading
- * a database because a reader will just be unable to query
- * them. */
-{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
- "indexed MIME types", "w" },
-{ NOTMUCH_FEATURE_LAST_MOD,
- "modification tracking", "w" },
-/* Existing databases will work fine for all queries not involving
- * 'body:' */
-{ NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY,
- "index body and headers separately", "w" },
-};
-
const char *
notmuch_status_to_string (notmuch_status_t status)
{
@@ -817,83 +780,6 @@ _notmuch_database_new_revision (notmuch_database_t
*notmuch)
return new_revision;
}
-/* Parse a database features string from the given database version.
- * Returns the feature bit set.
- *
- * For version < 3, this ignores the features string and returns a
- * hard-coded set of features.
- *
- * If there are unrecognized features that are required to open the
- * database in mode (which should be 'r' or 'w'), return a
- * comma-separated list of unrecognized but required features in
- * *incompat_out suitable for presenting to the user. *incompat_out
- * will be allocated from ctx.
- */
-static _notmuch_features
-_parse_features (const void *ctx, const char *features, unsigned int version,
-char mode, char **incompat_out)
-{
-_notmuch_features res = static_cast<_notmuch_features>(0);
-unsigned int namelen, i;
-size_t llen = 0;
-const char *flags;
-
-/* Prior to database version 3, features were implied by the
- * version number. */
-if (version == 0)
- return NOTMUCH_FEATURES_V0;
-else if (version == 1)
- return
