quoting: (was: Re: bug#56442: gnus-search-run-search: Hits notmuch command line length limits)
On Sat, Jul 09 2022, Eric Abrahamsen wrote:
>
> Huh, I tried this a couple months ago with a more complicated query,
> like:
>
> thread:{from:bob or from:jane}
>
> and I remember notmuch barking at me about spaces or the "or" or
> something -- anyway I got the impression that it couldn't accept
> multi-part queries inside the "thread:{}" syntax. But looking at your
> patch maybe I just needed to quote differently?
I am not sure whether this is relevant, but I give a shot...
I tried the following queries from command line:
$ notmuch search 'thread:"{from:bob or from:jane}"'
thread:0b91 2009-01-20 [6/6] ...
thread:15fa 2008-11-26 [2/2] ...
and
$ notmuch search 'from:bob or from:jane}':
thread:0b91 2009-01-20 [1/6] ...
thread:15fa 2008-11-26 [1/2] ...
The outer '':s protect from shell expansion so what is
inside those goes verbatim to notmuch and from there to
xapian query parser (IIRC).
Former includes all messages in thread where at least
one message matched to the query and latter just those
messages that match the query.
Like Sean mentioned in another message, Xapian converts ""
to " in query string (probably something irrelevant in this
discussion) the thread:"{...}" syntax binds the whole expression
to this thread: prefix (Why {...} is not enough is beyond me ;D)
Tomi
>
> Anyway I'll give this a test this weekend. It would be *very* nice if we
> could use this syntax rather than the ugly home-grown one.
>
> Thanks for the report!
>
> Eric
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: bug#56442: gnus-search-run-search: Hits notmuch command line length limits
Hello, On Sat 09 Jul 2022 at 09:11AM -07, Eric Abrahamsen wrote: > But looking at your patch maybe I just needed to quote differently? Right, notmuch has this unusual " -> "" quoting thing. > Anyway I'll give this a test this weekend. It would be *very* nice if > we could use this syntax rather than the ugly home-grown one. Let me know the results of your test, and if it's also continuing to work for me by then, I'll install the change. Thanks for taking a look. -- Sean Whitton ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: bug#56442: gnus-search-run-search: Hits notmuch command line length limits
Sean Whitton writes:
> Hello,
>
> I'm running a pretty innocent notmuch query over a fairly small Maildir:
>
> "((List:debian-devel.lists.debian.org) or ... or
> (List:debian-haskell.lists.debian.org) or
> (List:debconf-discuss.lists.debian.org)) and (not path:annex/**)"
>
> but gnus-search-run-search fails to return any results. The reason is
> that Gnus first runs the query with --output=threads to obtain a list of
> thread ids, and then runs another query with --output=files and a query
> constructed from the output of the first query: "thread:d9d0
> or thread:d9e0 or thread:d268 or ..."
>
> The resulting command fails completely:
>
> emacs: /usr/bin/notmuch: Argument list too long
>
> Instead of running two searches like this, we can just surround the
> whole query like this: "thread:{QUERY}". The manual says it's exactly
> equivalent:
>
> ... the user should think of the query thread:{} as
> expanding to all of the thread IDs which match ; not‐
> much then performs a second search using the expanded query.
>
> This should be faster, too, with only running a single external command.
> Here is the patch I'm thinking I'll apply, if anyone has comments.
Huh, I tried this a couple months ago with a more complicated query,
like:
thread:{from:bob or from:jane}
and I remember notmuch barking at me about spaces or the "or" or
something -- anyway I got the impression that it couldn't accept
multi-part queries inside the "thread:{}" syntax. But looking at your
patch maybe I just needed to quote differently?
Anyway I'll give this a test this weekend. It would be *very* nice if we
could use this syntax rather than the ugly home-grown one.
Thanks for the report!
Eric
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
bug#56442: gnus-search-run-search: Hits notmuch command line length limits
Hello,
I'm running a pretty innocent notmuch query over a fairly small Maildir:
"((List:debian-devel.lists.debian.org) or ... or
(List:debian-haskell.lists.debian.org) or
(List:debconf-discuss.lists.debian.org)) and (not path:annex/**)"
but gnus-search-run-search fails to return any results. The reason is
that Gnus first runs the query with --output=threads to obtain a list of
thread ids, and then runs another query with --output=files and a query
constructed from the output of the first query: "thread:d9d0
or thread:d9e0 or thread:d268 or ..."
The resulting command fails completely:
emacs: /usr/bin/notmuch: Argument list too long
Instead of running two searches like this, we can just surround the
whole query like this: "thread:{QUERY}". The manual says it's exactly
equivalent:
... the user should think of the query thread:{} as
expanding to all of the thread IDs which match ; not‐
much then performs a second search using the expanded query.
This should be faster, too, with only running a single external command.
Here is the patch I'm thinking I'll apply, if anyone has comments.
--
Sean Whitton
>From d62398cc27abd317d31d8f07e151a82081444217 Mon Sep 17 00:00:00 2001
From: Sean Whitton
Date: Thu, 7 Jul 2022 09:34:04 -0700
Subject: [PATCH] Use notmuch thread:{} operator instead of performing two
searches
This also avoids some command line length limit problems.
* gnus-search.el
(gnus-search-run-search (engine gnus-search-notmuch)): Delete method.
(gnus-search-indexed-search-command (engine gnus-search-notmuch)):
When searching for threads, wrap whole query in thread:{} operator.
Always use --output=files, never --output=threads.
---
lisp/gnus/gnus-search.el | 46 +---
1 file changed, 5 insertions(+), 41 deletions(-)
diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index 369df81d9b..53b6d1b4c6 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -1672,43 +1672,6 @@ gnus-search-transform-expression
(format "date:%s.." (notmuch-date (cdr expr
(t (ignore-errors (cl-call-next-method))
-(cl-defmethod gnus-search-run-search :around ((engine gnus-search-notmuch)
- server query groups)
- "Handle notmuch's thread-search routine."
- ;; Notmuch allows for searching threads, but only using its own
- ;; thread ids. That means a thread search is a \"double-bounce\":
- ;; once to find the relevant thread ids, and again to find the
- ;; actual messages. This method performs the first \"bounce\".
- (if (alist-get 'thread query)
- (with-slots (program proc-buffer) engine
- (let* ((qstring
- (gnus-search-make-query-string engine query))
- (cp-list (gnus-search-indexed-search-command
- engine qstring query groups))
- thread-ids proc)
- (with-current-buffer proc-buffer
- (erase-buffer)
- (setq proc (apply #'start-process (format "search-%s" server)
- proc-buffer program cp-list))
- (while (process-live-p proc)
- (accept-process-output proc))
-(goto-char (point-min))
- (while (re-search-forward
-"^thread:\\([^[:space:]\n]+\\)"
-(point-max) t)
- (cl-pushnew (match-string 1) thread-ids :test #'equal)))
- (cl-call-next-method
- engine server
- ;; If we found threads, completely replace the query with
- ;; our new thread-based one.
- (if thread-ids
- `((query . ,(mapconcat (lambda (thrd)
-(concat "thread:" thrd))
- thread-ids " or ")))
- query)
- nil)))
-(cl-call-next-method engine server query groups)))
-
(cl-defmethod gnus-search-indexed-search-command ((engine gnus-search-notmuch)
(qstring string)
query _groups)
@@ -1721,13 +1684,14 @@ gnus-search-indexed-search-command
(append
(list (format "--config=%s" config-file)
"search"
- (if thread
- "--output=threads"
- "--output=files"))
+ "--output=files")
(unless thread '("--duplicate=1"))
(when limit (list (format "--limit=%d" limit)))
switches
- (list qstring)
+ (list (if thread
+ (format "thread:\"{%s}\""
+ (string-replace "\"" "\"\"" qstring))
+ qstring))
;;; Mairix interface
--
2.30.2
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
