Re: Inhibiting gnus-get-new-news at startup
Eric S Fraga writes: > On Tuesday, 2 Jan 2024 at 21:34, Husain Alshehhi wrote: >> I might have spoken too soon. It appears that the issue is not with nntp >> groups, but with nnrss. I ran the profile and saw the following: > > Which is why I switched to elfeed for rss feeds. I would prefer to use > gnus for everything (email, news, blogs) but the performance hit is too > high. elfeed is very fast in comparison. There will be a newly-implemented backend for Atom landing in Gnus before too long, and I think we're hoping to make it work for RSS as well. I'll try to make sure that it does better with situations like this. I also find nnrss too slow for practical use, and hope it's something fixable.
Re: Inhibiting gnus-get-new-news at startup, Re: Inhibiting gnus-get-new-news at startup, Re: Inhibiting gnus-get-new-news at startup
James Thomas writes: > Husain Alshehhi wrote: > >> Is there a way to prevent gnus to get new news at startup time, and >> only do it manually? > > Eric Abrahamsen wrote: > >> The command `gnus-no-server' will likely do what you want. > > IIRC this will only stop the primary select method. By default it will only activate groups at levels 1 and 2. In my case, anyway, that only connects to the nndraft server.
Re: Inhibiting gnus-get-new-news at startup
Husain Alshehhi writes: > Hello, > > When gnus starts (M-x gnus), it automatically get new news. This is a > reasonable default. But when I start gnus multiple times to look up a > message, I have to wait few seconds for it to start. Is there a way to > prevent gnus to get new news at startup time, and only do it manually? The command `gnus-no-server' will likely do what you want.
Re: Gnus and authinfo
"Otto J. Makela" writes: > Sorry if this is a very much FAQ item, but I really don't know where > else I could ask this question. > > With an operating system upgrade (to Fedora Linux 39) I also got a emacs > upgrade (to version 29.1) Some time in the ancient past I've simply set > > (setq gnus-select-method '(nntp "news.eternal-september.org")) > > in my .gnurc, and this still seems to work to select this as my default > nntp server. However, for some reason, the gnus 5.13 that came with this > emacs version no longer seems to read the plain-text .authinfo file from > my home directory. > > So at every gnus start time it asks me to manually enter the username > and password for this nntp server. What has changed, has the plain-text > file been depreciated in favor of the pgp-encrypted version? I couldn't > find any information on such a change. Both encrypted and unencrypted versions are supported; maybe the `auth-sources' option only points at the encrypted version?
Re: Gnus threads too many articles together
John Haman writes: > Everyday I get an email from my bank saying what my account balance > is. These are all different emails, but Gnus puts them all in one > thread, I think because they have the same subject. If I go into my > bank group and insert old articles, I get > > > O Jan 24, 2023 Alliant Alerts 0 Account Balance Notification > O Jan 24, 2023 Alliant Alerts 0 ╰─► > O Jan 25, 2023 Alliant Alerts 0 ╰─► > O Jan 27, 2023 Alliant Alerts 0 ╰─► > O Jan 30, 2023 Alliant Alerts 0 ╰─► > O Jan 31, 2023 Alliant Alerts 0 ╰─► > > > I don't know the most informative way to ask this question, but is there a > way to make Gnus just thread distinct conversations together (like Outlook.) > > I'd like to not disable threads entirely. By default, `gnus-summary-thread-gathering-function' is set to the function `gnus-gather-threads-by-subject', which produces the behavior you're seeing. Try setting it to `gnus-gather-threads-by-references' instead.
Re: How to search for ticked articles
Julien Cubizolles writes: > Julien Cubizolles writes: > > >> I'm using the imap search engine. When I shut off search parsing, >> the "FLAGGED" search query does indeed return the ticked articles, when >> "mark:flag" didn't. But I tried again (with search parsing on) with the >> query "mark:flagged" and it works ! > > Actually, I was wrong, I'm using notmuch to search a maildir group. The > results, with gnus-search-use-parsed-queries turned off, are : > > No results with : > mark:flag > tag:flag > flagged > > Success with : > mark:flagged > tag:flagged > FLAGGED Is this a maildir that used to be managed by IMAP (like Dovecot)? I can't believe that notmuch uses a "FLAGGED" notation on its own. Nor does it use a "mark" search term. Are you sure it isn't just finding messages with "mark:flagged" in them?
Re: Correct ordering of items in mail threads
Adam Sjøgren writes: > Eric writes: > >> I meant the articles don't always arrive at Gnus in the order they >> were originally authored. > > Got it :-) > > I have configured Gnus to sort a bunch of groups by date instead of by > number, and I can't feel the difference (in speed) -- this on a 4 year > old laptop. > > Maybe I'm just growing slower ;-) Or all this just seems to matter less!
Re: Correct ordering of items in mail threads
Adam Sjøgren writes: > Eric writes: > >> sorting by number _usually_ means sorting by arrival time, but >> there's no contract there. > > When do the numbers given to articles in Gnus not reflect _arrival_ > time? Sorry, that was backwards -- I meant the articles don't always arrive at Gnus in the order they were originally authored.
Re: Correct ordering of items in mail threads
Pankaj Jangid writes: > Eric S Fraga writes: > >> On Saturday, 21 Oct 2023 at 10:41, Pankaj Jangid wrote: >>> ;; Threading >>> '(gnus-thread-sort-functions >>>'(gnus-thread-sort-by-number)) >>> '(gnus-summary-thread-gathering-function >>>'gnus-gather-threads-by-references) >>> >> >> I have >> >> (setq gnus-sort-gathered-threads-function 'gnus-thread-sort-by-date) >> >> along with similar (but slightly different) settings for the variables >> you've set. This variable says how to sort *within* a thread which may >> be what you are looking for. > > In fact, I wanted to use 'gnus-thread-sort-by-date for > gnus-thread-sort-functions. But the docstring discouraged me. > > Also note that sorting by date is often much slower than sorting by > number, and the sorting order is very similar. (Sorting by date means > sorting by the time the message was sent, sorting by number means > sorting by arrival time.) I think this is the trade-off, though -- sorting by number _usually_ means sorting by arrival time, but there's no contract there. It's entirely possible for messages to arrive out-of-order, and when they do, you get a weird display. I think a lot of what was slow in Gnus is gradually becoming less slow (with things like native compilation and whatnot) and you might try sorting by date and just seeing how bad it is.
Re: How to search for ticked articles
Julien Cubizolles writes: > Eric Abrahamsen writes: > >> Julien Cubizolles writes: >> >>> I used to use the "mark:flag" search query to look for ticked articles >>> with gnus-search. I recently noticed that it doesn't return any articles >>> anymore in groups with ticked articles. >>> >>> What exactly does the "flag" mark represent, and is there a better way >>> to look for dormant or ticked articles ? >> >> These things are all search-engine dependent. Notmuch will turn >> "mark:flag" into "tag:flag", and IMAP will turn it into "FLAGGED". >> >> What search engine are you using in this case? The first thing to try >> is always shutting off search parsing (either fully, by setting >> `gnus-search-use-parsed-queries' to nil, or temporarily, using the >> prefix argument). Then get the search working using the search engine's >> native syntax. Then come back here and tell me how it went :) > > I'm using the imap search engine. When I shut off search parsing, > the "FLAGGED" search query does indeed return the ticked articles, when > "mark:flag" didn't. But I tried again (with search parsing on) with the > query "mark:flagged" and it works ! Hmm, that doesn't seem right; you can see that both "mark:flag" and "mark:flagged" get transformed into the same IMAP search term, namely "FLAGGED". (let ((srv (make-instance 'gnus-search-imap)) (flag (gnus-search-parse-query "mark:flagged")) (flagged (gnus-search-parse-query "mark:flag"))) (format "%s == %s" (gnus-search-transform srv flag) (gnus-search-transform srv flagged))) => "FLAGGED == FLAGGED" I just tried it with an IMAP server, and all of "mark:flag", "mark:flagged", and "mark:!" were transformed into "FLAGGED" and produced the right search results. If you have a moment, you could try edebugging the `gnus-search-imap-search-command' function, and seeing what's happening there. The QUERY parameter should already be "FLAGGED" at that point. Thanks, Eric
Re: Does Gnus read and sync the read with IMAP?
Björn Bidar writes: > Dan Christensen writes: > >> On Oct 20, 2023, Eric Abrahamsen wrote: >> >>> Dan Christensen writes: >>> >>>> Here's a followup question: how can I make Gnus update the marks >>>> for >>>> *all* IMAP groups. >>> >>> I once asked Lars about this -- since the IMAP server should be the >>> source of truth for all information about the messages, why does >>> Gnus >>> store its own marks data for IMAP groups at all? Why not just > > How many imap server support imap metadata? Clients such as Thunderbird > already safe their metadata in imap. > >>> refresh >>> from the server at each startup? I think he said that for many large >>> groups that would end up being prohibitively slow. >> >> I agree that it would be too slow for many people, especially with >> remote IMAP servers. So I'd like an optional setting. But I don't >> think it should have anything to do with Gnus shutdown and startup. > > Why would it be to slow? Other mail clients fetch the read > status for each mail when they update the state of each > imap mailbox. I have never tried it, so I don't know if it would be slow or not. But this wouldn't just update the read status, this would remove all existing marks and refresh them from the server: ticks, replied, forwarded, everything. I could imagine that being pretty slow.
Re: How to search for ticked articles
Julien Cubizolles writes: > I used to use the "mark:flag" search query to look for ticked articles > with gnus-search. I recently noticed that it doesn't return any articles > anymore in groups with ticked articles. > > What exactly does the "flag" mark represent, and is there a better way > to look for dormant or ticked articles ? These things are all search-engine dependent. Notmuch will turn "mark:flag" into "tag:flag", and IMAP will turn it into "FLAGGED". What search engine are you using in this case? The first thing to try is always shutting off search parsing (either fully, by setting `gnus-search-use-parsed-queries' to nil, or temporarily, using the prefix argument). Then get the search working using the search engine's native syntax. Then come back here and tell me how it went :)
Re: Does Gnus read and sync the read with IMAP?
Dan Christensen writes: > On Oct 19, 2023, Björn Bidar wrote: > >> How can I make Gnus to fetch the read status from Imap? > > In the *Group* buffer, you can make Gnus update the marks from the IMAP > server using `M-g'. > > Here's a followup question: how can I make Gnus update the marks for > *all* IMAP groups. I've tried various incantations. `g' doesn't work, > nor does `M-g' on a topic containing the IMAP groups. In fact, I'd > like Gnus to always refresh IMAP groups in the thorough way that `M-g' > on the group does, since I'm accessing a local IMAP server. I once asked Lars about this -- since the IMAP server should be the source of truth for all information about the messages, why does Gnus store its own marks data for IMAP groups at all? Why not just refresh from the server at each startup? I think he said that for many large groups that would end up being prohibitively slow. I guess that might be true, I don't know. I do think that Gnus should be much, much more willing to dump its marks store and refresh if anything at all seems to be getting out of whack. Maybe we could put in a customization option that would clear out Gnus' nnimap group marks at shutdown, which would force a full refresh at the next startup. That feels like a pretty major change, I'm not sure what the knock-on consequences might be. The other common problem (that Bob Newell ran into recently) is that Gnus' article numbers are purely an internal convention, and don't have any real correspondence to external reality, yet are relied upon pretty heavily within Gnus to "make sense" as an indicator of sequence. When they get out of whack you are SOL. I don't see a great solution to that. Eric
Re: Does Gnus read and sync the read with IMAP?
Michael Heerdegen writes: > Bob Newell writes: > >> This goes well beyond my very limited knowledge so I did an >> experiment. I went to the gmail web interface and found an >> already-opened message in the All Mail folder (really a label) >> that I knew for certain I had originally opened in gnus, so >> therefore gnus would have known about the message. I marked >> it unread (on the web) and moved it back to INBOX. >> >> Then I started up gnus and gnus saw the message as an unread >> INBOX message. Based on this it appears the answer is yes, >> gnus did update the status of this previously seen email. >> However I don't know how much if any status information gnus >> actually keeps (and I don't use the registry) so this may or >> may not be meaningless. It certainly seems as if, when >> reopening the group subsequent to shutting down gnus at some >> point, gnus gets a fresh set of information. > > Thanks for doing this experiment. I hoped for that result. > > AFAIU Gnus stores such stuff in a file named ".newsrc.eld" (AFAIR > ".newsrc.el" in older Emacs versions). And indeed, among other things > there seems to be information about "seen" status of messages in that > file, as well as saved status, ticked and dormant status, all of that > Gnus specific stuff, but nothing related to the "read" (or "old") > status. So I guess the "read" status is fetched every time when you > open the group, along with the rest (list of existing messages etc). Here's the mapping between Gnus' marks and IMAP flags, I believe things like "gnus-save" are set as user flags on servers that support that: (defvar nnimap-mark-alist '((read "\\Seen" %Seen) (tick "\\Flagged" %Flagged) (reply "\\Answered" %Answered) (expire "gnus-expire") (dormant "gnus-dormant") (score "gnus-score") (save "gnus-save") (download "gnus-download") (forward "gnus-forward")))
Re: slow saving of scores when leaving a virtual (nnml+) group
Eric S Fraga writes: > On Thursday, 6 Jul 2023 at 08:43, Eric Abrahamsen wrote: >> I've attached a patch that reverts it, you might give this a try? If it >> clearly fixes the problem, I would open a bug report and cc Stefan. > > Hi Eric, > > thank you for this patch. Unfortunately, it does not seem to have had > any impact in my case. Bummer. You can always escalate and just open an Emacs bug -- this is a perfectly normal type of thing to open a bug about.
Re: slow saving of scores when leaving a virtual (nnml+) group
Eric S Fraga writes: > Hello all, > > recently (past week or two), I've noticed a slow down in leaving some of > my nnml groups. Using the profiler, I see the outcomes shown below, > capturing cpu and memory when entering a virtual group, that collects 3 > different nnml groups, and then immediately leaving that group. A > significant amount of time is taken saving scores, as far as I can tell. > I use adaptive scoring. Nothing with respect to scoring has changed in > my configuration in some time (years probably). > > The offending function appears to be "lisp--local-defform-body-p" with > large memory and cpu use. > > Cpu report (partly expanded): > >10133 79% - command-execute > 8519 66% - funcall-interactively > 4767 37% - gnus-summary-exit > 4659 36%- gnus-score-save > 4655 36% - gnus-pp > 4655 36% - pp > 4655 36% - pp-to-string > 4655 36%- pp-fill > 4647 36% - pp--object > 4627 36% - pp-fill > 4615 36% - pp-fill > 4555 35%- pp-fill > 4263 33% - pp-fill > 4243 33% - indent-according-to-mode > 4243 33% - lisp-indent-line > 4163 32%- calculate-lisp-indent > 4163 32% - lisp-indent-function > 4163 32%lisp--local-defform-body-p > 48 0%+ lisp-ppss A few weeks ago Stefan Monnier made a change to lisp-ppss. It was meant to improve performance, but the timing looks very suspicious and it may have had unexpected side effects. The commit is f411cc3a9578eae4ea4549, I've attached a patch that reverts it, you might give this a try? If it clearly fixes the problem, I would open a bug report and cc Stefan. diff --git a/lisp/emacs-lisp/lisp-mode.el b/lisp/emacs-lisp/lisp-mode.el index 1990630608d..3613c8d88ec 100644 --- a/lisp/emacs-lisp/lisp-mode.el +++ b/lisp/emacs-lisp/lisp-mode.el @@ -876,7 +876,7 @@ lisp-ppss 2 (counting from 0). This is important for Lisp indentation." (unless pos (setq pos (point))) (let ((pss (syntax-ppss pos))) -(if (and (not (nth 2 pss)) (nth 9 pss)) +(if (nth 9 pss) (let ((sexp-start (car (last (nth 9 pss) (parse-partial-sexp sexp-start pos nil nil (syntax-ppss sexp-start))) pss)))
Re: Gnus fetch freezes emacs
Stephen Berman writes: > On Wed, 05 Jul 2023 11:55:36 -0700 Eric Abrahamsen > wrote: > >> On 07/05/23 10:04 AM, Stephen Berman wrote: >>> On Tue, 04 Jul 2023 20:50:05 -0700 Eric Abrahamsen >>> wrote: > [...] >>>> Using `nntp-connection-timeout' is the proper fix for this problem, it's >>>> just got a bit of unfortunate behavior that needs to be remedied. I'd be >>>> inclined to start a whole new bug report for a fix for that, because >>>> it's really a new issue, with its own larger-reaching design decisions. >>>> I suppose we could merge #52735 with that, though. >>> >>> Feel free to open a new bug for fixing nntp-connection-timeout. I don't >>> know if I can help, other than trying out suggestions and providing >>> feedback. In the meantime I'll keep using my workaround replacement >>> function. >>> >>> But I wonder, could this issue have been triggered by some change in >>> news.gmane.io around early to mid December 2021? Because that's when >>> the problem start for me, and prior to that I don't recall ever having >>> this problem (perhaps sporadically but not with such persistance). >> >> The nntp-connection-timeout variable has been present and nil since >> 1999. I put my original buggy fix in at the end of October 2021, so that >> seems suspicious, but that should only have had an effect if you had set >> nntp-connection-timeout to something other than the default of nil. > > I've never touched nntp-connection-timeout, I only became aware of it > when you mentioned it in this thread. Also, October 2021 is too early: > the early to mid December date stands out to me because in early > December of that year there was a change in my internet service, and I > started experiencing the Gnus problem soon thereafter. At first I > thought it might be due to my change in internet service, but the > problem really was and remains only with fetching news with Gnus (and > not with fetching email with Gnus, which I do from different mail > servers). I also only use news.gmane.io for fetching news, so that's > why I thought maybe something changed there. It's entirely possible! The buggy fix was reverted in early May, so if you stopped seeing the bad behavior then, that would be pretty conclusive.
Re: Gnus fetch freezes emacs
On 07/05/23 10:04 AM, Stephen Berman wrote: > On Tue, 04 Jul 2023 20:50:05 -0700 Eric Abrahamsen > wrote: > >> Stephen Berman writes: >> >>> On Tue, 04 Jul 2023 10:02:34 -0700 Eric Abrahamsen >>> wrote: >>> >>>> Stephen Berman writes: >> >> [...] >> >>> This: >>> >>> (defun srb-gnus-group-get-new-news ( arg one-level) >>>(interactive "P") >>>(with-timeout (1 (kill-buffer (nntp-find-connection-buffer >>> nntp-server-buffer)) >>>(gnus-group-get-new-news)) >>> (gnus-group-get-new-news arg one-level))) >>> >>> (define-key gnus-group-mode-map "g" 'srb-gnus-group-get-new-news) >>> >>>> Eric F is just describing the >>>> unfortunate behavior of nntp-connection-timeout, which interrupts the >>>> entire fetching process when it hits the timeout. >>> >>> Is that different than what the above function does with the kill-buffer >>> sexp? (Not a rhetorical question, I know next to nothing about news >>> servers and their connectivity issues.) >> >> The `nntp-connection-timeout' variable has different behavior in that >> NNTP servers are allowed one "retry" if the connection fails. The code >> around that is very confusing to me (which is why my earlier fix was >> buggy). > > I don't follow you, but no need to elaborate further here. > >>>>>> Yeah, I'd put in a dumb fix for this that turned out to be buggy, so we >>>>>> just recently reverted it. I have a more thorough fix in progress >>>>>> somewhere here, that would report a server connection failure without >>>>>> interrupting the rest of the servers, but it's not done yet. I've had >>>>>> very little time for coding recently, but will get to it At Some Point. >>>>>> >>>>>> Glad it's at least better than it was. I wonder if we should have some >>>>>> generous timeout set by default... >>>>> >>>>> It might make sense to continue this discussion in bug#52735. >>>> >>>> This doesn't seem like the same issue -- this problem is pretty well >>>> understood. >>> >>> Hm, I had understood from both Prashant Tak and Eric Fraga that the >>> problem they have is essentially the same as I do and what I reported in >>> that bug. But that problem doesn't seem to be understood. If by the >>> understood problem you mean the effect of nntp-connection-timeout, >>> doesn't that just mean using it isn't a real fix for the hang the three >>> of us (at least) are experiencing? That's why I thought other >>> approaches need to be considered and bug#52735 seems like the >>> appropriate venue for that. But I'm fine with continuing the discussion >>> here instead. >> >> Oh I see what you mean. In your bug report I'd gotten the idea that >> something was going wrong with accepting process output, and had a >> missed-the-forest-for-the-trees moment around it simply being a dead >> process. >> >> Using `nntp-connection-timeout' is the proper fix for this problem, it's >> just got a bit of unfortunate behavior that needs to be remedied. I'd be >> inclined to start a whole new bug report for a fix for that, because >> it's really a new issue, with its own larger-reaching design decisions. >> I suppose we could merge #52735 with that, though. > > Feel free to open a new bug for fixing nntp-connection-timeout. I don't > know if I can help, other than trying out suggestions and providing > feedback. In the meantime I'll keep using my workaround replacement > function. > > But I wonder, could this issue have been triggered by some change in > news.gmane.io around early to mid December 2021? Because that's when > the problem start for me, and prior to that I don't recall ever having > this problem (perhaps sporadically but not with such persistance). The nntp-connection-timeout variable has been present and nil since 1999. I put my original buggy fix in at the end of October 2021, so that seems suspicious, but that should only have had an effect if you had set nntp-connection-timeout to something other than the default of nil. In general it's pretty hard to say, though...
Re: Getting Keywords: from parent article in message-setup-hook when following up
Adam Sjøgren writes: > Eric writes: > >> When you think about it, updating a global variable is all the Gnus code >> is doing! But better to rely on someone else's ugly hack, I suppose... > > Haha, yes, and if you take a couple of more steps back, all computing > is manipulation of global memory ;-) > > But I think not having one hook storing the information and another > one using it will be a little less ugly, so thanks for the hint - the > naming of these variables is often not obvious to me. Ugh, don't get me started. What's the difference between gnus-current-article and gnus-article-current? No one will ever know.
Re: Gnus fetch freezes emacs
Stephen Berman writes: > On Tue, 04 Jul 2023 10:02:34 -0700 Eric Abrahamsen > wrote: > >> Stephen Berman writes: [...] > This: > > (defun srb-gnus-group-get-new-news ( arg one-level) >(interactive "P") >(with-timeout (1 (kill-buffer (nntp-find-connection-buffer > nntp-server-buffer)) > (gnus-group-get-new-news)) > (gnus-group-get-new-news arg one-level))) > > (define-key gnus-group-mode-map "g" 'srb-gnus-group-get-new-news) > >> Eric F is just describing the >> unfortunate behavior of nntp-connection-timeout, which interrupts the >> entire fetching process when it hits the timeout. > > Is that different than what the above function does with the kill-buffer > sexp? (Not a rhetorical question, I know next to nothing about news > servers and their connectivity issues.) The `nntp-connection-timeout' variable has different behavior in that NNTP servers are allowed one "retry" if the connection fails. The code around that is very confusing to me (which is why my earlier fix was buggy). >>>> Yeah, I'd put in a dumb fix for this that turned out to be buggy, so we >>>> just recently reverted it. I have a more thorough fix in progress >>>> somewhere here, that would report a server connection failure without >>>> interrupting the rest of the servers, but it's not done yet. I've had >>>> very little time for coding recently, but will get to it At Some Point. >>>> >>>> Glad it's at least better than it was. I wonder if we should have some >>>> generous timeout set by default... >>> >>> It might make sense to continue this discussion in bug#52735. >> >> This doesn't seem like the same issue -- this problem is pretty well >> understood. > > Hm, I had understood from both Prashant Tak and Eric Fraga that the > problem they have is essentially the same as I do and what I reported in > that bug. But that problem doesn't seem to be understood. If by the > understood problem you mean the effect of nntp-connection-timeout, > doesn't that just mean using it isn't a real fix for the hang the three > of us (at least) are experiencing? That's why I thought other > approaches need to be considered and bug#52735 seems like the > appropriate venue for that. But I'm fine with continuing the discussion > here instead. Oh I see what you mean. In your bug report I'd gotten the idea that something was going wrong with accepting process output, and had a missed-the-forest-for-the-trees moment around it simply being a dead process. Using `nntp-connection-timeout' is the proper fix for this problem, it's just got a bit of unfortunate behavior that needs to be remedied. I'd be inclined to start a whole new bug report for a fix for that, because it's really a new issue, with its own larger-reaching design decisions. I suppose we could merge #52735 with that, though. Eric
Re: Gnus fetch freezes emacs
Stephen Berman writes: > On Mon, 03 Jul 2023 09:36:26 -0700 Eric Abrahamsen > wrote: > >> Eric S Fraga writes: >> >>> On Sunday, 2 Jul 2023 at 16:59, Eric Abrahamsen wrote: >>>> If everyone's hitting this with NNTP servers, you can set >>>> `nntp-connection-timeout' to a number of seconds. It is nil by default, >>>> which I guess would result in permanent hangs. > > Is this variable supposed to be set in the value of gnus-select-method? > For example, like this: > > (setq gnus-select-method '(nntp "news.gmane.io" > (nntp-connection-timeout 3))) It's a defvoo, so it can either be set globally, or as a server parameter. >>> So this works, in the sense that it stops me waiting forever... However, >>> it seems (early days yet) that when it fails to open the connection to >>> an NNTP server, it stops retrieving news and I have to hit 'g' again to >>> get the counts etc. updated for other servers. [...] > > That sounds basically like what the function I'm using in place of > gnus-group-get-new-news (see my first post in this thread) does. Could > such a function take effect if added to one of the server hook variables > nntp-server-opened-hook, nntp-server-action-alist or > nntp-open-connection-function? From the descriptions in the manual it > isn't clear to me. Or is there some better Gnus hook variable for this > purpose? If not could one be added? I'm not sure what function you mean. Eric F is just describing the unfortunate behavior of nntp-connection-timeout, which interrupts the entire fetching process when it hits the timeout. >> Yeah, I'd put in a dumb fix for this that turned out to be buggy, so we >> just recently reverted it. I have a more thorough fix in progress >> somewhere here, that would report a server connection failure without >> interrupting the rest of the servers, but it's not done yet. I've had >> very little time for coding recently, but will get to it At Some Point. >> >> Glad it's at least better than it was. I wonder if we should have some >> generous timeout set by default... > > It might make sense to continue this discussion in bug#52735. This doesn't seem like the same issue -- this problem is pretty well understood. Eric
Re: Getting Keywords: from parent article in message-setup-hook when following up
Adam Sjøgren writes: > Eric writes: > >> I think the "big" solution is to visit `message-reply-buffer', if that's >> a live buffer, and parse its contents. > > That sounds better than the ugly hack I have now, where I update a > global variable from gnus-article-prepare-hook and use it in > message-setup-hook - eeek. When you think about it, updating a global variable is all the Gnus code is doing! But better to rely on someone else's ugly hack, I suppose...
Re: Getting Keywords: from parent article in message-setup-hook when following up
Adam Sjøgren writes: > How do I get hold of the parent articles' Keywords: in > message-setup-hook when I do a follow up? > > I can get eg From: by lookin in message-reply-headers - but Keywords: > are not in there. > > Is there a trick I can use? I think the "big" solution is to visit `message-reply-buffer', if that's a live buffer, and parse its contents.
Re: Gnus fetch freezes emacs
Eric S Fraga writes: > On Sunday, 2 Jul 2023 at 16:59, Eric Abrahamsen wrote: >> If everyone's hitting this with NNTP servers, you can set >> `nntp-connection-timeout' to a number of seconds. It is nil by default, >> which I guess would result in permanent hangs. > > So this works, in the sense that it stops me waiting forever... However, > it seems (early days yet) that when it fails to open the connection to > an NNTP server, it stops retrieving news and I have to hit 'g' again to > get the counts etc. updated for other servers. But much better than > waiting forever. Yeah, I'd put in a dumb fix for this that turned out to be buggy, so we just recently reverted it. I have a more thorough fix in progress somewhere here, that would report a server connection failure without interrupting the rest of the servers, but it's not done yet. I've had very little time for coding recently, but will get to it At Some Point. Glad it's at least better than it was. I wonder if we should have some generous timeout set by default...
Re: Gnus fetch freezes emacs
Prashant Tak writes: > Stephen Berman writes: > >> On Fri, 30 Jun 2023 20:03:11 +0530 Prashant Tak >> wrote: >> >>> Gnus has been freezing sporadically when `gnus-group-get-new-news` is run. >>> And it keeps on going for hours, I have to manually intercept and signal >>> `keyboard-quit` and then perform the fetch operation again. This happens >>> in a very unpredictable manner, so it's hard to replicate. I did manage >>> to get a profiler report when that happened. >>> >>> 6416 86% - nntp-accept-response >>> 6068 82%- nntp-accept-process-output >>> 5873 79% - nnheader-accept-process-output >>> 19 0% + accept-process-output >>> >>> The main culprit seems to be `nnheader-accept-process-output` but I >>> don't know how to proceed further. Appreciate any help/input into the >>> matter. >> >> This sounds like the issue I've been having with gnus-group-get-new-news >> and similar Gnus commands for more than a year and a half, see >> bug#52735. As reported there, I did some debugging but couldn't >> pinpoint the problem, nor have I tried profiling yet. But as a >> workaround, I've been using the following replacement for >> gnus-group-get-new-news: >> >> (defun srb-gnus-group-get-new-news ( arg one-level) >> (interactive "P") >> (with-timeout (1 (kill-buffer (nntp-find-connection-buffer >> nntp-server-buffer)) >> (gnus-group-get-new-news)) >> (gnus-group-get-new-news arg one-level))) >> >> (define-key gnus-group-mode-map "g" 'srb-gnus-group-get-new-news) >> >> This usually suffices but not always. When Gnus (and hence Emacs) hangs >> even when using this workaround, I've resorted to manually killing the >> server buffer " *server news.gmane.io nntp *nntpd**" and then typing `g' >> pretty reliably works again. >> >> This is a very annoying issue, and if what you're experiencing is the >> same, I commiserate with you, but your report also gives me hope that >> it's not just some quirk of my setup or network connection. Now we just >> need some Gnus expert to chime in and guide us to try and track down the >> cause of this issue and get it fixed. > > I took a look at your bug report and indeed, the behaviour described is > identical to what I've been experiencing, hopefully someone does chime > in with an idea on how to improve the situation. Thanks for sharing your > solution for the meantime though. If everyone's hitting this with NNTP servers, you can set `nntp-connection-timeout' to a number of seconds. It is nil by default, which I guess would result in permanent hangs.
Re: Is gnus-demon-scan-news supposed to scan nnselect search groups?
Julien Cubizolles writes: > I noticed that the permanent search groups I've created don't get > rescanned by (gnus-demon-scan-news), I need to run > (gnus-summary-rescan-group) in its summary to get the new search > results. Is there a way to trigger a rescan of all nnselect groups when > the contents of the groups searched has changed ? The Select Groups section of the manual has information on this: Refreshing the selection of an nnselect group by running the ‘nnselect-function’ may take a long time to complete. Consequently nnselect groups are not refreshed by default when ‘gnus-group-get-new-news’ is invoked. In those cases where running the function is not too time-consuming, a non-‘nil’ group parameter of ‘nnselect-rescan’ will allow automatic refreshing. A refresh can always be invoked manually through ‘gnus-group-get-new-news-this-group’.
Re: wrong gnus-search in nnml folder
Fernando de Morais writes:
> Hello Angel and Eric,
>
> Angel de Vicente writes:
>
>> What I don't get to work is compound queries like since:3d from:alex
>> (they work separetely, but nothing gets returned if I try to combine
>> two queries...
>
> I use mairix as my search engine on nnmaildir servers and this behavior
> has always annoyed me, although I wasn't able to figure out why it
> cannot perform this kind of search, until Eric pointed out the
> possibility of debugging `gnus-search-run-search'.
Please report all annoyance as a bug! :)
> The mairix command interprets each query as an individual argument.
> When we pass a query like:
>
> #+begin_example
> Query: from:alex since:3d
> #+end_example
>
> In `gnus-search-run-search' it will become a sole string
> ("f:alex d:20230129-"), and mairix will return nothing.
>
> Just for a simple experiment, I reevaluated the `gnus-search-run-search'
> with the following lines changed:
>
> --- #
> +++ #
> @@ -1332,7 +1332,8 @@
> (message "Doing %s query on %s..." program groups)
> (message "Doing %s query..." program))
>(setq proc (apply #'start-process (format "search-%s" server)
> - buffer program cp-list))
> + buffer program (flatten-list (mapcar #'split-string
> + cp-list
>(while (process-live-p proc)
> (accept-process-output proc))
>(setq exitstatus (process-exit-status proc))
>
> And the result of a search like the give example above was the same
> given by mairix via terminal.
>
>
> P.S.: To clarify, I'm using Emacs 28.2.
That's very interesting! And odd, since none of the other
command-line-type search engines seem to care that the whole query is
passed in as a single string. I wonder if that behavior is an accident,
though, and the mairix behavior is actually the one we should be
expecting. Instead of the patch above, would you confirm that eval'ing
the following produces the right behavior?
(cl-defmethod gnus-search-indexed-search-command ((engine gnus-search-mairix)
(qstring string)
query _groups)
(with-slots (switches config-file) engine
(append `("--rcfile" ,config-file "-r")
switches
(when (alist-get 'thread query) (list "-t"))
(split-string qstring
Though I'll continue investigating this, and might end up implementing
this for all engines.
Thanks for the bug-hunting,
Eric
Re: wrong gnus-search in nnml folder
Angel de Vicente writes: > Hello, > > Angel de Vicente writes: >> oh, indeed it was nil. Changed it and queries like "since:3d" work now >> beautifully. What I don't get to work is compound queries like since:3d >> from:alex (they work separetely, but nothing gets returned if I try to >> combine two queries... > > just tried again today. I thought I would have to fire edebug again, but > I tried quoting the whole query and it works fine. > > So, to clarify: > > + for simple queries I just type: > > Query: since:3d > > + for compound queries I need to quote it like: > > Query: "from:alex since:3d" > > I take it from section 9.3 of the Gnus manual that this is not intended. No, that's not how it's supposed to work! So here's our query: (gnus-search-transform (make-instance 'gnus-search-mairix) (gnus-search-parse-query "from:alex since:3d")) -> "f:alex d:20230128-" And you've confirmed that running "mairix f:alex d:20230128-" works correctly on the command line? There's not much other processing going on. Mairix is weird about how it ANDs and ORs search terms -- you can only OR on the same key -- but that shouldn't be coming into play here. For the record, you don't really need to be using gnus-search. If you're happy with mairix's search language, and you don't expect to ever run searches against both nnml and nnimap backends at the same time, it probably won't get you much. Eric
Re: wrong gnus-search in nnml folder
Angel de Vicente writes: > Hello, > > Angel de Vicente writes: >> I will try, though I'm really rusty with Emacs debugging, I will have to >> refresh my memory.. > > ok, it was really easy to remember how to use Edebug. What I found is > that apparently I cannot use "since:3d", "from:angel", etc. and instead I > just have to use the mairix search patterns > > so, "d:3d-" or "f:angel" work fine. > > I guess the translation from the common format to the mairix specific > one is missing somewhere? That just sounds like you're not using gnus-search at all. What's your value of `gnus-search-use-parsed-queries'? It is nil by default, and my guess is you've still got it at nil. If it is t, how about `gnus-search-mairix-raw-queries-p'? Or have you added config to the mairix search engine to only use raw queries?
Re: wrong gnus-search in nnml folder
Angel de Vicente writes: >>> You haven't said what search engine you're using, but if it's notmuch: >> >> it is ages since I configured this, but I'm not using notmuch, it is >> just whatever stock method comes with Gnus, *I think* > > should be mairix. > > my gnus-search-default-engines variable' value is: > > , > | ((nnimap . gnus-search-imap) > | (nnml . gnus-search-mairix)) > ` So that's: (gnus-search-transform (make-instance 'gnus-search-mairix) (gnus-search-parse-query "since:3d")) -> "d:20230127-" As I read the spec, that's the correct format (actually, mairix could also handle the "3d" directly, but there's currently no option to pass the unparsed search string through). Can you check on the command line, outside of Gnus, and confirm that you see the correct messages returned? You can also edebug `gnus-search-run-search' for the `gnus-search-indexed' engine (gnus-search.el:1394 in master), and see if anything funky is happening in there.
Re: wrong gnus-search in nnml folder
Angel de Vicente writes: > Hello, > > piggypbacking from the thread started by Julien Cubizolles... > > I have all my mail in nnml folders, but searching seems really broken > here. In a folder where I keep all my 2022 mails, if I do > > G G since:3d > > I get mails for the following dates: > > , > | O (20-Jan-2022) ( 11k) > | O (02-Feb-2022) (1.1k) > | O (10-Mar-2022) ( 10k) > | OA (10-Mar-2022) ( 11k) > | O (10-Mar-2022) (0.6k) > | O (10-Mar-2022) ( 18k) > | O (11-Mar-2022) ( 17k) > | OA (12-Mar-2022) ( 18k) > | O (12-Mar-2022) (0.6k) > | O (12-Mar-2022) (1.7k) > | O (17-Mar-2022) (2.5k) > | O (17-Mar-2022) (3.0k) > | O (17-Mar-2022) (5.6k) > | O (24-Mar-2022) (9.6k) > | O (12-May-2022) ( 36k) > | O (04-Jun-2022) ( 11k) > | OA (29-Aug-2022) (5.1k) > | O (30-Aug-2022) (1.1k) > | O (30-Aug-2022) (3.0k) > ` > > G G since:5d > > or > > G G since:1d > > return nothing. > > Any idea what could be going on in here? Okay, same test for you! What does this return? (gnus-search-query-parse-date "3d") (gnus-search-parse-query "since:3d") You haven't said what search engine you're using, but if it's notmuch: (gnus-search-transform (make-instance 'gnus-search-notmuch) (gnus-search-parse-query "since:3d")) -> "date:1/27/2023.." And can you tell me what locale your machine is in? Does Julien's last patch fix the problem?
Re: gnus-search since: not working for nnimap
Julien Cubizolles writes: > Julien Cubizolles writes: > >> 'gnus-search-transform seems to be at fault here. It uses >> '(format-time-string "%e-%b-%Y") where %b returns the "locale’s >> abbreviated month name" when imap commands expects the english abbreviated >> month name. > > It's working with the attached patch. I'm not sure this is the best way > to go but I'm happy with it. > > diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el > index 27c71fa6c6..0d9a62f911 100644 > --- a/lisp/gnus/gnus-search.el > +++ b/lisp/gnus/gnus-search.el > @@ -1330,9 +1330,10 @@ elements are present." > (1- nyear) > nyear)) > (setq dmonth 1 > -(format-time-string > - "%e-%b-%Y" > - (encode-time 0 0 0 dday dmonth dyear > +(with-locale-environment "C" > + (format-time-string > + "%e-%b-%Y" > + (encode-time 0 0 0 dday dmonth dyear) > > (cl-defmethod gnus-search-imap-handle-string ((engine gnus-search-imap) > (str string)) Ooh, that's a great catch, thank you very much. I'll look around and see if there are any other similar issues in this library, and push a fix to master soon. Thanks for the report, Eric
Re: gnus-search since: not working for nnimap
Julien Cubizolles writes: > Eric Abrahamsen writes: > >> You can play with the bits and pieces to see where things might be going >> wrong: >> >> (gnus-search-query-parse-date "1y") -> (28 1 2022) >> (gnus-search-parse-query "since:1y") - > ((since 28 1 2022)) >> >> So far so good. >> >> (gnus-search-transform >> (make-instance 'gnus-search-imap) >> (gnus-search-parse-query "since:1y")) -> "SINCE 28-Jan-2022" >> >> Still looking correct (right?). > > Everything is fine up until this point. > >> Try doing "C-u G G" on an nnimap group to shut off gnus-search parsing, >> and enter "SINCE 28-Jan-2022" as the search string. Does that return >> anything? > > Sill nothing: > > Group nnselect:nnselect-87cz6x1zv6.fsf contains no messages We'll have to get closer to the IMAP server, then. If you don't have TLS enabled on your local dovecot you can probably log in and test with: telnet localhost 143 Then: a login b select "INBOX" c UID SEARCH SINCE 28-Jan-2022 To confirm that messages are found. Then it's a matter of seeing that same conversation between Gnus and Dovecot, to find where things are going wrong. You can set `nnimap-record-commands' to t, then conduct a search, and also find which of your " *nnimap localhost nil *nntpd**-373470" buffers corresponds to this server. That should allow you to reconstruct the conversation. Another option would be to edebug the `gnus-search-imap-search-command' function, and see what we're getting back from Dovecot. Hopefully something in there will be revealing.
Re: gnus-search since: not working for nnimap
Julien Cubizolles writes: > I've gnus-search configured to use gnus-search-imap for a nnimap > method. G G (gnus-group-read-ephemeral-search-group) doesn't return any > result when using a query like "since:1y" on a nnimap group whereas: > > + the same query works in a nnml group > + a query like "from:user" returns many results in this time range on > the same nnimap group. > > Isn't "since:1y" a valid query for nnimap ? It should be! > The nnimap group is managed by a local dovecot server. Does it > matter ? No, I don't think so. I tried it on one of my own nnimap servers, backed by local dovecot, and it returned several thousand messages. You can play with the bits and pieces to see where things might be going wrong: (gnus-search-query-parse-date "1y") -> (28 1 2022) (gnus-search-parse-query "since:1y") - > ((since 28 1 2022)) So far so good. (gnus-search-transform (make-instance 'gnus-search-imap) (gnus-search-parse-query "since:1y")) -> "SINCE 28-Jan-2022" Still looking correct (right?). Try doing "C-u G G" on an nnimap group to shut off gnus-search parsing, and enter "SINCE 28-Jan-2022" as the search string. Does that return anything? Eric
Re: Removing the first nntp server from the list
Emanuel Berg writes: > Richmond wrote: > >> Thanks it worked. I thought you had a repeating n key, but >> it's really a thing, nnnil. >> >> https://github.com/emacs-mirror/emacs/blob/master/lisp/gnus/nnnil.el > > What's nn anyway, network news? The naming is built on the original protocol, nntp: Network News Transfer Protocol. As Gnus grew the ability to speak more protocols, they all got the same nn* prefix. I actually find it sort of handy for distinguishing discussions about IMAP or maildir from the Gnus backends, but I agree there shouldn't be any need for nnnil.
Re: Keeping IMAP connection alive when using it in mail-sources
Adam Sjøgren writes:
> Hi,
>
>
> I was wondering if there is a way to use IMAP in mail-sources ("as
> POP"), but without closing the connection after every
> gnus-group-get-new-news?
Not with the code as it's written! There's a very definite
(imap-close buf) at the end of the mail source fetching, then the buffer
is deleted. I guess it wouldn't be a bad idea to add a keepalive option
to imap.el, but someone would have to do that!
Re: delete downloaded marked emails after specific days & pop3 advice
"jindam, vani" writes:
> after much trials, i found its impossible to run
> dovecot on debian 11 on userland (1). i have a
> copy of gnus manual (by lars magne, 1995-2015,
> no version). recently disvovered check paren
> for looking errors on .gnus
>
> i have few questions:
>
> (a) is it possible to download all emails on my
> local machine, i dont want to retain email copies
> on server
mbsync doesn't offer this: the only available setting is the Sync
directive in the config file, and that doesn't allow deleting messages
remotely but not locally. This sounds more like using pop3 as a
mail-source, with the :leave setting at `nil'.
> (b) do i have to specify all email directories
> on .gnus for downloading, ex: spam, draft
With "Patterns *" in your mbsync config, mbsync will sync all folders.
Gnus will make all those folders available through the nnmaildir
backend, though you can also subscribe/unsubscribe to them within Gnus, so you
can choose what you see.
> (c) how do i auto expunge all marked emails on
> .gnus or mbsync.rc after specific period
See the "Expiring Mail" section of the Gnus manual. With the settings
you've shown below, when Gnus eventually deletes a message that deletion
will get synced to the server, as well.
> (d) seriously, reconsidering to use getmail
> to fetch all emails through pop3, is it good
> idea. pop3 supports or recognises only one
> directory —inbox & does it mean it will include
> spam emails in inbox?
If you don't care about the state of the server, and would prefer to not
leave messages there, then yes I think using a pop3 mail-source might be
a better option than mbsync. You can still use nnmaildir locally, with
split rules that send incoming mail there.
I don't use this and I don't know if pop will pull messages from
multiple folders; the docs don't mention any relevant config.
Note that you've got this in your nnmaildir config:
(get-new-mail t)
That *only* comes into play if you're using mail sources. That means
that given the config below you don't need it: mbsync will add messages
to nnmaildir, which nnmaildir will detect automatically.
> * mbsync.rc:
> IMAPAccount disroot
> Host disroot.org
> User jindam.v...@disroot.org
> SSLType IMAPS
> SSLVersions TLSv1.2
>
> # you may have to tweak the awk here + filename to your netrc/authinfo:
> # PassCmd "gpg2 -q --for-your-eyes-only --no-tty -d $HOME/.netrc.gpg |
> awk '$2==\"mail.messagingengine.com\" && $4==\"m...@fastmail.com\"
> {print $6;exit(0)}'"
> # Port 992
> # Port 992 has a "flat" hierarchy; use with
> # mail_location = maildir:~/.Maildir/:LAYOUT=fs:INBOX=~/.Maildir/INBOX
> # in dovecot (see also MaildirStore.Inbox below)
>
> IMAPStore disroot-remote
> Account disroot
>
> MaildirStore disroot-local
> # The trailing "/" is meaningful
> Path ~/.Maildir/
> Inbox ~/.Maildir/INBOX
> SubFolders Legacy
>
> Channel disroot
> Master :disroot-remote:
> Slave :disroot-local:
> Patterns *
> Create Slave
> Sync All
> Expunge Both
> SyncState *
>
> * .gnus:
> (setq gnus-select-method '(nnnil ""))
>
> (add-to-list 'gnus-secondary-select-methods
> '(nnmaildir ""
> (directory "/home/jin/.Maildir/")
> (get-new-mail t)
> (directory-files nnheader-directory-files-safe)))
>
> (setq mail-user-agent 'gnus-user-agent)
> (setq read-mail-command 'gnus)
>
>
> (1) https://github.com/CypherpunkArmory/UserLAnd
>
> regards,
> jindam, vani
Re: Help needed: nnmaildir and expire-group parameter.
Fernando de Morais writes: > Hello Eric, > > Eric Abrahamsen writes: > >> Look for the value of "pgname", and the 'expire-age parameter, and >> later the 'expire-group parameter. Something in there should jump out >> at you as "wrong". > > This was a very wise advice: in the end, `nnmaildir' was looking for the > parameters of the group "inbox", not "nnmaildir:inbox". That did the > trick! [...] > Thank you very much! Glad it wasn't any harder to resolve! > *P.S.:* I imagine there must be a rationale behind this and it was > defined a long time ago, but why doesn't `nnmaildir' make use of the > `expiry-target' parameter like other backends do? Apparently it's not using the same parameter names as everyone else, because nnmaildir lets the user put arbitrary Lisp expressions in as values -- it runs the values through `eval' at Gnus startup time. I think the idea is that, because the parameter values are handled differently, the parameter names should be different. This doesn't particularly make sense, and the use of `eval' in regular code is pretty heavily discouraged these days. This code seems to have been in nnmaildir.el since The Dawn of Time, and I'm sure that at one point someone depended heavily on being able to `eval' their parameter values. It would be relatively simple to fix: we could leave the existing handling of nnmaildir's own parameters in place, but also start retrieving the normal expiry parameters, using Gnus' regular `gnus-group-parameter-*' functions. Eric
Re: Help needed: nnmaildir and expire-group parameter.
Fernando de Morais writes:
> Hello everyone,
>
> According to the manual[1][2], `nnmaildir' uses its own group parameters
> to handle expiration.
>
> Excepting `nnmail-expiry-wait(-function)' variables, to handle
> expiration per group, the user needs to set values for `expire-age' and
> `expire-group' parameters. Ok, so considering the snippet below:
>
> #+begin_src emacs-lisp
> (customize-set-variable 'gnus-parameters
> '("nnmaildir:inbox"
> (expire-age . 0) ; <-- Here
> (expire-group . "nnmaildir:trash"))) ; <-- Here
> (customize-set-variable 'gnus-secondary-select-methods
> '((nntp "news.gwene.org")
> (nnmaildir ""
>(directory "~/Mail/")
>(target-prefix "")
>(get-new-mail t
> (customize-set-variable 'gnus-select-method '(nnnil ""))
> (customize-set-variable 'gnus-total-expirable-newsgroups
> (regexp-opt '("nnmaildir:trash")))
> #+end_src
A quick check you can do is to evaluate (after Gnus has loaded):
(nnmaildir--param "nnmaildir:inbox" 'expire-age)
And see what that gives you. I'm not entirely sure that
"nnmaildir:inbox" is correct, and I wonder if the problem you're seeing
comes from the fact that your nnmaildir server has a blank string for a
name.
The real check would be to edebug `nnmaildir-request-expire-articles'
and see what it's doing. You may already be familiar with edebug; if not
then put point inside that function and hit "C-u C-M-x". The next time
expiry runs the process will halt and display this function, and you can
step through it with . As point passes over various sexps, their
return value will be shown in the minibuffer. Look for the value of
"pgname", and the 'expire-age parameter, and later the 'expire-group
parameter. Something in there should jump out at you as "wrong".
(Apologies if you already know edebug.)
Eric
Re: Help needed: nnmaildir and create-directory parameter.
Andrew Cohen writes: >>>>>> "EA" == Eric Abrahamsen writes: > > EA> Fernando de Morais writes: > >> Hello Eric, > >> > >> I've managed to figure out. The TL;DR is: simply, don't use the > >> `create-directory' server parameter. Using only `target-prefix' > >> is what is needed. > >> > >> Eric Abrahamsen writes: > >> > > [...] > > > EA> I just don't understand how any of this is supposed to work -- I > EA> don't even know why you'd use symlinks in your maildirs to begin > EA> with. This part of the manual was written by Andy Cohen and I'm > EA> cc'ing him here, though he doesn't have a lot of time for Emacs > EA> these days and we might not get him. > > False and true:) I didn't have anything to do with this part of the > manual (or the nnmaildir code), and I have very little time for emacs > these days. Thanks for the quick response! magit-blame in gnus.texi found your commit 8a1cdce, but you might have been merging docs from different places. But no matter! So long as the question is resolved. > [...] > > EA> (I suspect the end result of all this will be the removal of > EA> 'create-directory altogether.) > > Looking back through git I see that Paul Jarc (the author of nnmaildir) > replaced 'create-directory with 'target-prefix in 2003. It was supposed > to be a /complete/ replacement, but it appears that 'create-directory > was accidentally left in the code and the manual . Eric's suspicion is > exactly right: don't use 'create-directory, and the variable should be > removed from nnmaildir.el and the manual. Excellent, thanks a lot for this confirmation. I'll yank that stuff out in a bit. Eric
Re: Help needed: nnmaildir and create-directory parameter.
Fernando de Morais writes: > Hello Eric, > > I've managed to figure out. The TL;DR is: simply, don't use the > `create-directory' server parameter. Using only `target-prefix' is what > is needed. > > Eric Abrahamsen writes: > >> So maybe try taking out the (target-prefix "") server config completely? > > With something like this: > > #+begin_src emacs-lisp > ... > (nnmaildir "" > (directory "~/Dir") > ;; (target-prefix "") > (get-new-mail t) > (create-directory "")) > ... > #+end_src > > Gnus will create groups (and respective folders) specified in > `nnmail-split-method', however it will raise an error, saying that the > directories already exists (?) and we will get a crash box, again. With > my testes, any other value besides `""' will result in error. > > But, uncommenting the `target-prefix' line and removing the > `create-directory' one, we'll end with the expected behaviour, without > errors. I'm guessing this is because, if you use create-directory instead of target-prefix, the empty string additionally gets run through `file-name-as-directory', which converts the "" to "./" and sets that as the value of target-prefix, and lord knows what that ends up doing. I just don't understand how any of this is supposed to work -- I don't even know why you'd use symlinks in your maildirs to begin with. This part of the manual was written by Andy Cohen and I'm cc'ing him here, though he doesn't have a lot of time for Emacs these days and we might not get him. Andy, you wrote this section of the manual 10+ years ago, do you still remember how it's supposed to work? What is the intended relationship between 'remove-prefix and 'create-directory, when the latter simply gets copied to the former if the former is nil? (I suspect the end result of all this will be the removal of 'create-directory altogether.) Thanks, Eric
Re: Help needed: nnmaildir and create-directory parameter.
Fernando de Morais writes:
> Hello everyone,
>
> According to the manual, for a `nnmaildir' server, if my split rules
> create new groups, I need to supply a `create-directory' server
> parameter[1]. Ok, so considering the following example snippet:
>
> #+begin_src emacs-lisp
> (customize-set-variable 'gnus-select-method '(nnnil ""))
> (customize-set-variable 'gnus-secondary-select-methods
> '((nntp "news.gwene.org")
> (nnmaildir ""
>(directory "~/Mail/maildirs")
>(target-prefix "")
>(get-new-mail t)
>(create-directory ... ; <-- Here
> (customize-set-variable 'mail-sources '((file :path "~/Mail/emaple_mbox")))
> (customize-set-variable 'nnmail-split-methods '(("inbox" "")))
> #+end_src
>
> Following the example snippet above, what is the expected value for the
> parameter `create-directory'? Unfortunately the manual brings no
> details about it.
I won't claim to understand how this is supposed to work, but in
`nnmaildir-open-server' we've got the following chunk of
lisp-as-imperative-code:
(setq x (assq 'target-prefix defs))
(if x
(progn
(setq x (cadr x)
x (eval x t)) ;FIXME: Why `eval'?
(setf (nnmaildir--srv-target-prefix server) x))
(setq x (assq 'create-directory defs))
(if x
(progn
(setq x (cadr x)
x (eval x t) ;FIXME: Why `eval'?
x (file-name-as-directory x))
(setf (nnmaildir--srv-target-prefix server) x))
(setf (nnmaildir--srv-target-prefix server) "")))
This makes it look to me like, if you've got a 'target-prefix set (even
if it's the empty string), then 'create-directory never comes into play
at all.
So maybe try taking out the (target-prefix "") server config completely?
I really don't know how this is meant to work. If you figure it out we
should update the manual.
Thanks,
Eric
Re: gnus-read-init-file: Error in ~/.gnus: disroot.org
"jindam, vani" writes: > November 22, 2022 at 6:56 PM, "Adam Sjøgren" wrote: > > >> >> vani writes: >> >> > >> > running M-x gnus has thrown another error: >> > Debugger entered--Lisp error: (void-function nnimap) > > i intentionally did not provided full log: > > Debugger entered--Lisp error: (void-function nnimap) > (nnimap "disroot.org" (nnimap-inbox "INBOX") (nnimap-split-methods > default) (nnimap-expunge t) (nnimap-stream ssl)) > eval-buffer(# nil "/home/jindam/.gnus" nil t) ; > Reading at buffer position 440 You're missing a quote before this definition. However you've got it in your init file, say if you're defining this as a part of `gnus-secondary-select-methods', the value of that variable needs to be quoted: (setq gnus-secondary-select-methods '((nnimap "disroot.org" (nnimpa... Note the quote before the value.
Re: Frequent daemon errors
Juan José García-Ripoll writes: > > Hi, > > I am using Gnus currently to access both IMAP and nntp > messages. However, I frequently find error messages such as > "*nntpd**-629172 has no process" or similar for nnimap. > > If I go back to the groups frame and enter "G" to gather all new emails > and messages, things work again, so it does not seem to be critical, but > it is very annoying -- specially because I have my system setup for > debugging any error messages that pop up. > > Any idea how to investigate this further or solve it? If you're running debug-on-error, could you post the backtrace you get in these situations?
Re: Subscribing to nnimap folders that are subscribed on IMAP servers
Ryan Kavanagh via "Announcements and discussions for GNUS, the GNU Emacs Usenet newsreader (in English)" writes: > Hi, > > I am trying to use gnus as an IMAP mail client. I have folders marked as > subscribed on my IMAP server (listed by LSUB). I would like to have gnus > automatically subscribe to these folders, or at least list them so that > I can manually subscribe gnus to them myself. How can I do so? I am > using gnus 5.13 under Emacs 27.1. Best to read the "New Groups" section of the Gnus manual: that will give you the background on `gnus-check-new-newsgroups' (you could also read its docstring), as well as methods of subscribing to new groups. Let us know if that doesn't clear it up! Eric
Re: gnus-summary-followup-with-original not working (nnheader-parse-naked-head)
Angel de Vicente writes: > Hello, > > I've been using this command for ages, but yesterday I did upgrade my > system, and now it has stopped working. I wonder if anybody has seen > this error and/or has some advice on fixing it. > > When using the command gnus-summary-followup-with-original, I now get a > reply buffer without the original mail text, and a message in the > minibuffer that reads: > > , > | save-restriction: Symbol’s function definition is void: > nnheader-parse-naked-head > ` > > My Emacs and Gnus versions are: > > + GNU Emacs 28.1 (build 1, x86_64-pc-linux-gnu) of 2022-04-27 > + Gnus v5.13 That's kind of looking like an out-of-date third party package, though it's hard to be sure. You mention upgrading your system, and Emacs was a part of that system upgrade, right? (Meaning, you're not running Emacs from its development repository?) Try running M-x toggle-debug-on-error and triggering that error again -- that should show us what is calling `nnheader-parse-naked-head'.
Re: nnimap gmail "A T" and "^" not working
JibStyle writes: > Eric Abrahamsen writes: > >> Hey there! Thanks for the report. It looks like Gmails' IMAP weirdness >> might be weird enough to break the thread-searching code in Gnus. I >> think it's definitely worth special-casing code for Gmail; give me a few >> days and I'll see what I can figure out. > > Thank you for investigating. Are you able to reproduce the issue? I am > unsure if there is a problem with Gnus <--> Gmail, or if I am doing > something wrong. That is why I ask how other users approach nnimap gmail > threading. > > My `gnus-secondary-select-methods' (primary method is nnnil): > > ((nnimap "imap.gmail.com" > (nnimap-address "imap.gmail.com") > (nnimap-server-port 993) > (nnimap-stream ssl) > (nnimap-split-methods default) > (nnimap-inbox "INBOX") > (nnimap-expunge immediately) > (nnmail-expiry-wait immediate)) > (nntp "news.gmane.io")) > > Please let me know if any additional information is useful, or any > suggested workarounds. Thank you! I technically have a gmail account, but can't access it from Gnus, so I probably won't be able to reproduce. But it's a good idea to adapt some code specifically for Gmail anyway, so I hope to be able to provide some code to test on the list here in the next week or so.
Re: nnimap gmail "A T" and "^" not working
JibStyle writes: > Hi, I am new Gnuser coming from Gmail web interface. I have mostly > configured my nnimap server for gmail and I see it has great power! > > Unfortunately, I am unable to get "A T" and "^" to work in the nnimap > summary buffer. I would like to reconstruct threads, and some articles > may be in another group. Even if they are in same group, "A T" only gets > parents but not siblings or children. I have spent days trying the > various configurations cited in `(gnus) Finding the Parent' info page to > no avail. > > Does anyone have any tips? What is your configuration for nnimap gmail > threading? Hey there! Thanks for the report. It looks like Gmails' IMAP weirdness might be weird enough to break the thread-searching code in Gnus. I think it's definitely worth special-casing code for Gmail; give me a few days and I'll see what I can figure out. Eric
Re: G g and delete mails with Emacs
Michael Heerdegen writes: > Eric Abrahamsen writes: > >> > I use G g to create a search group, open it, process mark the messages I >> > want to archive, and use b to move the messages to my local archive >> > folder. >> > >> > But after that procedure, those messages are still available in my INBOX >> > (and displayed prefixed with "O"). >> >> I'm not the one to make this decision, but this sounds like it might be >> potentially surprising behavior to users, maybe something that should be >> behind a user option similar to `nnselect-allow-ephemeral-expiry'. > > Somebody here who wants to decide? Or say something about the idea? I expect Andy Cohen would be the one to ask about this. I've cc'd him, he might not be looking at this list.
Re: G g and delete mails with Emacs
Michael Heerdegen writes: > Hello, > > since threads are not a super reliable way to find all messages related > to a discussion, I decided to try to delete related messages in my > INBOX (consulted via imap) like the following: > > I use G g to create a search group, open it, process mark the messages I > want to archive, and use b to move the messages to my local archive > folder. > > But after that procedure, those messages are still available in my INBOX > (and displayed prefixed with "O"). > > Is there a way to vary this procedure so that those messages on the imap > server are deleted? I'm not the one to make this decision, but this sounds like it might be potentially surprising behavior to users, maybe something that should be behind a user option similar to `nnselect-allow-ephemeral-expiry'.
Re: Update topic unread counts on group exit?
Adam Sjøgren writes: > Eric writes: > >> I'll remove the checks now. > > The line the error is triggered in is: > > (defun gnus-topic-update-topic-line (topic-name reads) > (let* ((top (gnus-topic-find-topology topic-name)) > (type (cadr top)) > (children (cddr top)) > (entries (gnus-topic-find-groups >(car type) (car gnus-group-list-mode) >(cdr gnus-group-list-mode))) > (all-groups (gnus-topic-find-groups >(car type) (car gnus-group-list-mode) >(cdr gnus-group-list-mode) t)) <-- HERE > (parent (gnus-topic-parent-topic topic-name)) > > gnus-topic-find-groups: Wrong type argument: number-or-marker-p, t > > gnus-group-list-mode is (5) here, so the cdr is nil, and > gnus-topic-find-groups doesn't like that, apparently. I didn't have time > to step through that to pinpoint it further. Looks like Dick Chiang has already opened a bug report containing the fix, so expect this to work Real Soon Now.
Re: Update topic unread counts on group exit?
Adam Sjøgren writes: > Adam writes: > >> Eric writes: >> >>> It looks to me like you could add a function to >>> `gnus-summary-exit-hook', which calls >>> `gnus-topic-update-topics-containing-group' on the value of >>> `gnus-newsgroup-name'. At that stage in summary exit, >>> `gnus-newsgroup-name' won't have been cleared yet, so it should still >>> hold the name of the group you're coming out of. > > [...] > >> I will see if I can get your suggestion going after dinner :-) > > This is what I did: > > (defun asjo-update-topics () > (gnus-topic-update-topics-containing-group gnus-newsgroup-name)) > > (add-hook 'gnus-summary-exit-hook 'asjo-update-topics) > > But simpler than my hack - but nothing is updated. > > When I edebug-defun it, I can see that gnus-newsgroup-name is indeed the > full newsgroup name, e.g. "nntp+gm:gmane.emacs.gnus.user", but the topic > counts aren't updated. And if I'd looked at that function before recommending it, I would have immediately seen that it's buggy! That and `gnus-topic-update-topic' check to see if the major-mode is 'gnus-topic-mode, which of course it never is, because gnus-topic-mode is a minor mode. In fact, gnus-topic.el already thinks it's doing what you want -- updating topic lines on group exit -- but because of the bug in `gnus-topic-update-topic', the update never happens. Lars added that major-mode check as part of a larger commit last year, I have to assume it was a mistake. I'll remove the checks now. Eric
Re: Update topic unread counts on group exit?
Adam Sjøgren writes: > Eric writes: > >>> Is there a way to get Gnus to update the counts automatically on group >>> exit as well? > >> It looks to me like you could add a function to >> `gnus-summary-exit-hook', which calls >> `gnus-topic-update-topics-containing-group' on the value of >> `gnus-newsgroup-name'. At that stage in summary exit, >> `gnus-newsgroup-name' won't have been cleared yet, so it should still >> hold the name of the group you're coming out of. > > Any reason for Gnus not to do that by default? I can't think of any reason off the top of my head. It's not like it's such a costly procedure. gnus-topic is a little fiddly about how it hooks into the rest of the Gnus machinery, but it shouldn't be hard to figure out. > I will see if I can get your suggestion going after dinner :-) It ought to work without the hacky dance, but I'll be curious to hear!
Re: Update topic unread counts on group exit?
Adam Sjøgren writes: > Hi, > > > When I exit a group, the unread count in the topic it is under doesn't > update. > > If I collapse the topic, the count gets updated. That's cool. > > Is there a way to get Gnus to update the counts automatically on group > exit as well? It looks to me like you could add a function to `gnus-summary-exit-hook', which calls `gnus-topic-update-topics-containing-group' on the value of `gnus-newsgroup-name'. At that stage in summary exit, `gnus-newsgroup-name' won't have been cleared yet, so it should still hold the name of the group you're coming out of.
Re: using agent to accumlate post in agentized groups
hput via "Announcements and discussions for GNUS, the GNU Emacs Usenet newsreader (in English)" writes: > I once had some working knowledge of using gnus agent. But it leaked > away over the last 20 and more yrs. As it became faster and faster to > get news. > > I want to acquire several thousand messages from certain groups. > > Ones that likely carry info I most want to access. So back then I used to > download Thousands of old messages overtime so as not to bog down any > servers. > > I lost all my old thousands of messages over many host changes, moves > and fool blunders. Have you considered setting up your own local nntp server, a Leafnode instance or something? Listening to what you're describing, I think that might end up being a more feasible approach, and might give you several other side benefits.
Re: Split mail
Edouard Debry writes: > Hello, > > I try to split my mail with gnus, without much success at the moment. > > Here is the relevant configuration section : > > > <> > (setq gnus-select-method '(nnnil "")) > > (setq gnus-secondary-select-methods > '((nntp "gmane" (nntp-address "news.gmane.io")) > (nnimap "gmail" > (nnimap-inbox "nnimap+gmail:INBOX") > (nnimap-address "imap.gmail.com") > (nnimap-server-port "imaps") > (nnir-search-engine imap) > (nnimap-stream ssl) > (nnimap-authinfo-file "~/.authinfo.gpg") > (nnmail-expiry-target "nnimap+gmail:[Gmail]/Corbeille") > (nnmail-expiry-wait 'immediate) > (nnimap-split-predicate "UNDELETED") > (nnimap-split-fancy > '(| >(to ".*emacs-orgm...@gnu.org" "orgmode") >"nnimap+gmail:INBOX")) > (nnimap-split-methods 'nnimap-split-fancy) > ))) > <> > > As far as I understand, this should create a gnus group "orgmode" in > which gnus would move all received mails which Cc to > emacs-orgm...@gnu.org, because "to" searches both "To" and "Cc" fields. > > But nothing so far happens. > > If I change (nnimap-inbox "nnimap+gmail:INBOX") into (nnimap-inbox > "INBOX") and, > > (nnimap-split-fancy > '( >(to ".*emacs-orgm...@gnu.org" "orgmode") > "nnimap+gmail:INBOX")) > > into > (nnimap-split-fancy > '( >(to ".*emacs-orgm...@gnu.org" "orgmode") > "INBOX")) > > then, I get the error : > > Error in ‘nnmail-split-methods’; using ‘bogus’ mail group: > (wrong-type-argument arrayp (| (to ".*emacs-orgm...@gnu.org" > "nnimap+gmail:orgmode") "INBOX")) [39 times] Your entire select method config is already quoted, so there's no need to additionally quote the value of `nnimap-split-methods' or `nnimap-split-fancy'. And yes, the inbox should be a plain (unprefixed) group name. (Also, is there actually a `nnimap-authinfo-file' variable? What version of Emacs/Gnus are you using?)
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
Eric Abrahamsen writes:
> jao writes:
>
>> On Thu, Dec 30 2021, Eric Abrahamsen wrote:
>>
>>
>> [...]
>>
>>> Okay, here goes the next try. A few things to note:
>>>
>>> - I realized notmuch already has a "thread:{}" syntax that
>>> does the double search I was doing in elisp, so now we just use that
>>> instead.
>>
>> makes sense.
>>
>>> - In all my testing I couldn't see that having "duplicate=1" on thread
>>> searches causes any problems, so I've taken it off. Can you please
>>> doublecheck this? If it's still mucking it up for you, I'll put it
>>> back in. I wish I really understood what the problem is (I think it
>>> has to do with notmuch potentially storing the same message in
>>> multiple locations, using symlinks).
>>
>> hmm, are you sure you've removed it? i can see, after applying your
>> diff, at least for files searches:
>>
>> (cl-defmethod gnus-search-indexed-search-command ((engine
>> gnus-search-notmuch)
>>(qstring string)
>>query _groups)
>> ;; Theoretically we could use the GROUPS parameter to pass a
>> ;; --folder switch to notmuch, but I'm not confident of getting the
>> ;; format right.
>> (let ((limit (alist-get 'limit query)))
>> (with-slots (switches config-file) engine
>> (append
>>(list (format "--config=%s" config-file)
>> "search"
>> "--output=files"
>> "--duplicate=1")
>>(when limit (list (format "--limit=%d" limit)))
>>switches
>>(list qstring)
>>
>> at any rate, i had already tried searches without it in my patched
>> version and haven't seen any adverse effects. my understanding is that
>> notmuch is clever enough to detect duplicate messages with different
>> filenames .
>>
>>
>>> - The search result filtration now won't filter on group names if the
>>> search is a thread search. This should resolve the issue you were
>>> seeing where "A T" would only search within the group you had searched
>>> in to begin with. I guess I think that an explicit thread search by
>>> the user should result in a full scan of the server. We can see if
>>> that surprises/annoys anyone, though.
>>
>> the behaviour for me is the same as with my previous patch: A T stays in
>> the nnselect group. a thing to notice is that, in general, there is no
>> single "the group you had searched in to begin width" (pretty often, i
>> do searches accross all my nnml groups, of which i have plenty)... a
>> full scan of the server is, i think, precisely what a notmuch user would
>> expect :) (but i don't know if this is really supposed to work for
>> gnus-search: in general, collecting all messages of a thread will return
>> messages from a list of different gnus groups: should we be able to show
>> all of them in an ephemeral group then?).
>>
>> be it as it may, even with the full original thread belonging to a
>> single nnml group, A T is leaving me in nnselect with only the messages
>> that were already there (i.e., it's not equivalent to A W followed by A
>> T... but then again, maybe it's not supposed to be?)
>
> I can't believe how long this is taking me...
>
> I was confusing myself because there are two separate problems: notmuch
> thread searching was simply broken, and referring a thread from within
> an nnselect group only finds messages already in that select group.
>
> I've pushed a patch that should simply get thread searching into a
> working state (notmuch's "thread{}" syntax turned out to be more
> complicated and less useful than I thought, so I dropped it).
>
> Next, I think it's a reasonable design decision to say that referring a
> thread from within an nnselect group should search that group's
> constituent groups, not the group itself. What I'm seeing is that
> nnselect actually does that (if we're using search for thread-referral,
> it nixes out the group argument and searches the whole original
> server(s)), but then an error is raised later on, when we try to fetch
> the headers (line 656). I can see that all the thread article numbers
> are returned correctly, but then when we get to `gnus-fetch-headers' I
> get an args-out-of-range error, it looks like because we're looking for
> the index of an
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
Andrew Cohen writes: >>>>>> "EA" == Eric Abrahamsen writes: > > > [...] > > > EA> Next, I think it's a reasonable design decision to say that > EA> referring a thread from within an nnselect group should search > EA> that group's constituent groups, not the group itself. What I'm > EA> seeing is that nnselect actually does that (if we're using > EA> search for thread-referral, it nixes out the group argument and > EA> searches the whole original server(s)), but then an error is > EA> raised later on, when we try to fetch the headers (line 656). I > EA> can see that all the thread article numbers are returned > EA> correctly, but then when we get to `gnus-fetch-headers' I get an > EA> args-out-of-range error, it looks like because we're looking for > EA> the index of an article number in the original (smaller) list of > EA> newsgroup headers. This was referring a thread from a summary > EA> buffer that contained only a single message. > > EA> Andy, does that sound familiar? > > Err, no. I use this constantly in nnselect groups and have no > errors. And I just tried it with an nnselect summary buffer with a > single message with no trouble. Thanks for the swift and thorough response! > Maybe its backend specific? I am using imap... I suspect it is still > some kind of notmuch problem. I really, really hope not, as the whole point of all this was complete code isolation. I don't see how it could be, as new-nnselect-artlist was populated correctly, and after that notmuch doesn't seem to be involved. > I admit the code for this is a bit hairy, but it tries to do the > following: > > Identify the potential groups that might hold related articles (this is > a bit more than just searching all the groups; If > =gnus-refer-thread-use-search= is true, search ALL groups on the > corresponding servers in the current nnselect group, but this isn't > relevant here). > > If any hits are found they are MERGED into the nnselect group (by > extending the selection). Then the headers are retrieved with > =gnus-fetch-headers=. > > Gory details to help debugging: > > The search results are held in new-nnselect-artlist, so you can inspect > this to see if the search result is correct---it should be the usual vector of > vectors e.g. [["nnimap+server:group" 47 100] ["nnimap+server:group" 49 > 100]]. > > The mapc merges these into the current selection (its a merge rather > than append because some of the searched-for articles are already in the > selection; the virtual group numbers of these articles are collected in > =old-arts=). As a side effect it counts the number of genuinely new > articles, numbered =first= to =last=. > > Then we fetch the headers of all the articles in the thread (both the > old and the new). That is the =gnus-fetch-headers= in the next code > block. You can check this by comparing with the > =gnus-newsgroup-selection=; this is the full vector of articles (all the > original articles in the nnselect plus all the new ones). The virtual > group article number is just the location in the vector. So you can > compare old-arts, first, and last to see that it is all correct. In my edebugging test, "new-nnselect-artlist" held the correct (newly-found) article vectors. Then when the code got here: (append (sort old-arts #'<) (number-sequence first last)) old-arts held all the newly-found article numbers. So the mapc seemed to add all articles in "new-nnselect-artlist". Then "first" was 2 and "last" was 1, so `number-sequence' returned nil -- that's expected, yes? Then I stepped forward to the `gnus-fetch-headers' call, and got args out of range, the sequence being a vector holding the single vector of the _original_ message that was in the nnselect group, and the index being 14, the highest of the newly-found article numbers. Setting up testing takes a little bit, I'm going to do some automation for gnus-mock, and then jump back into this. Thanks again, Eric
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
jao writes:
> On Thu, Dec 30 2021, Eric Abrahamsen wrote:
>
>
> [...]
>
>> Okay, here goes the next try. A few things to note:
>>
>> - I realized notmuch already has a "thread:{}" syntax that
>> does the double search I was doing in elisp, so now we just use that
>> instead.
>
> makes sense.
>
>> - In all my testing I couldn't see that having "duplicate=1" on thread
>> searches causes any problems, so I've taken it off. Can you please
>> doublecheck this? If it's still mucking it up for you, I'll put it
>> back in. I wish I really understood what the problem is (I think it
>> has to do with notmuch potentially storing the same message in
>> multiple locations, using symlinks).
>
> hmm, are you sure you've removed it? i can see, after applying your
> diff, at least for files searches:
>
> (cl-defmethod gnus-search-indexed-search-command ((engine
> gnus-search-notmuch)
> (qstring string)
> query _groups)
> ;; Theoretically we could use the GROUPS parameter to pass a
> ;; --folder switch to notmuch, but I'm not confident of getting the
> ;; format right.
> (let ((limit (alist-get 'limit query)))
> (with-slots (switches config-file) engine
> (append
>(list (format "--config=%s" config-file)
> "search"
> "--output=files"
> "--duplicate=1")
>(when limit (list (format "--limit=%d" limit)))
>switches
>(list qstring)
>
> at any rate, i had already tried searches without it in my patched
> version and haven't seen any adverse effects. my understanding is that
> notmuch is clever enough to detect duplicate messages with different
> filenames .
>
>
>> - The search result filtration now won't filter on group names if the
>> search is a thread search. This should resolve the issue you were
>> seeing where "A T" would only search within the group you had searched
>> in to begin with. I guess I think that an explicit thread search by
>> the user should result in a full scan of the server. We can see if
>> that surprises/annoys anyone, though.
>
> the behaviour for me is the same as with my previous patch: A T stays in
> the nnselect group. a thing to notice is that, in general, there is no
> single "the group you had searched in to begin width" (pretty often, i
> do searches accross all my nnml groups, of which i have plenty)... a
> full scan of the server is, i think, precisely what a notmuch user would
> expect :) (but i don't know if this is really supposed to work for
> gnus-search: in general, collecting all messages of a thread will return
> messages from a list of different gnus groups: should we be able to show
> all of them in an ephemeral group then?).
>
> be it as it may, even with the full original thread belonging to a
> single nnml group, A T is leaving me in nnselect with only the messages
> that were already there (i.e., it's not equivalent to A W followed by A
> T... but then again, maybe it's not supposed to be?)
I can't believe how long this is taking me...
I was confusing myself because there are two separate problems: notmuch
thread searching was simply broken, and referring a thread from within
an nnselect group only finds messages already in that select group.
I've pushed a patch that should simply get thread searching into a
working state (notmuch's "thread{}" syntax turned out to be more
complicated and less useful than I thought, so I dropped it).
Next, I think it's a reasonable design decision to say that referring a
thread from within an nnselect group should search that group's
constituent groups, not the group itself. What I'm seeing is that
nnselect actually does that (if we're using search for thread-referral,
it nixes out the group argument and searches the whole original
server(s)), but then an error is raised later on, when we try to fetch
the headers (line 656). I can see that all the thread article numbers
are returned correctly, but then when we get to `gnus-fetch-headers' I
get an args-out-of-range error, it looks like because we're looking for
the index of an article number in the original (smaller) list of
newsgroup headers. This was referring a thread from a summary buffer
that contained only a single message.
Andy, does that sound familiar?
One other bug I'm not sure what to do with: notmuch only accepts
message-ids as search terms *without* the angle brackets. If the user is
using unparsed (raw) queries, presumably they know not to include angle
brackets.
Re: Replying with a specified posting style
Alberto Luaces writes: [...] >> The "match" part of the posting style can be more complicated than a >> simple string match on group name. If you need to match on both "From" >> header and group name, I think the simplest thing might be to use your >> own function that checks those two things (the function is called with >> point in the message that you're replying to), and then sets the From >> and GCC and maybe SMTP server appropriately. >> >> Does that make sense? > > Yes, it does. My use case is not generalist, since I do not have any > rules that I can write beforehand, but from time to time someone > addresses me at the wrong email identity. Ah, well posting styles are by definition automatic rules, so that's probably not the right solution here. > I was somehow looking for a function that could set up those fields you > mentioned in the reply process, but if it is not straightforward, I am > sticking with my procedure: move the message to the group it should > belong, and then benefit from its posting style when replying. I think you might look into one of the packages that do "identity" in Gnus -- see https://www.emacswiki.org/emacs/CategoryGnus#h5o-5
Re: Replying with a specified posting style
Alberto Luaces writes: > Hi, this smells like a FAQ, but I couldn't find anything about it: > > Sometimes I receive messages to my account A, while I want to reply to > them from my account in server B. > > As far as I know, posting styles are only selected when composing new > messages. No, posting styles are in effect based on replies/followups as well. In fact, I think the only time they *aren't* in effect is when composing new messages, unless you call a message composition command with a prefix arg, in which case the group under point is used. > As a low-tech solution, I just move the message from A to B's INBOX and > reply from there. > > Is there a more elegant solution? The "match" part of the posting style can be more complicated than a simple string match on group name. If you need to match on both "From" header and group name, I think the simplest thing might be to use your own function that checks those two things (the function is called with point in the message that you're replying to), and then sets the From and GCC and maybe SMTP server appropriately. Does that make sense? Eric
Re: gnus and spam filtering; it is not working well
physiculus writes: > hello, > i try to make gnus a bit more useful for spam detection, but it does > only work partially. > if header is flagged as spam, gnus put the message into the spam folder. > but checking with bogofilter is not working. Looks like the first thing to try, after Gnus has started up, is to check the value of spam-bogofilter-valid. If it is 'unknown, try running M-: (spam-bogofilter-valid).
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
Andrew Cohen writes: >>>>>> "EA" == Eric Abrahamsen writes: > > [...] > > > EA> `gnus-refer-thread-use-search' should definitely be > EA> respected. Let me think about this. If that variable is nil, > EA> search isn't used at all, in which case no code in > EA> gnus-search.el matters. > > I don't think this is right (at least for imap): search is still used > even if in a single group. (nnselect-request-thread warps to the > original group the article came from, and then calls > gnus-summary-refer-thread, which in turn will initiate a search in this > group). > > > Haven't actually looked at the detailed patch yet. The patch doesn't touch imap code, only the indexed search engines. Which, come to think of it, will mean that behavior diverges between different engines, which is bad. Bah, something is still wrong here, and I need to go out. I'll do another round of stupid patches tomorrow. Eric
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
Andrew Cohen writes: >>>>>> "EA" == Eric Abrahamsen writes: > > EA> "Jose A. Ortega Ruiz" writes: > > [...] > > EA> - The search result filtration now won't filter on group names > EA> if the search is a thread search. This should resolve the issue > EA> you were seeing where "A T" would only search within the group > EA> you had searched in to begin with. I guess I think that an > EA> explicit thread search by the user should result in a full scan > EA> of the server. We can see if that surprises/annoys anyone, > EA> though. > > Umm, maybe I am misunderstanding this paragraph, but there is a variable > for explicit control of this: > > (defcustom gnus-refer-thread-use-search nil > "Search an entire server when referring threads. > A nil value will only search for thread-related articles in the > current group." > :version "28.1" > :group 'gnus-thread > :type 'boolean) > > I usually expect a full server search but not always. Hence this > variable. I hope that the methods will respect it so that the user can > control the result. Of course if some method is incapable of doing both > kinds of searches then there would be no choice. > > Now that I am sending this, perhaps the original bug resulted from the > default nil value for this variable? Ugh, that's what I get for not re-reading the whole thread every time I sit down to work on something like this. Yes, the problem (and the only problem) was that notmuch was constructing its thread queries incorrectly, and I'm pretty sure that problem is now fixed. `gnus-refer-thread-use-search' should definitely be respected. Let me think about this. If that variable is nil, search isn't used at all, in which case no code in gnus-search.el matters. If it is t, then search is used, but the "groups" parameter is set to nil, in which case the edit in the previously-attached patch doesn't matter. In fact the only thing the patch changes is how gnus-search behaves if the user enters a search query including "thread:t". So it was my message that was wrong. And my last confusion was thinking that I wasn't seeing a conflict between --thread and --duplicate=1, when in fact the attached patch simply never uses --thread. So, despite much dumbness, I think the patch is still correct. WDYT? Eric
Re: Gnorb gnorb-ebdb how to use instead of bbdb?
physiculus writes: > hello, > i want to use gnorb-ebdb instead of bbdb, because i do not use bbdb at > all. instead i successfully use ebdb with gnus.. > i loaded gnorb-ebdb after gnorb without errors. > > my emacs version is 28.0.90. > > but how could i use it? It doesn't do very much right now! I've started working on generalizing the contact management part of Gnorb, but haven't finished it, and have a pile of unreleased code related to this on one of my computers. Just keep it installed, and I'll update code before too long. :)
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
"Jose A. Ortega Ruiz" writes:
> Hi again,
>
> So, summing up what we have so far (patch attached), the effect of these
> changes is to make the errors go away, but i'm not sure if the behaviour
> is expected: with the patch applied, A T will "work" for me in an
> nnselect group, with the caveat that, when gnus-refer-thread-use-search
> is t (or when calling gnus-refer-thread with C-u otherwise), the result
> is limited to the email already present in the nnselect group... that
> might be expected, in which case i think the patch can be applied (it's
> basically what you already propose, Eric, except it constructs the query
> as an alist).
Okay, here goes the next try. A few things to note:
- I realized notmuch already has a "thread:{}" syntax that
does the double search I was doing in elisp, so now we just use that instead.
- In all my testing I couldn't see that having "duplicate=1" on thread
searches causes any problems, so I've taken it off. Can you please
doublecheck this? If it's still mucking it up for you, I'll put it
back in. I wish I really understood what the problem is (I think it
has to do with notmuch potentially storing the same message in
multiple locations, using symlinks).
- The search result filtration now won't filter on group names if the
search is a thread search. This should resolve the issue you were
seeing where "A T" would only search within the group you had searched
in to begin with. I guess I think that an explicit thread search by
the user should result in a full scan of the server. We can see if
that surprises/annoys anyone, though.
Thanks again,
Eric
diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index d64c0cb90c..4fb1fa3ec3 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -1352,7 +1352,10 @@ gnus-search-indexed-parse-output
server query groups)
(let ((prefix (or (slot-value engine 'remove-prefix)
""))
-(groups (mapcar #'gnus-group-short-name groups))
+(groups (unless (alist-get 'thread query)
+ ;; If we're searching threads, return messages from
+ ;; all groups.
+ (mapcar #'gnus-group-short-name groups)))
artlist article group)
(goto-char (point-min))
;; Prep prefix, we want to at least be removing the root
@@ -1529,6 +1532,17 @@ gnus-search-transform
(_query null))
"*")
+(cl-defmethod gnus-search-make-query-string :around ((_engine gnus-search-notmuch)
+ query-spec)
+ "Use Notmuch's thread:{} syntax.
+This packs the whole original query into , then
+returns all threads containing all matched messages."
+ (let ((query-string (cl-call-next-method))
+(threadp (alist-get 'thread query-spec)))
+(if threadp
+(format "thread:{%s}" query-string)
+ query-string)))
+
(cl-defmethod gnus-search-transform-expression ((engine gnus-search-notmuch)
(expr (head near)))
(format "%s near %s"
@@ -1586,55 +1600,22 @@ gnus-search-transform-expression
(format "date:%s.." (notmuch-date (cdr expr
(t (ignore-errors (cl-call-next-method))
-(cl-defmethod gnus-search-run-search :around ((engine gnus-search-notmuch)
- server query groups)
- "Handle notmuch's thread-search routine."
- ;; Notmuch allows for searching threads, but only using its own
- ;; thread ids. That means a thread search is a \"double-bounce\":
- ;; once to find the relevant thread ids, and again to find the
- ;; actual messages. This method performs the first \"bounce\".
- (if (alist-get 'thread query)
- (with-slots (program proc-buffer) engine
- (let* ((qstring
- (gnus-search-make-query-string engine query))
- (cp-list (gnus-search-indexed-search-command
- engine qstring query groups))
- thread-ids proc)
- (set-buffer proc-buffer)
- (erase-buffer)
- (setq proc (apply #'start-process (format "search-%s" server)
- proc-buffer program cp-list))
- (while (process-live-p proc)
- (accept-process-output proc))
- (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t)
- (push (match-string 1) thread-ids))
- (cl-call-next-method
- engine server
- ;; Completely replace the query with our new thread-based one.
- (mapconcat (lambda (thrd) (concat "thread:" thrd))
- thread-ids " or ")
- nil)))
-(cl-call-next-method engine server query groups)))
-
(cl-defmethod gnus-search-indexed-search-command ((engine gnus-search-notmuch)
(qstring string)
query _groups)
;; Theoretically we could use the GROUPS parameter to pass a
;; --folder switch to notmuch, but I'm not confident of getting the
;; format right.
- (let ((limit (alist-get 'limit query))
- (thread (alist-get 'thread query)))
+ (let ((limit (alist-get 'limit query)))
(with-slots (switches config-file) engine
- `(,(format "--config=%s" config-file)
-
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
"Jose A. Ortega Ruiz" writes: > On Wed, Dec 22 2021, Eric Abrahamsen wrote: > > [...] > >> My guess is that we need to go to point-min right here. >> >>> (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t) >>> (push (match-string 1) thread-ids)) >>> (cl-call-next-method >>>engine server >>>;; Completely replace the query with our new thread-based one. >>>(mapconcat (lambda (thrd) (concat "thread:" thrd)) >>> thread-ids " or ") >>>nil))) >>> (cl-call-next-method engine server query groups))) > > it's not enough. before that, there's a problem with invoking the > search for threads, because notmuch is passed the flag --duplicate=1, > and that's not accepted for format thread: > > Error: --duplicate=N is only supported with --output=files and > --output=messages. > > funny thing is that that flag is added for no apparent reason in line > 1633 of gnus-search.el: > > "--duplicate=1" ; I have found this necessary, I don't know why. > > i don't know why either because searches seem to work without it :). > if one eliminates that flag and adds your suggested (goto-char > (point-min)) we are not out the woods yet: the thread id is read, but > including an eol, so one needs > > (while (re-search-forward "^thread:\\([^\n ]+\\)" (point-max) t) > > > and then, we still fail, because that leads, somehow to an error of the > form: > > Debugger entered--Lisp error: (wrong-type-argument listp > "thread:0001a830") > alist-get(parsed-query "thread:0001a830") > #f(compiled-function (engine query-spec) # 0x1fd1b560df8c2360>)(# > "thread:0001a830") > apply(#f(compiled-function (engine query-spec) # 0x1fd1b560df8c2360>) # > "thread:0001a830") > gnus-search-make-query-string(# gnus-search-notmuch-157d5a02af62> "thread:0001a830") > > which indicates that the notmuch method for > gnus-search-make-query-string is buggy: it doesn't know how to parse > "thread:0001a830" queries... and here i ran out of steam, but > the fix seems nearby :) Here's what I've got so far, would you try it out? I'm not sure about notmuch not knowing how to parse a "thread:0001a830" query, it looks like it handles it fine to me: (let* ((engine (make-instance 'gnus-search-notmuch)) (query "thread:23434223455") (parsed-query (gnus-search-parse-query query))) (gnus-search-make-query-string engine `((query ,query) (parsed-query . ,parsed-query => "thread:23434223455" Am I misunderstanding something? diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el index d64c0cb90c..bb7fd47eb6 100644 --- a/lisp/gnus/gnus-search.el +++ b/lisp/gnus/gnus-search.el @@ -1606,13 +1606,17 @@ gnus-search-run-search proc-buffer program cp-list)) (while (process-live-p proc) (accept-process-output proc)) - (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t) - (push (match-string 1) thread-ids)) + (goto-char (point-min)) + (while (re-search-forward "^thread:\\([^[:space:]\n]+\\)" (point-max) t) + (cl-pushnew (match-string 1) thread-ids :test #'equal)) (cl-call-next-method engine server - ;; Completely replace the query with our new thread-based one. - (mapconcat (lambda (thrd) (concat "thread:" thrd)) - thread-ids " or ") + ;; If we found threads, completely replace the query with + ;; our new thread-based one. + (if thread-ids + (mapconcat (lambda (thrd) (concat "thread:" thrd)) + thread-ids " or ") + query) nil))) (cl-call-next-method engine server query groups))) @@ -1629,8 +1633,9 @@ gnus-search-indexed-search-command "search" ,(if thread "--output=threads" - "--output=files") - "--duplicate=1" ; I have found this necessary, I don't know why. + ;; No one knows why this "duplicate" flag is necessary, but + ;; it doesn't work with a thread search. + "--output=files --duplicate=1") ,@switches ,(if limit (format "--limit=%d" limit) "") ,qstring
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
"Jose A. Ortega Ruiz" writes: > On Wed, Dec 22 2021, Eric Abrahamsen wrote: > > [...] > >> My guess is that we need to go to point-min right here. >> >>> (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t) >>> (push (match-string 1) thread-ids)) >>> (cl-call-next-method >>>engine server >>>;; Completely replace the query with our new thread-based one. >>>(mapconcat (lambda (thrd) (concat "thread:" thrd)) >>> thread-ids " or ") >>>nil))) >>> (cl-call-next-method engine server query groups))) > > it's not enough. before that, there's a problem with invoking the > search for threads, because notmuch is passed the flag --duplicate=1, > and that's not accepted for format thread: > > Error: --duplicate=N is only supported with --output=files and > --output=messages. > > funny thing is that that flag is added for no apparent reason in line > 1633 of gnus-search.el: > > "--duplicate=1" ; I have found this necessary, I don't know why. Hmm, I inherited this code, and didn't know why it was necessary either, so I didn't take it out. I think notmuch can return multiple search results if the same message exists in multiple folders, but since we're filtering folders to begin with, maybe we don't need this. > i don't know why either because searches seem to work without it :). > if one eliminates that flag and adds your suggested (goto-char > (point-min)) we are not out the woods yet: the thread id is read, but > including an eol, so one needs > > (while (re-search-forward "^thread:\\([^\n ]+\\)" (point-max) t) > Obviously I was hoping this would Just Work™ without me having to actually install notmuch and test it. > and then, we still fail, because that leads, somehow to an error of the > form: > > Debugger entered--Lisp error: (wrong-type-argument listp > "thread:0001a830") > alist-get(parsed-query "thread:0001a830") > #f(compiled-function (engine query-spec) # 0x1fd1b560df8c2360>)(# > "thread:0001a830") > apply(#f(compiled-function (engine query-spec) # 0x1fd1b560df8c2360>) # > "thread:0001a830") > gnus-search-make-query-string(# gnus-search-notmuch-157d5a02af62> "thread:0001a830") > > which indicates that the notmuch method for > gnus-search-make-query-string is buggy: it doesn't know how to parse > "thread:0001a830" queries... and here i ran out of steam, but > the fix seems nearby :) You have done heroic work, and I hope it may yet be unnecessary for me to install and configure notmuch. Telling notmuch what to do with "thread:xxx" queries is not difficult, and I should have a solution for this in the next day or so. Thanks for the extensive debugging, Eric
Re: How to do you manage Threads, your inbox, your sent folder, your archive
dal-bla...@onenetbeyond.org writes: > Hi, > > I was reading : > https://libredd.it/r/emacs/comments/rhxfqx/threaded_and_syncced_emails_with_gnus_how_can_i/ > > I come from mu4e. With mu4e I never had to worry about threads, I could > enable or disable their display with a keybinding and that's was > it. Simpler times without sieve and dovecot. > > With gnus, I am a bit confuse. > > We have the variable `gnus-message-archive-method' and > `gnus-message-archive-group' to archive sent mails. > > We have also `gnus-posting-styles' to fill headers when composing a > mail, to eg, CC, BCC or GCC (what does exactly means GCC?), eventually > to myself or one of my folders. > > Then we can set `gnus-parameters' to `gcc-self' for some groups (but > here I am lost). > > In top of that if you use gmane.io, as another secondary method than you > imap/pop mail server, you eventually have to tweak > `gnus-refer-thread-use-search' to t and `gnus-refer-article-method' to > (something ?) to be able to retrieve your threads between back-ends. > > Is it possible to archive the whole threads of whom I initiate/reply ? > > What do you do with all theses variables ? > > For reference in my excursion in gnus, I read the dotfiles of jao and > jwiegley, if you'are kind enough to share yours, thank's you. It's hard to tell exactly what you're asking here: threads and archiving are completely separate concerns. I'm not sure what you mean by "archiving the whole threads". Archiving says what to do with your sent messages. By default, the `gnus-message-archive-method' and `gnus-message-archive-group' options will be combined to indicate where your sent messages should go. If any outgoing message has a GCC header, the sent message will be saved to that method+group instead, and the previous two options will be ignored. That's about sent messages. Threads... are just threads. You can toggle thread display with C-M-t. Are you asking about including your sent messages in the thread display? If you set `gnus-refer-thread-use-search' to t, Gnus will look in all groups in the current server for messages that belong to this thread, and pull them in. Your archive server will be different from gmane.io, but if you use "A T" in a gmane group, your own messages to the thread will also be fetched (from gmane, not from your archive server), so you'll still see the whole conversation. That's my best guess as to what you're after! Let me know what further questions you've got. Eric
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
Eric Abrahamsen writes: > "Jose A. Ortega Ruiz" writes: > >> On Tue, Dec 21 2021, Andrew Cohen wrote: >> >>>>>>>> "dal-blazej" == dal-blazej writes: >>> >>> [...] >>> >>> dal-blazej> To reproduce the issue : >>> >>> dal-blazej> 1. In the *server* buffer, use >>> dal-blazej> `gnus-group-read-ephemeral-search-group' with the query >>> dal-blazej> "from:j...@gnu.org" >>> >>> dal-blazej> 2. The first search succeed ; *in the ephemeral search >>> dal-blazej> buffer*, now use `gnus-summary-refer-thread' on an >>> dal-blazej> article. I get: (wrong-type-argument listp "") >>> >>> I don't know if this is related but I just fixed a rather rare bug in >>> thread referral on master (it happens when the subject of the message on >>> which you initiate the thread referral is nearly empty). >>> >>> You might give this a try to see if it fixes your problem. >> >> I've just been able to reproduce the problem in latest master, so i am >> afraid the answer is no :) This is the beginning of the backtrace I get >> when A T in a nnselect buffer (with notmuch as its engine): >> >> Debugger entered--Lisp error: (wrong-type-argument listp "") >> alist-get(parsed-query "") >> #f(compiled-function (engine query-spec) #> 0x99ea4e27204d6f9>)(# >> "") >> apply(#f(compiled-function (engine query-spec) #> 0x99ea4e27204d6f9>) # >> "") >> gnus-search-make-query-string(#> gnus-search-notmuch-1556d745508e> "") >> #f(compiled-function (engine server query groups) "Run QUERY against >> SERVER using ENGINE.\nThis method is common to all indexed search >> engines.\n\nReturns a list of [group article score] vectors." >> #)(#> gnus-search-notmuch-1556d745508e> "nnml:" "" nil) >> apply(#f(compiled-function (engine server query groups) "Run QUERY >> against SERVER using ENGINE.\nThis method is common to all indexed >> search engines.\n\nReturns a list of [group article score] vectors." >> #) (#> gnus-search-notmuch-1556d745508e> "nnml:" "" nil)) >> #f(compiled-function ( cnm-args) #> 0x1b59543183000dee>)(#> gnus-search-notmuch-1556d745508e> "nnml:" "" nil) >> #f(compiled-function (cl--cnm engine server query groups) "Handle >> notmuch's thread-search routine." #> 0x1d4ee8fb5ee95069>)(#f(compiled-function ( cnm-args) #> 0x1b59543183000dee>) #> gnus-search-notmuch-1556d745508e> "nnml:" ((parsed-query (or (id . >> "CAAwB6VDPO=pT-XU8cPNfZRtxXC48LD-OXepiGejb-+QAzoymm...") (or (id . >> "t7sEZR5CwAuextCqAQyuzZ4N-BAsbVMaKTaGiG6o56GNhGyz0U...") (or (id . >> "Ivxz0PU9N9S5LLCmCuI5-UE8pt2AVamHqzLc3LibJxLdYCvo1X...") (or (id . >> "a2wBK5wGXfG6kT2lSke4Mmqc14JC2SLMqP2nsMxZAizapFFBuH...") (id . >> "V00BOpL5XPuHuS_tIuCyc-9zxFq7mR1gLUgK-BhIUZjJOXoFdr...")) (query . >> "id:> nil) > > This is a bit hard to look at, but it seems like the problem is in > notmuch's thread-specific search handling. What is supposed to happen > is that notmuch first uses those message-ids to find the messages, then > finds the thread: statements for all of the search results, then > does > *another* search using the thread ids from the first search. So > obviously something's going wrong with that. The relevant method is > currently line 1589 in gnus-search.el, I'm also pasting it below. Would > you mind edebugging it and stepping through to see which part exactly is > wrong? > > Thanks. > > (cl-defmethod gnus-search-run-search :around ((engine gnus-search-notmuch) > server query groups) > "Handle notmuch's thread-search routine." > ;; Notmuch allows for searching threads, but only using its own > ;; thread ids. That means a thread search is a \"double-bounce\": > ;; once to find the relevant thread ids, and again to find the > ;; actual messages. This method performs the first \"bounce\". > (if (alist-get 'thread query) > (with-slots (program proc-buffer) engine > (let* ((qstring > (gnus-search-make-query-string engine query)) > (cp-list (gnus-search-indexed-search-command >engine qstring query groups)) > thread-ids proc) > (set-buffer proc-buffer) > (erase-buffer) > (setq proc (apply #'start-process (format "search-%s" server) > proc-buffer program cp-list)) > (while (process-live-p proc) > (accept-process-output proc)) My guess is that we need to go to point-min right here. > (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t) > (push (match-string 1) thread-ids)) > (cl-call-next-method > engine server > ;; Completely replace the query with our new thread-based one. > (mapconcat (lambda (thrd) (concat "thread:" thrd)) > thread-ids " or ") > nil))) > (cl-call-next-method engine server query groups)))
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
"Jose A. Ortega Ruiz" writes: > On Tue, Dec 21 2021, Andrew Cohen wrote: > >>> "dal-blazej" == dal-blazej writes: >> >> [...] >> >> dal-blazej> To reproduce the issue : >> >> dal-blazej> 1. In the *server* buffer, use >> dal-blazej> `gnus-group-read-ephemeral-search-group' with the query >> dal-blazej> "from:j...@gnu.org" >> >> dal-blazej> 2. The first search succeed ; *in the ephemeral search >> dal-blazej> buffer*, now use `gnus-summary-refer-thread' on an >> dal-blazej> article. I get: (wrong-type-argument listp "") >> >> I don't know if this is related but I just fixed a rather rare bug in >> thread referral on master (it happens when the subject of the message on >> which you initiate the thread referral is nearly empty). >> >> You might give this a try to see if it fixes your problem. > > I've just been able to reproduce the problem in latest master, so i am > afraid the answer is no :) This is the beginning of the backtrace I get > when A T in a nnselect buffer (with notmuch as its engine): > > Debugger entered--Lisp error: (wrong-type-argument listp "") > alist-get(parsed-query "") > #f(compiled-function (engine query-spec) # 0x99ea4e27204d6f9>)(# > "") > apply(#f(compiled-function (engine query-spec) # 0x99ea4e27204d6f9>) # > "") > gnus-search-make-query-string(# gnus-search-notmuch-1556d745508e> "") > #f(compiled-function (engine server query groups) "Run QUERY against > SERVER using ENGINE.\nThis method is common to all indexed search > engines.\n\nReturns a list of [group article score] vectors." > #)(# gnus-search-notmuch-1556d745508e> "nnml:" "" nil) > apply(#f(compiled-function (engine server query groups) "Run QUERY > against SERVER using ENGINE.\nThis method is common to all indexed > search engines.\n\nReturns a list of [group article score] vectors." > #) (# gnus-search-notmuch-1556d745508e> "nnml:" "" nil)) > #f(compiled-function ( cnm-args) # 0x1b59543183000dee>)(# gnus-search-notmuch-1556d745508e> "nnml:" "" nil) > #f(compiled-function (cl--cnm engine server query groups) "Handle > notmuch's thread-search routine." # 0x1d4ee8fb5ee95069>)(#f(compiled-function ( cnm-args) # 0x1b59543183000dee>) # gnus-search-notmuch-1556d745508e> "nnml:" ((parsed-query (or (id . > "CAAwB6VDPO=pT-XU8cPNfZRtxXC48LD-OXepiGejb-+QAzoymm...") (or (id . > "t7sEZR5CwAuextCqAQyuzZ4N-BAsbVMaKTaGiG6o56GNhGyz0U...") (or (id . > "Ivxz0PU9N9S5LLCmCuI5-UE8pt2AVamHqzLc3LibJxLdYCvo1X...") (or (id . > "a2wBK5wGXfG6kT2lSke4Mmqc14JC2SLMqP2nsMxZAizapFFBuH...") (id . > "V00BOpL5XPuHuS_tIuCyc-9zxFq7mR1gLUgK-BhIUZjJOXoFdr...")) (query . > "id: nil) This is a bit hard to look at, but it seems like the problem is in notmuch's thread-specific search handling. What is supposed to happen is that notmuch first uses those message-ids to find the messages, then finds the thread: statements for all of the search results, then does *another* search using the thread ids from the first search. So obviously something's going wrong with that. The relevant method is currently line 1589 in gnus-search.el, I'm also pasting it below. Would you mind edebugging it and stepping through to see which part exactly is wrong? Thanks. (cl-defmethod gnus-search-run-search :around ((engine gnus-search-notmuch) server query groups) "Handle notmuch's thread-search routine." ;; Notmuch allows for searching threads, but only using its own ;; thread ids. That means a thread search is a \"double-bounce\": ;; once to find the relevant thread ids, and again to find the ;; actual messages. This method performs the first \"bounce\". (if (alist-get 'thread query) (with-slots (program proc-buffer) engine (let* ((qstring (gnus-search-make-query-string engine query)) (cp-list (gnus-search-indexed-search-command engine qstring query groups)) thread-ids proc) (set-buffer proc-buffer) (erase-buffer) (setq proc (apply #'start-process (format "search-%s" server) proc-buffer program cp-list)) (while (process-live-p proc) (accept-process-output proc)) (while (re-search-forward "^thread:\\([^ ]+\\)" (point-max) t) (push (match-string 1) thread-ids)) (cl-call-next-method engine server ;; Completely replace the query with our new thread-based one. (mapconcat (lambda (thrd) (concat "thread:" thrd)) thread-ids " or ") nil))) (cl-call-next-method engine server query groups)))
Re: gnus-search-engine set to gnus-search-notmuch and refer threads
dal-bla...@onenetbeyond.org writes: > To be sure I upgraded both softwares : > > emacs-version 29.0.50 > notmuch 0.34.2+35~ged03bab > > > Maybe my first mail was a bit confuse ? > > To reproduce the issue : > > 1. In the *server* buffer, use `gnus-group-read-ephemeral-search-group' > with the query "from:j...@gnu.org" > > 2. The first search succeed ; *in the ephemeral search buffer*, now use > `gnus-summary-refer-thread' on an article. I get: > (wrong-type-argument listp "") Thanks for keeping up with this. I can't immediately reproduce this. What's your value of `gnus-refer-thread-use-search'?
Re: question about org-link gnus message and gnus registry
physiculus writes: > hello, > i'm just struggeling about the consistency of an org-link to a gnus message > after moving this message into another folder on same server or a > different one. > > scenario: > i create a org link with org and insert the link into an org file. > works. > > now i move the message inside gnus to another folder. > > open the link inside the org file is not valid anymore. > > i thought, that the registry is responsible for saving the position of > any message. therefore it saves the message-id and use this to locate > the message. > > and i thought, that org-link "knows" this message-id, though the link to > the message is valid without the position of the folder or better the > position of the message inside the folder. > like a database index. > but now i'm quite sure, that my thoughts were completely wrong?? > or is it possible to locate a message through org-link? To your original question! Yes, theoretically `org-gnus-follow-link' could try to locate the message the regular way, and if that doesn't work, could check `gnus-registry-enabled' and then try `gnus-registry-get-id-key' to get the new group. Someone would have to write that code, though! Eric
Re: question about org-link gnus message and gnus registry
Michael Heerdegen writes: > Eric Abrahamsen writes: > >> Crap, it's in a commit I've had locally since... late 2020, but never >> released. > > Then now it's one of the best tested commits ever ;-) > > I have a lot of these myself... When we say battle-tested...
Re: question about org-link gnus message and gnus registry
Michael Heerdegen writes: > Eric Abrahamsen writes: > >> I'm still just seeing a call to `cl-second', which is required at >> compile time. And my, there are a lot of compiler warnings! But nothing >> about second or cl-second... > > This is where I found it: > > #+begin_src emacs-lisp > (defun gnorb-reply-to-gnus-link (link) > "Start a reply to the linked message." > (let* ((link (org-link-unescape link)) >(group (car (org-split-string link "#"))) >(id (gnorb-bracket-message-id > (second (org-split-string link "#" >(backend > (car (gnus-find-method-for-group group > (gnorb-follow-gnus-link group id) > (call-interactively > (if (eq backend 'nntp) >'gnus-summary-followup-with-original >'gnus-summary-wide-reply-with-original > #+end_src Crap, it's in a commit I've had locally since... late 2020, but never released. Sorry about that, I'll do another version release in a moment. Thanks for finding this! Eric
Re: question about org-link gnus message and gnus registry
Michael Heerdegen writes: > Michael Heerdegen writes: > >> And indeed - look at `gnorb-reply-to-gnus-link'. > > OTOH you can also simply look at the compiler warnings... I'm still just seeing a call to `cl-second', which is required at compile time. And my, there are a lot of compiler warnings! But nothing about second or cl-second...
Re: question about org-link gnus message and gnus registry
physiculus writes:
> Am Do, 2021-12-02, 11:20 -0800, Eric Abrahamsen
> schrieb:
>
> hello,
> the version is 1.6.9 from elpa, installed today.
>
> this is the backtrace:
> Debugger entered--Lisp error: (void-function second)
> signal(void-function (second))
> gnorb-org-setup-message(("nnimap+GMail:gnus#87o860krsm@googlemail.com")
> nil nil nil nil nil nil "65281282-7850-4f1f-b9e3-e18413932a1a")
> gnorb-org-handle-mail(nil)
> funcall-interactively(gnorb-org-handle-mail nil)
> call-interactively(gnorb-org-handle-mail record nil)
> command-execute(gnorb-org-handle-mail record)
> execute-extended-command(nil "gnorb-org-handle-mail" nil)
> funcall-interactively(execute-extended-command nil "gnorb-org-handle-mail"
> nil)
> call-interactively(execute-extended-command nil nil)
> command-execute(execute-extended-command)
I honestly don't know what's going on here. There's no call to `second'
in `gnorb-org-setup-message', not even a call to `cl-second'. Are you
comfortable with stepping through functions using edebug? That might be
the best way of getting to the bottom of it.
Re: question about org-link gnus message and gnus registry
physiculus writes: > Am Mi, 2021-12-01, 13:47 -1000, Bob Newell schrieb: > > hello, > thanks for the reply. > i gave gnorb a try, but if i move a message, that is linked with gnorb, > i got the following error: > gnorb-org-setup-message: Symbol’s function definition is void: second > > do you know what it means? All calls to `second' in the Gnorb codebase should have been replaced by `cl-second' years ago (like in 2014). Can you tell me your Gnorb version, and also run M-x toggle-debug-on-error and trigger the error again, then send me the backtrace? Thanks, Eric
Re: Filtering out calendar invites?
Lars-Johan Liman writes: > Whooa! > > I'm backing up! My hack actually seems to work! ;-) > > In case anyone googles this thread: this seems to work for me. Happy to > hear feedback from others. > > I put this as one of the entries in my nnmail-split-fancy variable. It's > a bit crude as it doesn't assert that the string is actually a MIME tag > and not just part of the text in the message, If you really want to go down a rabbit hole, you can call the function `gnus-article-mime-handles' while gnus-article-buffer is current. That ought to give you a full list of the actual mime attachments on the message.
Re: gnus-search & imap: always "CHARSET UTF-8" when literal+ is supported
David Edmondson writes:
> On Friday, 2021-10-22 at 10:47:42 -07, Eric Abrahamsen wrote:
>
>> David Edmondson writes:
>>
>>> Using current emacs git head, talking to outlook.office365.com over
>>> IMAP.
>>>
>>> Attempts to use gnus-search always fail with the server reporting:
>>>
>>> (("NO" ("BADCHARSET" "(US-ASCII)") "The" "specified" "charset" "is" "not"
>>> "supported."))
>>>
>>> Looking at gnus-search.el, `gnus-search-imap-search-command' always
>>> sends CHARSET UTF-8 if the server supports literal+ (which this one
>>> does). Sending US-ASCII (or no charset at all) causes the server to
>>> return the required results in simple test cases.
>>>
>>> Is there a way to determine whether a server supports UTF-8 in searches,
>>> and adjust the command sent accordingly? If not, could the use of UTF-8
>>> be controlled with (yet another!) variable?
>>
>> A bit of internet research seems to indicate that Exchange can't handle
>> UTF-8 encoded search strings, and also there's no way to test that in
>> advance apart from simply seeing if it errors. Awesome!
>
> That's also the impression I gained.
>
>> I think what this means is that it's impossible to search for non-ascii
>> text on an Exchange server (can that be true?!). If that's true, then
>> the imap search command should be using the presence of a multibyte
>> string as the test for whether to use CHARSET UTF-8 or not. You're not
>> going to be able to search for a multibyte string, anyway.
>>
>> Would you try eval'ling the below, and tell me if it works okay when
>> searching for a string with no non-ascii characters in it?
>
> This works in a few simple tests, yes. Thanks!
Great, I'll put this change in.
>> Also, when you do get the error message above, how does that present to
>> the user? Did you have to go digging to find it?
>
> I had to dig. The observed behaviour is that no messages match the
> search.
I'm hoping to make some changes to Gnus error reporting that should make
this less difficult. The user should definitely see the difference
between "no results" and an actual error.
Thanks for the report.
Re: gnus-search & imap: always "CHARSET UTF-8" when literal+ is supported
David Edmondson writes:
> Using current emacs git head, talking to outlook.office365.com over
> IMAP.
>
> Attempts to use gnus-search always fail with the server reporting:
>
> (("NO" ("BADCHARSET" "(US-ASCII)") "The" "specified" "charset" "is" "not"
> "supported."))
>
> Looking at gnus-search.el, `gnus-search-imap-search-command' always
> sends CHARSET UTF-8 if the server supports literal+ (which this one
> does). Sending US-ASCII (or no charset at all) causes the server to
> return the required results in simple test cases.
>
> Is there a way to determine whether a server supports UTF-8 in searches,
> and adjust the command sent accordingly? If not, could the use of UTF-8
> be controlled with (yet another!) variable?
A bit of internet research seems to indicate that Exchange can't handle
UTF-8 encoded search strings, and also there's no way to test that in
advance apart from simply seeing if it errors. Awesome!
I think what this means is that it's impossible to search for non-ascii
text on an Exchange server (can that be true?!). If that's true, then
the imap search command should be using the presence of a multibyte
string as the test for whether to use CHARSET UTF-8 or not. You're not
going to be able to search for a multibyte string, anyway.
Would you try eval'ling the below, and tell me if it works okay when
searching for a string with no non-ascii characters in it?
Also, when you do get the error message above, how does that present to
the user? Did you have to go digging to find it?
(cl-defmethod gnus-search-imap-search-command ((engine gnus-search-imap)
(query string))
"Create the IMAP search command for QUERY.
Currently takes into account support for the LITERAL+ capability.
Other capabilities could be tested here."
(with-slots (literal-plus) engine
(when (and literal-plus
(string-match-p "\n" query))
(setq query (split-string query "\n")))
(cond
((consp query)
;; We're not really streaming, just need to prevent
;; `nnimap-send-command' from waiting for a response.
(let* ((nnimap-streaming t)
(call
(nnimap-send-command
"UID SEARCH CHARSET UTF-8 %s"
(pop query
(dolist (l query)
(process-send-string (get-buffer-process (current-buffer)) l)
(process-send-string (get-buffer-process (current-buffer))
(if (nnimap-newlinep nnimap-object)
"\n"
"\r\n")))
(nnimap-get-response call)))
(t (nnimap-command "UID SEARCH %s" query)
Re: Mail source unreachable - continue yes/no?
Eric Abrahamsen writes: > Lars Ingebrigtsen writes: > >> Eric Abrahamsen writes: >> >>> The pop-up part of it is very easy to fix, it could even be added to >>> the Gnus window configuration stuff so people can have the >>> warnings/reports visible in *Group* but not elsewhere, etc. >> >> Perhaps it could be an opt-in thing. > > I'll work something up. `nnheader-report' is fine specifically for > backend-related errors, but it seems like this could fill a nice gap for > "other errors" that aren't necessarily related to a particular backend. I'm going to open a separate bug report for this.
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> In the meantime, what do you think about removing the call to `error' at >> the end of `nntp-report'? > > I'm not sure what the repercussions would be, but it sounds like it > should be possible. I'm running it now, and will push in a couple days if it looks okay.
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> The pop-up part of it is very easy to fix, it could even be added to >> the Gnus window configuration stuff so people can have the >> warnings/reports visible in *Group* but not elsewhere, etc. > > Perhaps it could be an opt-in thing. I'll work something up. `nnheader-report' is fine specifically for backend-related errors, but it seems like this could fill a nice gap for "other errors" that aren't necessarily related to a particular backend. In the meantime, what do you think about removing the call to `error' at the end of `nntp-report'?
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> Looking over the code, I'm inclined to agree with Lars-Johan here: there >> isn't really any need to halt the process, what's important is that the >> user be made aware of the failure. > > I agree. > >> Allow me to re-introduce my suggestion of using warnings! It's looking >> better and better the more I consider it. `delay-warning' is just what >> we want: it puts messages in the hopper, which aren't displayed until >> the current command is completely finished, instead of messages >> clobbering each other and getting buried. It has its own private buffer, >> keeping information separate. There are plenty of user-facing knobs, and >> facilities for hiding or silencing the warnings. > > I'm not sure I want to be popping up a buffer at the user for network > errors and the like -- it's expected that a news reader will have some > network problems, and putting up a buffer about it isn't very helpful. The pop-up part of it is very easy to fix, it could even be added to the Gnus window configuration stuff so people can have the warnings/reports visible in *Group* but not elsewhere, etc. The real advantages are 1) the warnings go into their own buffer, so they don't get lost in *Messages*, and 2) they can be displayed after the current command returns. Right now, the message about mail source failure message will never be seen by the user. We could easily emulate that behavior, it's just that warnings give it to us for free. There are other approaches, as well. I could also do a limited version of the custom error approach, only for mail sources.
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> In the meantime I got a backtrace on my stop-the-world nntp connection >> error, which I've posted below. I guess there's nothing mysterious about >> it -- the process dies (I can't tell if it's actually the timeout doing >> it or not), and nntp-accept-process-output/nntp-report ends up raising >> the error, and there's nothing up the line to catch it. Does this look >> surprising or wrong for any reason? >> >> (if nntp--report-1 (progn (if nntp-record-commands (progn >> (nntp-record-command "*** CONNECTION LOST ***"))) (throw > > Network errors are common, so it shouldn't be throwing an error in the > first place. > > But I can't recall ever seeing the `nntp-report' function before, so who > knows what the logic is. :-/ That looks like a... debugging function? The function has existed and raised an error since the "dawn of time" (since CVS days), I guess I'm surprised this hasn't annoyed anyone else. Basically it shadows `nnheader-report', and gives the server a single chance to reconnect in case of failure -- the "nntp-with-open-group" mechanism -- before failing altogether. All that's fine, I wish nnimap had something similar, but raising the error seems wrong. In fact raising the error would be the right thing in the code sketch I posted above! This code is ahead of its time. Anyway, simply removing the call to error should do the trick. That will leave `nnheader-report' as the final form, which returns nil. `nntp-report' is the final form everywhere it's called, so the nil goes up to callers, which will (mostly) interpret that as failure. Eric
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes:
> Eric Abrahamsen writes:
>
>> On second thought you're not wrong here: the various *-open-server
>> deffoos report their own failures with `nnheader-report' instead of
>> signaling errors, and then `gnus-open-server' swallows any other errors
>> and converts them to messages.
>
> Ah. I knew there was something like that somewhere. :-)
>
>> Failures arrived at via `gnus-request-scan' mostly show up as actual
>> errors.
>
> Perhaps we should adjust those errors to be nnheader-report thingies
> instead?
>
>> - When the gnus-open-server deffoo processes fail, they (or their
>> downstream functions) signal 'gnus-server-connection-error. (I didn't
>> go through and do this for every backend, just enough to get the
>> idea.)
>> - Mail source fetching can fail, in which case 'gnus-mail-source-error
>> is signaled. nnmail.el no longer handles this error: it propagates up
>> and out through gnus-request-scan.
>
> I've just skimmed the patch, but the approach looks sound to me (in
> general).
>
>> Some observations and questions:
>>
>> - There are many places and depth-levels in Gnus where failure modes are
>> dealt with as messages (in effect working like `with-demoted-errors')
>> or as nil return values from functions, which are then converted into
>> messages, or sometimes even re-signaled as an actual `error' with a
>> new message string. This often ends up burying real errors, and/or
>> making debug on error hard to use. We could try flattening this out:
>> low-level functions signal errors, and only top-level functions in
>> gnus-start/gnus-group/gnus-sum get to catch them at the last minute,
>> and only if necessary (other layers can of course catch and re-signal
>> the errors if some cleanup work needs to be done). I have no idea if
>> this would end up working out, but I think it would be good to try.
>> And more use of `condition-case-unless-debug' at the top level.
>
> I doubt it's possible to find an overarching strategy to handle all
> kinds of errors on the "Gnus" level. Different errors are, well,
> different.
Yeah, I'm trying to decide if it's worth introducing this new approach
at all. I'd hate to put in something that ends up only being useful in
49% of cases, and now there's yet another system that future code readers
have to make sense of.
I'm also not sure it solves enough problems. For instance, I recently
committed something to nnimap.el that cleans up `nnimap-process-buffers'
when an IMAP connection dies. That happens in like three or four
different places. Ideally a signal handling approach would let us do
that cleanup in a single spot -- either an actual single spot, or a
macro we only have to write once. Kind of a generalization of
`nntp-with-open-group', except it catches any instance of
'gnus-server-error, rather than requiring an explicitly-thrown symbol. I
don't quite see that this is feasible right now.
ANYWAY... I sent a patch suggestion for the mail-source problem in a
separate message. I'm kind of waffling on the rest of this custom error
stuff.
In the meantime I got a backtrace on my stop-the-world nntp connection
error, which I've posted below. I guess there's nothing mysterious about
it -- the process dies (I can't tell if it's actually the timeout doing
it or not), and nntp-accept-process-output/nntp-report ends up raising
the error, and there's nothing up the line to catch it. Does this look
surprising or wrong for any reason?
(if nntp--report-1 (progn (if nntp-record-commands (progn
(nntp-record-command "*** CONNECTION LOST ***"))) (throw
'nntp-with-open-group-error t)) (if nntp-record-commands (progn
(nntp-record-command "*** CALLED nntp-report ***"))) (nnheader-report 'nntp
args) (debug) (apply #'error args))
nntp-report("Server closed connection")
nntp-accept-process-output(#)
nntp-accept-response()
#f(compiled-function () #)()
funcall(#f(compiled-function () #))
(condition-case nil (funcall bodyfun) (quit (if debug-on-quit nil
(nntp-close-server)) (signal 'quit nil)))
(setq nntp-with-open-group-internal (condition-case nil (funcall bodyfun)
(quit (if debug-on-quit nil (nntp-close-server)) (signal 'quit nil
(unwind-protect (setq nntp-with-open-group-internal (condition-case nil
(funcall bodyfun) (quit (if debug-on-quit nil (nntp-close-server)) (signal
'quit nil (if timer (progn (cancel-timer timer
(let ((timer (and nntp-connection-timeout (run-at-time
nntp-connection-timeout nil #'(lambda nil (let* ... ...)) (unwind-protect
(setq nntp-with-open-group-internal (condition-case nil (funcall bodyfun) (quit
(if debug-on-quit nil (nntp-close-server)) (signal 'quit nil (if timer
(progn (cancel-timer timer))
Re: Mail source unreachable - continue yes/no?
Lars-Johan Liman writes: > Lol! > > Thanks. OK, so I wasn't off the mark, then. :-) > > I'm not even sure this needs to be fixed. It's good that the fact that > the server couldn't be reached is signalled, but continuing seems like > the right thing to do. The message is somewhat misleading though. How > about changing the "question" to a "Please ack!" of some kind? Looking over the code, I'm inclined to agree with Lars-Johan here: there isn't really any need to halt the process, what's important is that the user be made aware of the failure. I'm trying to imagine why the user would _need_ to halt things here. Unless we've got some sort of restart situation, where the user can eg put in the correct password and try again, it doesn't seem useful. Allow me to re-introduce my suggestion of using warnings! It's looking better and better the more I consider it. `delay-warning' is just what we want: it puts messages in the hopper, which aren't displayed until the current command is completely finished, instead of messages clobbering each other and getting buried. It has its own private buffer, keeping information separate. There are plenty of user-facing knobs, and facilities for hiding or silencing the warnings. See attached! diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el index 606bd3a39a..3c1387b73f 100644 --- a/lisp/gnus/gnus-start.el +++ b/lisp/gnus/gnus-start.el @@ -45,6 +45,14 @@ gnus-agent-file-loading-local (defvar gnus-agent-file-loading-cache) (defvar gnus-topic-alist) +(define-error 'gnus-error "Gnus error") +(define-error 'gnus-mail-source-error + "Gnus mail source error" 'gnus-error) +(define-error 'gnus-server-error + "Gnus server error" 'gnus-error) +(define-error 'gnus-server-connection-error + "Error connecting to server" 'gnus-server-error) + (defcustom gnus-startup-file (nnheader-concat gnus-home-directory ".newsrc") "Your `.newsrc' file. `.newsrc-SERVER' will be used instead if that exists." @@ -1604,7 +1612,8 @@ gnus-get-unread-articles (gnus-agent-article-local-times 0) (archive-method (gnus-server-to-method "archive")) info group active method cmethod - method-type method-group-list entry) + method-type method-group-list entry + failed-methods) (gnus-message 6 "Checking new news...") (while newsrc @@ -1685,8 +1694,11 @@ gnus-get-unread-articles gnus-secondary-select-methods)) (when (and (not (assoc method type-cache)) (gnus-check-backend-function 'request-list (car method))) - (with-current-buffer nntp-server-buffer - (gnus-read-active-file-1 method nil) + (condition-case-unless-debug err + (with-current-buffer nntp-server-buffer + (gnus-read-active-file-1 method nil)) +(gnus-server-connection-error + (push (cons method err) failed-methods)) ;; Clear out all the early methods. (dolist (elem type-cache) @@ -1697,11 +1709,15 @@ gnus-get-unread-articles 'retrieve-group-data-early (car method)) (not (gnus-method-denied-p method))) (when (ignore-errors (gnus-get-function method 'open-server)) - (unless (gnus-server-opened method) - (gnus-open-server method)) - (when (gnus-server-opened method) - ;; Just mark this server as "cleared". - (gnus-retrieve-group-data-early method nil)) +(condition-case-unless-debug err + (progn + (unless (gnus-server-opened method) + (gnus-open-server method)) + ;; Just mark this server as "cleared". + (gnus-retrieve-group-data-early method nil)) + (gnus-server-connection-error + (push (cons method err) failed-methods) + (setq type-cache (delq elem type-cache ;; Start early async retrieval of data. (let ((done-methods nil) @@ -1725,7 +1741,6 @@ gnus-get-unread-articles ;; be unique at this point, but apparently it ;; does happen in the wild with some setups. (not (member sanity-spec done-methods)) - (gnus-server-opened method) (gnus-check-backend-function 'retrieve-group-data-early (car method))) (push sanity-spec done-methods) @@ -1744,12 +1759,23 @@ gnus-get-unread-articles (let ((updatep (gnus-check-backend-function 'request-update-info (car method ;; See if any of the groups from this method require updating. - (gnus-read-active-for-groups method infos early-data) - (dolist (info infos) - (inline (gnus-get-unread-articles-in-group - info (gnus-active (gnus-info-group info)) - updatep))) -(gnus-message 6 "Checking new news...done"))) +(condition-case-unless-debug err +(progn + (gnus-read-active-for-groups method infos early-data) + (dolist (info infos) + (inline (gnus-get-unread-articles-in-group + info (gnus-active (gnus-info-group info)) +
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> I put plain old calls to `error' inside `nnimap-request-scan', and that >> also derailed the "g" update process. So whatever handling there once >> was either is not around this particular piece of code, or else it might >> have gone away at some point. > > Yeah, perhaps it went missing at some point. > >> Are you likely to recall further how this once worked? :) I can start >> sketching out some custom errors, otherwise. > > I'm not sure about anything. I do recall (in the olden days) getting a > humongous appended error message from all the backends if the network > went down or something, though. On second thought you're not wrong here: the various *-open-server deffoos report their own failures with `nnheader-report' instead of signaling errors, and then `gnus-open-server' swallows any other errors and converts them to messages. Failures arrived at via `gnus-request-scan' mostly show up as actual errors. I still haven't looked into why nntp-connection-timeout ends up interrupting everything. > Custom errors sound good, though. Okay, here's a _very_ rough first sketch. There are a handful of errors defined, mostly for connection problems. I didn't do any handling for authentication failures, because that's a fairly distinct system, and I'm not sure there's any need to mess with it (I haven't tested what auth failures do to Gnus update), anyway that can be left for later. A few things happen in this patch: - When the gnus-open-server deffoo processes fail, they (or their downstream functions) signal 'gnus-server-connection-error. (I didn't go through and do this for every backend, just enough to get the idea.) - Mail source fetching can fail, in which case 'gnus-mail-source-error is signaled. nnmail.el no longer handles this error: it propagates up and out through gnus-request-scan. - There are `condition-case' wrappers in `gnus-get-unread-articles' now, which seems to be the main place where it's important not to interrupt a larger loop. Most other places the user is just "doing one thing", and it seems okay to let the error reach them directly. Some observations and questions: - There are many places and depth-levels in Gnus where failure modes are dealt with as messages (in effect working like `with-demoted-errors') or as nil return values from functions, which are then converted into messages, or sometimes even re-signaled as an actual `error' with a new message string. This often ends up burying real errors, and/or making debug on error hard to use. We could try flattening this out: low-level functions signal errors, and only top-level functions in gnus-start/gnus-group/gnus-sum get to catch them at the last minute, and only if necessary (other layers can of course catch and re-signal the errors if some cleanup work needs to be done). I have no idea if this would end up working out, but I think it would be good to try. And more use of `condition-case-unless-debug' at the top level. - There are many ways of logging, and messaging the user. Apart from `message' itself, there's `nnheader-report', `gnus-backend-trace', `nnheader-message(-maybe)', `gnus-message' and its action message log, along with `gnus-final-warning' and `gnus-error'. nnheader-report, in particular, seems to often be used as a substitute for actually signaling an error. A server is asked to do something, it fails for some reason and logs the reason with nnheader-report, then returns nil. The caller sees the nil, then retrieves the string with nnheader-get-report, and uses it to signal an error. I'm not claiming to have thought this through completely, much less tested anything, but I wonder if in most cases the underlying code couldn't just signal an error directly. - Lastly, I wanted to try out the warning facilities here, just to see if they might provide a good tool. nnheader-message/gnus-message/gnus-error could dispatch to display-warning. Anyway, those are some thoughts that occurred while poking around with this. diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el index 606bd3a39a..3c1387b73f 100644 --- a/lisp/gnus/gnus-start.el +++ b/lisp/gnus/gnus-start.el @@ -45,6 +45,14 @@ gnus-agent-file-loading-local (defvar gnus-agent-file-loading-cache) (defvar gnus-topic-alist) +(define-error 'gnus-error "Gnus error") +(define-error 'gnus-mail-source-error + "Gnus mail source error" 'gnus-error) +(define-error 'gnus-server-error + "Gnus server error" 'gnus-error) +(define-error 'gnus-server-connection-error + "Error connecting to server" 'gnus-server-error) + (defcustom gnus-startup-file (nnheader-concat gnus-home-directory ".newsrc") "Your `.newsrc' file. `.newsrc-SERVER' will be used instead if that exists." @@ -1604,7 +1
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> I've got my nntp server set in `gnus-select-method', maybe that's why? >> This has annoyed me off and on for years but I've never taken the time >> to look into it. My other servers are all localhost nnimap and I never >> see errors there. I just set debug-on-message appropriately, and will >> figure out exactly where things are going off. > > Tried > > (setq gnus-select-method '(nntp "localhost")) > > and that worked fine, too. There's probably errors that aren't > handled well, but there's infrastructure to collect all the erroring > backends and display the errors at the end. Well, at least there used > to be; haven't actually looked at the code now. :-) I put plain old calls to `error' inside `nnimap-request-scan', and that also derailed the "g" update process. So whatever handling there once was either is not around this particular piece of code, or else it might have gone away at some point. I had had `nntp-connection-timeout' set to 6, as it hangs indefinitely otherwise. I stuck a call to `error' inside `nntp-report'; so far I can't get an actual backtrace, I guess because there _is_ error handling for nntp. But even there I don't think it's "flow control" handling, it appears to just be error demotion. Are you likely to recall further how this once worked? :) I can start sketching out some custom errors, otherwise. Eric
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> `nnmail-get-new-mail' is called from within `gnus-request-scan', and >> there's no error ignoring around `gnus-request-scan' in >> `gnus-get-unread-articles' -- is there? I can't see anything. I would >> propose removing error handling from `nnmail-get-new-mail-1': if the >> fetching of mail sources well and truly fails (and the user hits "no" to >> give up on the process), it should just signal an error up the line. > > Yes, I think so... but isn't there error handling in gnus-request-scan? > I really thought there was... You mean in `gnus-get-unread-articles', around `gnus-request-scan'? No, there doesn't seem to be. In other places where `gnus-request-scan' is called, the code is mostly working with a single group, in which case you could argue that errors should just bubble up to the user. >> I think nntp servers are louder than the others. When I hit "g" and the >> nntp connection times out, I get a "server closed connection" message, >> and the whole update process halts. > > That sounds like a bug. :-) If I push, for instance > > (nntp "localhost") > > to gnus-secondary-select-methods and hit g, I get an error message and > then the server is marked as "denied", but Gnus carries on its merry > way. (Same if something times out or I hit `C-g' while it's connecting.) I've got my nntp server set in `gnus-select-method', maybe that's why? This has annoyed me off and on for years but I've never taken the time to look into it. My other servers are all localhost nnimap and I never see errors there. I just set debug-on-message appropriately, and will figure out exactly where things are going off.
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> We create a few custom error symbols for backend errors, including mail >> source failures, open-server failures, etc, and we put some >> condition-cases handling those errors in `gnus-get-unread-articles'. > > This is called from nnmail-get-new-mail-1, though, which ignores errors > on its own. `gnus-get-unread-articles' also has error-ignoring > capabilities and... `nnmail-get-new-mail' is called from within `gnus-request-scan', and there's no error ignoring around `gnus-request-scan' in `gnus-get-unread-articles' -- is there? I can't see anything. I would propose removing error handling from `nnmail-get-new-mail-1': if the fetching of mail sources well and truly fails (and the user hits "no" to give up on the process), it should just signal an error up the line. >> They collect the errors, continue with the next server, and at the end >> of the process display a nice tidy message indicating which servers >> failed to connect. > > ... does just that, I think? At least that's what happens when a server > fails for me. I think nntp servers are louder than the others. When I hit "g" and the nntp connection times out, I get a "server closed connection" message, and the whole update process halts.
Re: Mail source unreachable - continue yes/no?
Lars Ingebrigtsen writes: > Lars-Johan Liman writes: > >> My question: how is my answer (yes/no) used? (Which are the consequences >> of responding yes or no, respectively?) > > Answering "no" should beep an error, say "Cannot get new mail" and then > stop. Hm... but I see that the caller (in `nnmail-get-new-mail-1' just > ignores that, and carries on as if nothing happened), so it doesn't > matter whatever you respond. > > Hm... I'm not sure how to fix this, or what the right fix would be > here. Anybody got an opinion? I have an opinion :) We create a few custom error symbols for backend errors, including mail source failures, open-server failures, etc, and we put some condition-cases handling those errors in `gnus-get-unread-articles'. They collect the errors, continue with the next server, and at the end of the process display a nice tidy message indicating which servers failed to connect. It's always been a bit annoying that an nntp connection failure hoses the entire update process. WDYT? Eric
Re: synchronizing gnus sessions on multiple computers
"Roland Winkler" writes:
> On Fri, Sep 10 2021, Eric Abrahamsen wrote:
>> Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
>> the *Group* buffer), and I would absolutely not try to update files for
>> a Gnus that's still running, it will not handle that gracefully. It
>> should, however, be enough to shut Gnus down first (you can see the
>> `gnus-clear-system' function to see what happens then), rsync, then
>> restart Gnus.
>
> Thanks, it took me a while to get back to this. It seems that
> `gnus-clear-system' / quitting gnus is missing
> `gnus-save-newsrc-file-last-timestamp'. Quitting gnus, then rsync'ing
> gnus followed by a restart of gnus will use .newsrc.eld file from the
> other machine. But when this .newsrc.eld file is saved again, the
> timestamp of the file on disk is compared against the old timestamp of
> the old file that's already gone.
>
> I submitted a bug report (bug#51052).
And it looks like it's already fixed! Thanks a lot for doing that --
there's so much uncertainty and magical thinking around Gnus' behavior,
it's good to get one step closer to reliability.
Re: synchronizing gnus sessions on multiple computers
"Roland Winkler" writes:
> On Fri, Sep 10 2021, Bob Newell wrote:
>> I have rsync'd gnus for years with success. In addition to the files
>> already listed I sync .newsrc.eld, but that is updated infrequently.
>> Not strictly gnus entries, but related closely enough, I also sync
>> .authinfo and .msmtprc (as I use msmtp) and also .bbdb.
>
> Thanks to everyone who replied to this thread.
>
> When you are rsyncing gnus, are you doing this while emacs is running on
> the different machines? Or are you starting a fresh emacs session after
> all the files got updated on a machine?
>
> Digging through things I realized that .newsrc.eld seems to store
> article marks. For example it sets gnus-newsrc-alist that holds an
> assoc list of read articles. However, in a running emacs session the
> same information is stored also in gnus-newsrc-hashtb. So the question
> becomes: What steps are needed to fully restart gnus while emacs is
> running so that internal variables such as gnus-newsrc-hashtb get
> re-initialized, too.
>
> -- I realized that testing all this does not require two different
> machines. But two instances of emacs running on one computer
> effectively give the same situation to play with. It seems that a
> running gnus session does not warn the user that another emacs instance
> has fiddled with files such as .newsrc.eld. Is this true?
Yes, the .newsrc.eld file only gets written when you save Gnus ("s" in
the *Group* buffer), and I would absolutely not try to update files for
a Gnus that's still running, it will not handle that gracefully. It
should, however, be enough to shut Gnus down first (you can see the
`gnus-clear-system' function to see what happens then), rsync, then
restart Gnus.
Re: Gnus stuck in nnimap-wait-for-response with local dovecot
Christoph Groth writes:
> Hello,
>
> I’m still using a mail setup that involves Gnus with nnimap and a local
> dovecot that is run on demand. I sync mail between the local maildir
> (that is used by dovecot when it runs) and remote IMAP servers using
> isync/mbsync.
>
> I learned about this setup from Eric Abrahamsen on the ding mailing list
> in the old days. I believe that the original presentation of the idea
> was this blog post:
>
> https://web.archive.org/web/20160427162529/http://roland.entierement.nu/blog/2010/09/08/gnus-dovecot-offlineimap-search-a-howto.html
>
> On the Gnus side, the setup is based on setting nnimap-shell-program to
>
> "/usr/lib/dovecot/imap -o mail_location=maildir:$HOME/.mail/%s/
> 2>>$HOME/.dovecot.err"
>
> only that this happens inside a setup involving fancy splitting. (This
> way of using dovecot seems to be somewhat obscure, or at least
> I haven’t been able to find much documentation about it.)
>
> Anyway, my setup has been working like this for many years now, but
> there’s a wart. Whenever there is new mail and I launch M-x gnus, it
> will typically freeze during startup and I have to abort with C-g and
> retry. After retrying one or two times Gnus does start and everything
> seems to work normally. The new mail is visible in Gnus.
>
> Here is the kind of the backtrace that I can typically obtain in the
> situation
>
> Debugger entered: ("Quit")
> nnimap-wait-for-response(839)
> nnimap-get-groups()
I can't really guess why this would be happening. The call to
`nnimap-request-scan' should be setting everything up properly, so far
as I know. Perhaps someone else will have some better tips for
debugging. My only other suggestion would be switching to dovecot as an
always-on system service, and then switching Gnus to using "localhost"
as the IMAP server address. It seems entirely plausible that that will
improve things.
Re: cut here markers for code snippets in mail
Andreas Reuleaux writes: > Hi, > > I have seen people highlighting code snippets in mail > with "cut here markers": some ascii art for scissors, "cut here", > "start", "end", ..., like so: > > rm -f /etc/fonts/conf.d/70-no-bitmaps.conf > ln -f -s /usr/share/fontconfig/conf.avail/70-yes-bitmaps.conf > /etc/fonts/conf.d > > > which is kind of nice, and simple (as I don't want to get into > composing html mails with org-mime or the like). > > I get to see them "green on yellow" in gnus (once I have received > such mails), or when in draft status (in my drafts folder). > > Two questions (at least): > > For one thing, I wonder if this behaviour is anywhere > documented/specified/explained in more detail? > > And then: I would like to see my code green-on-yellow snippets in > message mode as well (when composing mail), or at least as a toggle: > see how my mail would look to the recipient. > > And how are people actually using these "cut here markers": > insert them by hand, as I did above? Or is there some automatic support > for this hidden somewhere? - Or any other mechanisms for > such code snippets highlighting - in org mode nevertheless perhaps [?]: > I don't want to send html mails, but maybe org-mime / orgmsg / etc. > are useful for composing mails? The command to run is `message-mark-inserted-region', bound to "C-c M-m" by default in message-mode. It's not terribly discoverable, but it is there in the "Message" menu. I don't believe there's currently any way to get a "preview" in message-mode, of how your outgoing message will look to the recipient.
Re: Highlight process-marked files in Gnus summary?
Roland Winkler writes: > On Tue, May 19 2020, Lars Ingebrigtsen wrote: >> Eric Abrahamsen writes: >> >>>> The obvious goal is to get a better overview of what is marked before I >>>> e.g. delete the articles. It's too easy to make a mistake with only >>>> those little # marks around. Ideally the used face would specify a >>>> background color (like hl-line) so it doesn't interfere with the stuff >>>> in `gnus-summary-highlight'. >>> >>> I haven't ever looked at the summary-highlight code, but would it be >>> hard to add another entry in there, call it `process', to specifically >>> target the process mark? >> >> I don't think that's necessary? Can't you just add highlighting based >> on pretty much anything in the highlighting code? (It's been a while >> since I looked at it...) > > I found this old thread. Could it make sense to have a minor mode that > puts the `invisible' text property on all lines without process-mark? I'm cc'd here but don't know this area of the code at all, sorry. > I have in mind a use case different from Michael who started this thread: > > Often I deal with threads that evolve (magically) such that they become > recognized as multiple threads. In such a case the process-mark can > help to "collect" all threads on a topic. Or are there better solutions > for this? I think `gnus-summary-reparent-thread' or some subset of the code therein might provide a solution. ___ info-gnus-english mailing list info-gnus-english@gnu.org https://lists.gnu.org/mailman/listinfo/info-gnus-english
Re: Does Washing work for IMAP backend?
Pankaj Jangid writes: > Especially, ‘nnmail-prepare-incoming-header-hook’. Does it work for IMAP > backend? It should -- as far as I know all the nnmail splitting/washing stuff happens on a _copy_ of the message buffer, ie you're not changing the actual underlying message, as the IMAP server sees it. ___ info-gnus-english mailing list info-gnus-english@gnu.org https://lists.gnu.org/mailman/listinfo/info-gnus-english
Re: Reply to attached message
Christoph Groth writes: > Eric Abrahamsen wrote: > >> Christoph Groth writes: > >> > I wonder if there is a way in Gnus to reply to a message that has >> > been forwarded as a MIME attachment. >> >> How does the message appear in the Gnus message? What happens when you >> hit "K v" (gnus-article-view-part) on it? > > It appears inline with colored headers, so Gnus does recognize it as > a message. But I cannot find a convenient way to open the attached > message in a separate message buffer so that message commands would work > on it. Huh, so it inlines by default. What's the MIME type on the attachment? Since people can already "K i" to explicitly inline, it seems like it would make sense to have "K v" put the message in a separate buffer in `gnus-article-mode'. > Here is one way that does work, but it is a bit clumsy: > > (1) Save the attachment to a file. > > (2) Create a group from that file (G f). > > (3) Open that group and reply to the message. Yes, that was going to be my original suggestion, but it's not a great solution and I figured you'd already done that! ___ info-gnus-english mailing list info-gnus-english@gnu.org https://lists.gnu.org/mailman/listinfo/info-gnus-english
