Re: Correcting message references
Al Haji-Ali writes: > I changed the message, removed "B" from "References" and deleted the > files of all old (and intermediate) drafts that have "B" in > "References". But no matter what I do, I have "B" grouped with "D" > and any other messages which I create with "In-Reply-To" being "A". How did you find the files to delete? One trap to watch out for is that if using notmuch, you should use notmuch search --exlude=false, to make sure messages are not being hidden because of their tags. > I suspect that somewhere in the database the IDs of "A" and "B" are > linked now. Is there a way (short of deleting the database and > re-indexing) to correct this and remove this connection? The database does not store relationships explicitely, only via messages with references to other messages. At a high level you can try the attached script to get a picture of the corresponding thread. If you can't run the script, or it doesn't help, you can interrogate the database directly without going through notmuch. if the message-id of B is 'f...@example.org' you can search with for replies with xapian-delve (in xapian-tools on Debian and derivatives). xapian-delve -d .local/share/notmuch/default/xapian \ -t 'xreplyto...@example.org' and for references xapian-delve -d .local/share/notmuch/default/xapian \ -t 'xreference...@example.org' That will give you Xapian record numbers, and you can turn those into files with something like xapian-delve -d .local/share/notmuch/default/xapian -r 801793 -1 | \ perl -ne 's/XF(D|O).*?:// && print' For records with multiple files, you will have to figure out with file goes with which directory (or just find the file names, which supposed ot be unique). draw-thread Description: Binary data ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
> Is there any good documentation on neomutt's notmuch support beyond what > is on their website? I currently use alot for its great notmuch support > (in particular tag completion with search and tag inyterfaces, multiple > "tabs"/views) but find myself switching to neomutt for its better mime > support (plus s/mime) and features like thread split & join, so far > mostly maildir backed. neomutt's doc probably tell you everything you > need to know to set up a notmuch-friendly user experience, but ... > I know multiple views will not be a thing in neomutt. Easy tagging and > nm-based search are a must, though. Being able to switch to the "maildir > view" of a message (so that you can rearrange messages within your > maildir folder tree) would be a plus. > > Michael I referred to this page a lot: https://neomutt.org/feature/notmuch There was an 11-month gap between releases (due to lingering bugs) so I suggest trying to get a version released this year (though the 20220429 version was working pretty well for me too). Searching: The vfolder-from-query command lets you type a notmuch search query and shows the results like a folder. To refresh the folder's content (e.g. after updating tags) I find it helpful to type "c" to switch folders, and type "^" as the folder name, which means "the folder I'm already at". For some reason "$" doesn't cause the folder to reload. The "entire-thread" command is useful too: if your search pulls in some emails from a thread and you want to see the reast, simply run the "entire-thread" command (I bind it to "+") with the cursor on one of them, and the rest will be added to the current virtual mailbox. Tagging: The modify-labels command lets you type e.g. "-inbox +spam" to change tags. It has tab completion. I've been meaning to add custom shortcuts for comman changes I make, e.g. -inbox. Maildir view: I don't know exactly what you mean by switching to the maildir view of a message. You can freely switch between real maildirs and notmuch search results when changing folders (see vfolder-from-query). Maybe you want a way to automatically go to the real Maildir folder containing the selected search result; I don't know if that's possible (and there could be ambiguity if files in different folders have the same message id). You might have to run "notmuch new" after making maildir changes? Other / example config: "%g" in the index_format option shows a message's tags. I've copied most of my own config below the signature. -- James ## ## Folder configuration ## set folder=~/var/mbsync/falsifian.org set spool_file="notmuch inbox" set postponed=+Drafts set record=+Sent ## ## Notmuch and mailboxes ## set nm_db_limit=1 set nm_exclude_tags="spam" set nm_record=yes named-mailboxes "notmuch inbox" "notmuch://?query=tag:inbox and not tag:reading" ## ## Aliases ## alias a-b agora-busin...@agoranomic.org # etc...; instead of aliases, I mostly use my address book stored in khard by # pressing ctrl-T after typing the beginning of a name; see the "query_command" # option below. ## ## Key bindings, including macros ## # Make it (hopefully) really hard to accidentally delete messages. unbind * d unbind * "\Cd" bind index S vfolder-from-query # From https://neomutt.org/feature/custom-tags bind index,pager \` modify-labels # From https://neomutt.org/feature/notmuch bind index,pager + entire-thread ## ## Miscellaneous ## set forward_format="Fwd: %s" set from="James Cook " set index_format="%[%g%m%d %H%M] %Z %-15.15L %s%* |%g %?E?%e/%E?" set pager_index_lines=5 set pager_stop set query_command="khard email --parsable %s" set send_charset="utf-8" # Without this, sending mail from a DragonflyBSD machine via my SMTP server to the Agora mail lists fails because the envelope-from ends up being falsifian@ instead of ag...@falsifian.org. set use_envelope_from=yes # I don't know if these are relevant to notmuch; I added these lines when I was # using IMAP with neomutt. # # Caching. header_cache_backend is appended later by drist script. # Use the same directory for both caches. (The manual says this will work.) set header_cache="~/.cache/neomutt" set message_cachedir="~/.cache/neomutt" # Per-recipient settings, and generic default settings. (Without the default # send-hook, the per-recipient settings would stick, since they last until the # end of the session.) send-hook . 'unmy_hdr From: ; set signature="echo James|"' send-hook '~t ^(agora-.*@agoranomic.org\|ag...@listserver.tue.nl)$' 'my_hdr From: Falsifian ; set signature="echo Falsifian|"' # Header weeding and ordering ignore * unignore from date subject to cc list-id reply-to hdr_order subject from date to cc list-id reply-to set header_cache_backend="kyotocabinet" ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
James Cook venit, vidit, dixit 2023-04-19 04:19:25: > On Wed, Apr 19, 2023 at 12:47:24AM +0200, Ralph Seichter wrote: > > Jon Fineman wrote: > > > > > [database] > > > path=/home/jjf/Maildir > > > > > > > and is your Notmuch DB directory /full/path/to/your/Maildir/.notmuch , > > > > i.e. nested inside your Maildir folder? > > > Yes. > > > > Then our respective directory paths are similar. aerc still does not > > work for me, though. I had suspected that the nested directories might > > be causing trouble for me, but that's apparently not the case. > > > > -Ralph > > You could try tracing aerc's systems calls (e.g. ktrace on BSD, strace > on Linux) to see what file it's trying to access. > > Incidentally, my current favourite notmuch client is neomutt. It > probably helps that I'm already used to it. I tried notmuch-emacs and > aerc, and they were okay, though aerc seemed a bit buggy (but that was a > while ago). I tried meli but it wasn't ready yet (maybe it is now?). I > haven't tried notmuch-vim. Is there any good documentation on neomutt's notmuch support beyond what is on their website? I currently use alot for its great notmuch support (in particular tag completion with search and tag inyterfaces, multiple "tabs"/views) but find myself switching to neomutt for its better mime support (plus s/mime) and features like thread split & join, so far mostly maildir backed. neomutt's doc probably tell you everything you need to know to set up a notmuch-friendly user experience, but ... I know multiple views will not be a thing in neomutt. Easy tagging and nm-based search are a must, though. Being able to switch to the "maildir view" of a message (so that you can rearrange messages within your maildir folder tree) would be a plus. Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
On Wed, Apr 19, 2023 at 12:47:24AM +0200, Ralph Seichter wrote: > Jon Fineman wrote: > > > [database] > > path=/home/jjf/Maildir > > > > > and is your Notmuch DB directory /full/path/to/your/Maildir/.notmuch , > > > i.e. nested inside your Maildir folder? > > Yes. > > Then our respective directory paths are similar. aerc still does not > work for me, though. I had suspected that the nested directories might > be causing trouble for me, but that's apparently not the case. > > -Ralph You could try tracing aerc's systems calls (e.g. ktrace on BSD, strace on Linux) to see what file it's trying to access. Incidentally, my current favourite notmuch client is neomutt. It probably helps that I'm already used to it. I tried notmuch-emacs and aerc, and they were okay, though aerc seemed a bit buggy (but that was a while ago). I tried meli but it wasn't ready yet (maybe it is now?). I haven't tried notmuch-vim. -- James ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Ralph Seichter wrote: > * Felipe Contreras: > > > I just installed neovim-ruby and tried to send an email with > > neovim+notmuch-vim: it was sent just fine. > > That's good to hear. Hopefully I am just doing something wrong, which > would be easiest to fix (at least if I knew what that "something" was). > > Today I have compiled Vim 9 from source code; Ruby support included. > I added the vim-plug plugin manager, installed notmuch-vim via > > Plug 'felipec/notmuch-vim' > > et voilà, it works. That includes sending email. Nice. Now I have a > basis to familiarise myself with notmuch-vim. If only I could get it to > work with Neovim, like you did, I'd be all set. Maybe you try to add some debugging around s:compose_send() to see what's going on. > > I'd be happy to consider other alternatives, but at this point which > > are those? > > Anything that is fast and can use libnotmuch, probably? Go comes to > mind. Or even Zig. I like Zig, but I'm not sure it's ready and I can't imagine how to write a vim plugin with it. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Jon Fineman wrote: > [database] > path=/home/jjf/Maildir > > > and is your Notmuch DB directory /full/path/to/your/Maildir/.notmuch , > > i.e. nested inside your Maildir folder? > Yes. Then our respective directory paths are similar. aerc still does not work for me, though. I had suspected that the nested directories might be causing trouble for me, but that's apparently not the case. -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Ralph Seichter writes: > * Jon Fineman: > >> [jjf-notmuch] >> maildir-store = ~/Maildir >> source = notmuch://~/Maildir >> outgoing = /usr/sbin/sendmail >> default = INBOX > > Thanks, Jon. Alas, this does not work for me. Does your .notmuch-config > contain something the following: > > [database] > path=/full/path/to/your/Maildir Yes [database] path=/home/jjf/Maildir > > and is your Notmuch DB directory /full/path/to/your/Maildir/.notmuch , > i.e. nested inside your Maildir folder? Yes. > > -Ralph > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
* Jon Fineman: > [jjf-notmuch] > maildir-store = ~/Maildir > source = notmuch://~/Maildir > outgoing = /usr/sbin/sendmail > default = INBOX Thanks, Jon. Alas, this does not work for me. Does your .notmuch-config contain something the following: [database] path=/full/path/to/your/Maildir and is your Notmuch DB directory /full/path/to/your/Maildir/.notmuch , i.e. nested inside your Maildir folder? -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
* Felipe Contreras: > I just installed neovim-ruby and tried to send an email with > neovim+notmuch-vim: it was sent just fine. That's good to hear. Hopefully I am just doing something wrong, which would be easiest to fix (at least if I knew what that "something" was). Today I have compiled Vim 9 from source code; Ruby support included. I added the vim-plug plugin manager, installed notmuch-vim via Plug 'felipec/notmuch-vim' et voilà, it works. That includes sending email. Nice. Now I have a basis to familiarise myself with notmuch-vim. If only I could get it to work with Neovim, like you did, I'd be all set. > I'd be happy to consider other alternatives, but at this point which > are those? Anything that is fast and can use libnotmuch, probably? Go comes to mind. Or even Zig. > That's aerc, nothing to do with notmuch-vim. > > Are you getting the two confused? Yeah, sorry, that part of my response was related to Jon Fineman's latest message. I was doing too many things at once yesterday. -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Ralph Seichter wrote: > * Felipe Contreras: > > > neovim aims to support most of what vim does. > > At the time of me writing this, the Neovim tracker on GitHub lists a > whopping 1380 open issues, so I'd imagine that goal has not yet been > reached. ;-) I imagine many of those issues are not related to vim compatibility. > > Did you install the "mail" gem as the installation instructions > > recommend? > > I did indeed. The message header was sent during each of my tests, but > not the body text. I just installed neovim-ruby and tried to send an email with neovim+notmuch-vim: it was sent just fine. Note that notmuch-vim sends email using the `sendmail` command. Do you have that properly configured? How would you send email from the command line? > > It sounds better to you as a user. > > > > Just try to program a paginated list of items in Lua and you might > > change your mind. > > I am grateful for you investing time and effort in your plugin, and I > don't want to talk you out of using Ruby, if that is your language of > choice. I have no problem using another language, since anyway I'm planning on rewriting the whole thing. But it has to be the right language, and right now the right language seems to be Ruby. I'd be happy to consider other alternatives, but at this point which are those? > My choice would be different, but that is off topic. I'd be glad to get > notmuch-vim working with Neovim, even if that means installing Ruby. > Currently the log file looks like this: > > INFO 2023/04/18 07:27:20.254544 main.go:180: Starting up version > 0.14.0-153-g6220711f90d7 +notmuch (go1.19.2 amd64 linux) > DEBUG 2023/04/18 07:27:20.254639 crypto.go:36: no internal pgp keyring, using > system gpg > DEBUG 2023/04/18 07:27:20.263046 receive.go:30: Starting Unix server: > /run/user/1000/aerc.sock > DEBUG 2023/04/18 07:27:20.264202 main.go:136: Setting terminal title > INFO 2023/04/18 07:27:20.269082 account.go:250: [Trial] connected. > ERROR 2023/04/18 07:28:07.049184 status.go:129: Cannot perform action. > Messages still loading > DEBUG 2023/04/18 07:28:07.049322 status.go:104: Cannot perform action. > Messages still loading > > aerc displays "(no folders)" and "(no messages)", on startup, so I > thought I'd try the search function with "date:yesterday..", causing the > "Message still loading" error. Running "notmuch search date:yesterday.." > in a shell returns the expected results. That's aerc, nothing to do with notmuch-vim. Are you getting the two confused? > > I would try in vim as well, just so you see how it's supposed to work. > > I guess I should do that. Right now, I don't know if my troubles are > caused by me somehow screwing up the notmuch-vim configuration or by > Neovim quirks. Speaking of my configuration: > > # Content of /home/trial/.config/aerc/accounts.conf > [Trial] > from = Trial User > maildir-store = /home/trial/.mail > source = notmuch:///home/trial > outgoing = /usr/sbin/sendmail Again: that's not notmuch-vim. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
* Felipe Contreras: > neovim aims to support most of what vim does. At the time of me writing this, the Neovim tracker on GitHub lists a whopping 1380 open issues, so I'd imagine that goal has not yet been reached. ;-) > Did you install the "mail" gem as the installation instructions > recommend? I did indeed. The message header was sent during each of my tests, but not the body text. > It sounds better to you as a user. > > Just try to program a paginated list of items in Lua and you might > change your mind. I am grateful for you investing time and effort in your plugin, and I don't want to talk you out of using Ruby, if that is your language of choice. My choice would be different, but that is off topic. I'd be glad to get notmuch-vim working with Neovim, even if that means installing Ruby. Currently the log file looks like this: INFO 2023/04/18 07:27:20.254544 main.go:180: Starting up version 0.14.0-153-g6220711f90d7 +notmuch (go1.19.2 amd64 linux) DEBUG 2023/04/18 07:27:20.254639 crypto.go:36: no internal pgp keyring, using system gpg DEBUG 2023/04/18 07:27:20.263046 receive.go:30: Starting Unix server: /run/user/1000/aerc.sock DEBUG 2023/04/18 07:27:20.264202 main.go:136: Setting terminal title INFO 2023/04/18 07:27:20.269082 account.go:250: [Trial] connected. ERROR 2023/04/18 07:28:07.049184 status.go:129: Cannot perform action. Messages still loading DEBUG 2023/04/18 07:28:07.049322 status.go:104: Cannot perform action. Messages still loading aerc displays "(no folders)" and "(no messages)", on startup, so I thought I'd try the search function with "date:yesterday..", causing the "Message still loading" error. Running "notmuch search date:yesterday.." in a shell returns the expected results. > I would try in vim as well, just so you see how it's supposed to work. I guess I should do that. Right now, I don't know if my troubles are caused by me somehow screwing up the notmuch-vim configuration or by Neovim quirks. Speaking of my configuration: # Content of /home/trial/.config/aerc/accounts.conf [Trial] from = Trial User maildir-store = /home/trial/.mail source = notmuch:///home/trial outgoing = /usr/sbin/sendmail -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Ralph Seichter wrote: > * Felipe Contreras: > > > There hasn't been any development because to be frank it just works. > > I did not mean to imply that "no repository changes" is necessarily a > bad thing. However, Neovim being quite volatile, I intuitively expected > some form of development for notmuch-vim being necessary to keep things > working. I don't see why. neovim aims to support most of what vim does. And I said: I don't use neovim (yet). > > I've also started to rewrite it from scratch because a lot of it I > > think can be done in a much simpler way, but that doesn't really > > affect the users, it's just the maintenance of the code. > > That tells me you are still invested in the subject, which is a welcome > thing to read. > > > I tried notmuch-vim on neovim with ruby support some time ago and it > > worked fine. I think there was some issue with slowness, but they > > fixed it. I don't really remember. > > I set up a virtual Ubuntu 22.04 Server with Notmuch for testing, and > managed to get Ruby 3 support for Neovim [1] working. I then used your > plugin via the "Lazy" package manager for Neovim. Some of the basics > work. Navigating and searching look OK, but I have yet to manage to send > email. So far, message bodies are always missing, so it is not usable > yet. Neovim keeps reporting tons of errors when I try to compose and > send email, and I have not yet delved deeper. Did you install the "mail" gem as the installation instructions recommend? % gem install mail It should actually be a requirement, but I haven't updated the instructions to say that. > > One alternative that I considered is to rewrite notmuch-vim in lua, > > but that would probably require some libnotmuch bindings for lua, > > which would take considerable amount of time to write. And that would > > only work for neovim, not vim. > > From what I understand, Vim 9 introduces a new, proprietary scripting > language, while Neovim uses LUA? That seems to foreshadow lots of work > if one wishes to provide plugins that work with both editors. But you don't have to use Vim 9. The old language will keep working. > > It might be better to just keep using ruby. It's a fantastic language, > > there's already pretty good bindings for libnotmuch, and it works for > > both vim and neovim. > > Ruby may be a worthwhile language, but in this particular scenario of > Neovim and Notmuch, I see the end user's perspective: I don't need Ruby > to run anything but notmuch-vim, so I don't want to install Ruby on > various systems if it can be avoided. Ruby is already installed on most systems I use, and it's only like 10 MiB. I believe one should use the best language for the task, and in this case that language is clearly Ruby. > I'll do it if there is no alternative, but LUA sure sounds better than Ruby > to me. Selfish, I know. ;-) It sounds better to you as a user. Just try to program a paginated list of items in Lua and you might change your mind. > > I think you should just give it a try. > > I began my experiments, but as stated above, I ran into issues that I > have yet to figure out. > > Thank you for your response and for your plugin, Felipe. I would try in vim as well, just so you see how it's supposed to work. To be frank notmuch-vim is so superior to anything I've used, that even if I move completey to neovim and it doesn't work there, I would keep using vim just for this tool. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Not OP. Version on Debian: aerc 0.14.0 +notmuch (go1.19.8 amd64 linux) accounts.conf: [jjf-notmuch] maildir-store = ~/Maildir source = notmuch://~/Maildir outgoing = /usr/sbin/sendmail default = INBOX Jon Ralph Seichter writes: > * in...@sindominio.net: > >> A different kind of alternative is to use aerc with nvim as an editor. > > I did not really have a separate mail client in mind, but I experimented > with aerc today. I built aerc from its source code, with Notmuch support > enabled. Alas, I am stuck at launch: > > - 8< - snip - 8< - > > INFO 2023/04/18 00:09:26.887462 main.go:180: Starting up version > 0.14.0-153-g6220711f90d7 +notmuch (go1.19.2 amd64 linux) > INFO 2023/04/18 00:09:26.897506 main.go:132: Terminal does not have status > line support > ERROR 2023/04/18 00:09:26.897669 worker.go:83: ProcessAction(*types.Connect) > failure: could not connect to notmuch db: Something went wrong trying to read > or write a file > ERROR 2023/04/18 00:09:26.899454 account.go:359: [aerc Trial] unexpected > error: could not connect to notmuch db: Something went wrong trying to read > or write a file > ERROR 2023/04/18 00:09:26.899498 status.go:129: aerc Trial: could not connect > to notmuch db: Something went wrong trying to read or write a file > > - 8< - snip - 8< - > > Not very specific, as error messages go, and searching the Internet did > not lead me to enlightenment. Would you perhaps be willing to share the > contents of your ~/.config/aerc folder (anonymised, of course)? > > -Ralph > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
* in...@sindominio.net: > A different kind of alternative is to use aerc with nvim as an editor. I did not really have a separate mail client in mind, but I experimented with aerc today. I built aerc from its source code, with Notmuch support enabled. Alas, I am stuck at launch: - 8< - snip - 8< - INFO 2023/04/18 00:09:26.887462 main.go:180: Starting up version 0.14.0-153-g6220711f90d7 +notmuch (go1.19.2 amd64 linux) INFO 2023/04/18 00:09:26.897506 main.go:132: Terminal does not have status line support ERROR 2023/04/18 00:09:26.897669 worker.go:83: ProcessAction(*types.Connect) failure: could not connect to notmuch db: Something went wrong trying to read or write a file ERROR 2023/04/18 00:09:26.899454 account.go:359: [aerc Trial] unexpected error: could not connect to notmuch db: Something went wrong trying to read or write a file ERROR 2023/04/18 00:09:26.899498 status.go:129: aerc Trial: could not connect to notmuch db: Something went wrong trying to read or write a file - 8< - snip - 8< - Not very specific, as error messages go, and searching the Internet did not lead me to enlightenment. Would you perhaps be willing to share the contents of your ~/.config/aerc folder (anonymised, of course)? -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
* Felipe Contreras: > There hasn't been any development because to be frank it just works. I did not mean to imply that "no repository changes" is necessarily a bad thing. However, Neovim being quite volatile, I intuitively expected some form of development for notmuch-vim being necessary to keep things working. > I've also started to rewrite it from scratch because a lot of it I > think can be done in a much simpler way, but that doesn't really > affect the users, it's just the maintenance of the code. That tells me you are still invested in the subject, which is a welcome thing to read. > I tried notmuch-vim on neovim with ruby support some time ago and it > worked fine. I think there was some issue with slowness, but they > fixed it. I don't really remember. I set up a virtual Ubuntu 22.04 Server with Notmuch for testing, and managed to get Ruby 3 support for Neovim [1] working. I then used your plugin via the "Lazy" package manager for Neovim. Some of the basics work. Navigating and searching look OK, but I have yet to manage to send email. So far, message bodies are always missing, so it is not usable yet. Neovim keeps reporting tons of errors when I try to compose and send email, and I have not yet delved deeper. [1] https://github.com/neovim/neovim-ruby > One alternative that I considered is to rewrite notmuch-vim in lua, > but that would probably require some libnotmuch bindings for lua, > which would take considerable amount of time to write. And that would > only work for neovim, not vim. >From what I understand, Vim 9 introduces a new, proprietary scripting language, while Neovim uses LUA? That seems to foreshadow lots of work if one wishes to provide plugins that work with both editors. > It might be better to just keep using ruby. It's a fantastic language, > there's already pretty good bindings for libnotmuch, and it works for > both vim and neovim. Ruby may be a worthwhile language, but in this particular scenario of Neovim and Notmuch, I see the end user's perspective: I don't need Ruby to run anything but notmuch-vim, so I don't want to install Ruby on various systems if it can be avoided. I'll do it if there is no alternative, but LUA sure sounds better than Ruby to me. Selfish, I know. ;-) > I think you should just give it a try. I began my experiments, but as stated above, I ran into issues that I have yet to figure out. Thank you for your response and for your plugin, Felipe. -Ralph ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
A different kind of alternative is to use aerc with nvim as an editor. Although the notmuch interface still has some quirks, I use it daily (and I am quite happy with it). El 17/17/2023 a las 2:41, Felipe Contreras dejó escrito: > Ralph Seichter wrote: > > Based on the Notmuch documentation [1] and Internet search results, > > Felipe Contreras' notmuch-vim plugin [2] used to be a popular method of > > combining Notmuch with Vim (at least classic Vim). However, there have > > not been any Git repository updates for this plugin for two years, while > > Neovim development has been very active during that time. > > There hasn't been any development because to be frank it just works. > > It does everything I need, there's only one patch that I've added that I'm > still not sure how to merge. > > I've also started to rewrite it from scratch because a lot of it I think can > be > done in a much simpler way, but that doesn't really affect the users, it's > just > the maintenance of the code. > > > Also, this particular plugin requires Ruby support in Vim, which I'd rather > > avoid, if possible. > > > > [1] https://git.notmuchmail.org/git/notmuch/blob/HEAD:/vim/README > > [2] https://github.com/felipec/notmuch-vim > > > > My questions are: > > > > 1. Do any of you fellow mailing list members use notmuch-vim with > >Neovim? > > I still use vim, but I've started to move to neovim (although it still has > many > issues for me). > > I tried notmuch-vim on neovim with ruby support some time ago and it worked > fine. I think there was some issue with slowness, but they fixed it. I don't > really remember. > > > 2. Can you recommend an alternative Notmuch/Neovim plugin? > > > > I am currently using Notmuch with Emacs, but I'd like to find a way to > > integrate Notmuch with Neovim instead. > > > > Your feedback is appreciated. > > One alternative that I considered is to rewrite notmuch-vim in lua, but that > would probably require some libnotmuch bindings for lua, which would take > considerable amount of time to write. And that would only work for neovim, not > vim. > > It might be better to just keep using ruby. It's a fantastic language, there's > already pretty good bindings for libnotmuch, and it works for both vim and > neovim. > > I think you should just give it a try. > > Cheers. > > -- > Felipe Contreras > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Recommended method to pair Notmuch with Neovim?
Ralph Seichter wrote: > Based on the Notmuch documentation [1] and Internet search results, > Felipe Contreras' notmuch-vim plugin [2] used to be a popular method of > combining Notmuch with Vim (at least classic Vim). However, there have > not been any Git repository updates for this plugin for two years, while > Neovim development has been very active during that time. There hasn't been any development because to be frank it just works. It does everything I need, there's only one patch that I've added that I'm still not sure how to merge. I've also started to rewrite it from scratch because a lot of it I think can be done in a much simpler way, but that doesn't really affect the users, it's just the maintenance of the code. > Also, this particular plugin requires Ruby support in Vim, which I'd rather > avoid, if possible. > > [1] https://git.notmuchmail.org/git/notmuch/blob/HEAD:/vim/README > [2] https://github.com/felipec/notmuch-vim > > My questions are: > > 1. Do any of you fellow mailing list members use notmuch-vim with >Neovim? I still use vim, but I've started to move to neovim (although it still has many issues for me). I tried notmuch-vim on neovim with ruby support some time ago and it worked fine. I think there was some issue with slowness, but they fixed it. I don't really remember. > 2. Can you recommend an alternative Notmuch/Neovim plugin? > > I am currently using Notmuch with Emacs, but I'd like to find a way to > integrate Notmuch with Neovim instead. > > Your feedback is appreciated. One alternative that I considered is to rewrite notmuch-vim in lua, but that would probably require some libnotmuch bindings for lua, which would take considerable amount of time to write. And that would only work for neovim, not vim. It might be better to just keep using ruby. It's a fantastic language, there's already pretty good bindings for libnotmuch, and it works for both vim and neovim. I think you should just give it a try. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: inbox-update: new competition of notmuch-lore
Michael J Gruber wrote: > > I'm moving from mbsync to public-inbox and I find there aren't many tools to > > make it work with notmuch. > > Looking at that, too. > > > I gave a try to notmuch-lore [1] but I found it too slow and had a couple of > > issues. > > > > So I wrote my own script to convert public-inbox mailing lists to Maildir > > format: notmuch-tools/inbox-update [2]. > > > > It's much faster at the initial clone, it deals with deleted mails, and > > YAML is > > a much better configuration format. > > Looking at both scripts: Is the speed-up mainly due to `git cat-file` > vs. `git show`? My guess is that it's due to using `git cat-file` in batch mode, so it's called only once, instead of thousands of times. Presumably this can be done in notmuch-lore as well, with something like: git rev-list | sed -e /$/:m/ | git cat-file --batch But this still has the issue that some commits remove mail, don't add. > > Also, you can configure which epochs you want to fetch (notmuch-lore fetches > > all of them). > > > > One thing it doesn't yet do is trim the repository once the mails have been > > converted, but that's probably easy to add later on. > > What kind of trimming are you thinking about here? Partial history? Same as notmuch-lore does: just the last commit. Once the mails have been extracted there's no need for those commits. > I guess this shows that public-inbox's repo format is simply not the > best choice for the purpose of mail readers. It is optimised for other > uses, and I always wondered why they use a non-bare repo at all. That > single file path m at the root creates absolutely meaningless diffs. > And the commit message doubles the info which is present in the blob. > notes-ref could have served better for inspiration of public-inbox. > (Barking up the wrong tree, I know.) I don't know if there's a better format, git stores shapshots anyway, so as long as the information is retrivable in some way, I think that' fine. And I clone the public-inbox repositories as bare (mirror, actually), that's something for the client to decide. > There are even tools in the public-inbox eco system which feed that > info into a xapian db, though not notmuch-like, as if notmuch hadn't > existed already. > > What I'm dreaming of is a notmuch "storage backend" which is git > object db based rather than maildir based, and compatible with > public-inbox (at least with the use case, i.e. v3 or v4...). I mean - > why do we need a checkout of basically immutable files which are > stored in blobs already, just so that notmuch can index them? Yeap, that's exactly what I want as well. It should not be that difficult to decouple notmuch from physical files and feed some virtual content. > We need them for the MUAs, I know, and we would need a solution for > them, too. Or simply a tree in public-inbox which allows clients to > use a mere checkout ... 99% of the time the content is not needed for the MUAs. So perhaps there could be a way to request the body of the message through libnotmuch, and some provider of virtual messages retrives it on demand. Maildir seems like a cumbersome intermediary to me, at the moment. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: inbox-update: new competition of notmuch-lore
Hi Felipe > I'm moving from mbsync to public-inbox and I find there aren't many tools to > make it work with notmuch. Looking at that, too. > I gave a try to notmuch-lore [1] but I found it too slow and had a couple of > issues. > > So I wrote my own script to convert public-inbox mailing lists to Maildir > format: notmuch-tools/inbox-update [2]. > > It's much faster at the initial clone, it deals with deleted mails, and YAML > is > a much better configuration format. Looking at both scripts: Is the speed-up mainly due to `git cat-file` vs. `git show`? > Also, you can configure which epochs you want to fetch (notmuch-lore fetches > all of them). > > One thing it doesn't yet do is trim the repository once the mails have been > converted, but that's probably easy to add later on. What kind of trimming are you thinking about here? Partial history? I guess this shows that public-inbox's repo format is simply not the best choice for the purpose of mail readers. It is optimised for other uses, and I always wondered why they use a non-bare repo at all. That single file path m at the root creates absolutely meaningless diffs. And the commit message doubles the info which is present in the blob. notes-ref could have served better for inspiration of public-inbox. (Barking up the wrong tree, I know.) There are even tools in the public-inbox eco system which feed that info into a xapian db, though not notmuch-like, as if notmuch hadn't existed already. What I'm dreaming of is a notmuch "storage backend" which is git object db based rather than maildir based, and compatible with public-inbox (at least with the use case, i.e. v3 or v4...). I mean - why do we need a checkout of basically immutable files which are stored in blobs already, just so that notmuch can index them? We need them for the MUAs, I know, and we would need a solution for them, too. Or simply a tree in public-inbox which allows clients to use a mere checkout ... Cheers, Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: API docs: notmuch_database_open_with_config with NULL config
James Cook writes: > It is possible that a short note indicating that detailed descriptions > will follow, placed at the top of the man page and/or at the top of the > "Functions" section, would have saved me from my impatience. I think I > do tend to read the first sentence or two of a man page before just > searching for what I'm looking for. But absent such a note, maybe I can > be excused for paging through a few screenfuls of API summary and > assuming that's all there was. I'm a bit reluctant to invest much effort in the current Doxygen based API docs (I'm hoping we eventually replace them with something more compatible with the rest of the sphinx-doc based documentation). On the other hand if someone else out there is a better Doxygen wrangler than I and the fix is relatively small, I don't see why we wouldn't apply that change. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: API docs: notmuch_database_open_with_config with NULL config
On Sat, Apr 15, 2023 at 12:26:33PM -0300, David Bremner wrote: > James Cook writes: > > > Hi list, > > > > If I'm not mistaken, calling notmuch_database_open_with_config with > > config_path = NULL causes notmuch to try to find the config file > > automatically. > > > > It would be nice if that were documented in the notmuch(3) man page. I > > can try drafting a patch if that would help. > > > > (This message brought to you by trying to understand neomutt's notmuch > > code...) > > Here is what i see in the notmuch(3) man page: > > config_path Path to config file. > >Config file is key-value, with mandatory sections. See > notmuch-config(5) for more >information. The key-value pair overrides the corresponding > configuration data stored in the >database (see notmuch_database_get_config) > >If config_path is NULL use the path specified > >• in environment variable NOTMUCH_CONFIG, if non-empty > >• by XDG_CONFIG_HOME/notmuch/ where XDG_CONFIG_HOME defaults to > '$HOME/.config'. > >• by $HOME/.notmuch-config > >If config_path is '' (empty string) then do not open any > configuration file. >profile Name of profile (configuration/database variant). > >If non-NULL, append to the directory / file path determined for > config_path and >database_path. > >If NULL then use > >• environment variable NOTMUCH_PROFILE if defined, > >• otherwise 'default' for directories and '' (empty string) for > paths. > > I'm not claiming the documentation is perfect, but it seems to be > documented? Is there maybe some version skew between your man pages and > library? > > d Oops, I have that too. I didn't realize there was further documentation after the summary in the "Functions" section. This is much better! Thanks. It is possible that a short note indicating that detailed descriptions will follow, placed at the top of the man page and/or at the top of the "Functions" section, would have saved me from my impatience. I think I do tend to read the first sentence or two of a man page before just searching for what I'm looking for. But absent such a note, maybe I can be excused for paging through a few screenfuls of API summary and assuming that's all there was. -- James ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: API docs: notmuch_database_open_with_config with NULL config
James Cook writes: > Hi list, > > If I'm not mistaken, calling notmuch_database_open_with_config with > config_path = NULL causes notmuch to try to find the config file > automatically. > > It would be nice if that were documented in the notmuch(3) man page. I > can try drafting a patch if that would help. > > (This message brought to you by trying to understand neomutt's notmuch > code...) Here is what i see in the notmuch(3) man page: config_path Path to config file. Config file is key-value, with mandatory sections. See notmuch-config(5) for more information. The key-value pair overrides the corresponding configuration data stored in the database (see notmuch_database_get_config) If config_path is NULL use the path specified • in environment variable NOTMUCH_CONFIG, if non-empty • by XDG_CONFIG_HOME/notmuch/ where XDG_CONFIG_HOME defaults to '$HOME/.config'. • by $HOME/.notmuch-config If config_path is '' (empty string) then do not open any configuration file. profile Name of profile (configuration/database variant). If non-NULL, append to the directory / file path determined for config_path and database_path. If NULL then use • environment variable NOTMUCH_PROFILE if defined, • otherwise 'default' for directories and '' (empty string) for paths. I'm not claiming the documentation is perfect, but it seems to be documented? Is there maybe some version skew between your man pages and library? d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v3 1/5] test: use bash specific test for feature tests
On Thu, Apr 13, 2023 at 2:25 PM Tomi Ollila wrote:
> On Sun, Apr 09 2023, David Bremner wrote:
> I would just use [ "${VAR-}" = 1 ] in shell script code (or barely
> [[ ${VAR-0} == 1 ]] if desired not to have ""s around variables...but...)
>
> ... using [ "${VAR-}" = 1 ] is also good in a sense that someone may
> use it as an example to write shell scripts with #!/bin/sh as a
> hashbang -- and it works fine on Fedora since there /bin/sh is bash,
> but would fail on Debian since there /bin/sh is dash.
Yeah, the world of shells is a universe, but long story short: you can
just use the above and it should work on all shells and all
configurations.
You can just do "$VAR", *but* it wouldn't work with `set -u`. I never
use -u, so I don't care, but if you do care, do "${VAR-}".
Cheers.
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v3 1/5] test: use bash specific test for feature tests
On Sun, Apr 09 2023, David Bremner wrote:
> It is desirable to have the tests consider these variables being
> undefined as equivalent to the feature not being present, and in
> particular for the tests not to generate errors.
>
> We know the test suite is tied to bash anyway, so this is a simple
> fix.
> ---
> test/T060-count.sh | 2 +-
> test/T081-sexpr-search.sh| 2 +-
> test/T150-tagging.sh | 2 +-
> test/T160-json.sh| 2 +-
> test/T220-reply.sh | 2 +-
> test/T240-dump-restore.sh| 2 +-
> test/T391-python-cffi.sh | 3 +--
> test/T392-python-cffi-notmuch.sh | 2 +-
> test/T520-show.sh| 2 +-
> test/T570-revision-tracking.sh | 2 +-
> test/T700-reindex.sh | 2 +-
> test/T800-asan.sh| 2 +-
> test/T810-tsan.sh| 2 +-
> test/T850-git.sh | 2 +-
> 14 files changed, 14 insertions(+), 15 deletions(-)
>
> diff --git a/test/T060-count.sh b/test/T060-count.sh
> index 48146706..4499b4cb 100755
> --- a/test/T060-count.sh
> +++ b/test/T060-count.sh
> @@ -157,7 +157,7 @@ print("4: {} messages".format(query.count_messages()))
> EOF
> test_expect_equal_file EXPECTED OUTPUT
>
> -if [ $NOTMUCH_HAVE_SFSEXP -eq 1 ]; then
> +if [[ $NOTMUCH_HAVE_SFSEXP -eq 1 ]]; then
Hmm, I send reply to Felipe's email, before seeing this, so this is
followup ... which, at the end, reaches the same "conclusion" :D
anyway, while this is good progress the [[ $v -eq 1 ]] is still somewhat
fragile (using -eu shows some interesting behaviors):
the easy try:
$ bash -eu -c ' [[ $AAA -ne 1 ]] && echo 1 || echo 0'
bash: line 1: AAA: unbound variable
weirder...:
$ bash -eu -c 'a="x"; [[ $a -eq 2 ]]'
bash: line 1: x: unbound variable
wat, as $a is not numeric, it tries to re-eval $x ???
$ bash -eu -c 'x=3; a="x"; [[ $a -eq 2 ]] && echo 1 || echo 0'
0
$ bash -eu -c 'x=2; a="x"; [[ $a -eq 2 ]] && echo 1 || echo 0'
1
for reference
$ bash -eu -c 'a="2"; [[ $a -eq 2 ]] && echo 1 || echo 0'
1
and again, w/ = instead of -eq
$ bash -eu -c 'a="x"; [[ $a = 2 ]] && echo 1 || echo 0'
0
and -- modified the first AAA case above:
$ bash -eu -c ' AAA=b; [[ ${AAA-} = 1 ]] && echo 1 || echo 0'
0
hmm, when using [[ ]] I recall == and != were to used... uh, see
bash manual about the pattern matching and such... tried also
with (( ... )) (documenter before [[ ]] in bash manual page
but now it fails even heavier..:
$ bash -eu -c 'XXX="a"; (( ${XXX-} == 2 )) && echo 1 || echo 0'
bash: line 1: a: unbound variable
$ bash -eu -c 'a=2; XXX="a"; (( ${XXX-} == 2 )) && echo 1 || echo 0'
1
I would just use [ "${VAR-}" = 1 ] in shell script code (or barely
[[ ${VAR-0} == 1 ]] if desired not to have ""s around variables...but...)
... using [ "${VAR-}" = 1 ] is also good in a sense that someone may
use it as an example to write shell scripts with #!/bin/sh as a
hashbang -- and it works fine on Fedora since there /bin/sh is bash,
but would fail on Debian since there /bin/sh is dash.
HTH ;D
Tomi
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v3 1/5] test: use bash specific test for feature tests
On Thu, Apr 13 2023, Felipe Contreras wrote:
> On Sun, Apr 9, 2023 at 9:26 AM David Bremner wrote:
>>
>> It is desirable to have the tests consider these variables being
>> undefined as equivalent to the feature not being present, and in
>> particular for the tests not to generate errors.
>
> This can be done easily with `[ ${FOO-0} -eq 1 ]`, or as I prefer: `[
> "$FOO" = 1 ]`.
[ "${FOO-0}" = 1 ]
if $FOO is to have $IFS characters...
Tomi
>
> --
> Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] doc: fix typos
Jakub Wilk writes: > --- > doc/man1/notmuch-config.rst | 2 +- > doc/man1/notmuch-git.rst | 4 ++-- > doc/man7/notmuch-sexp-queries.rst | 8 > doc/notmuch-emacs.rst | 6 +++--- > 4 files changed, 10 insertions(+), 10 deletions(-) applied to master, thanks d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v3 1/5] test: use bash specific test for feature tests
On Sun, Apr 9, 2023 at 9:26 AM David Bremner wrote:
>
> It is desirable to have the tests consider these variables being
> undefined as equivalent to the feature not being present, and in
> particular for the tests not to generate errors.
This can be done easily with `[ ${FOO-0} -eq 1 ]`, or as I prefer: `[
"$FOO" = 1 ]`.
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 3/3] ruby: remove FileNames object
Felipe Contreras writes: > Not used anymore now that we return an array of strings directly. series applied to master, with one whitespace tweak in patch 2/3 ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] notmuch-mutt: replace extra command with notmuch-native thread search feature
Paul Wise writes: > that changes the meaning of the query but I couldn't find any way to > prevent an arbitrary Message-Id from inserting parentheses into the > query. Also notmuch converts Message-Id (test)@hostname to just > @hostname in the Xapian database. I think that right now I am not sure > that re-implementing all the idiosyncrasies of notmuch queries and > Message-Id munging within notmuch-mutt is the way to go, especially > since Message-Id fields that intersect with notmuch features are rare. The code that does removal of spaces and () delited sequences goes back to Carl's 2009 implementation of manual header parsing (which we have mostly, but not entirely replaced with calls to gmime). It would probably be reasonable to disable that code (or maybe migrate to gmime parsing of message-ids?), but the cost/benefit analysis is not too clear to me. I had to look this up, but apparently parens mark a comment in RFC822 / RFC5322. If I understand RFC5322 correctly then comments are only permitted in the obsolete syntax, but I guess it makes sense to accept those. I don't know how many messages in the wild use this syntax; of the .01% in in my mail store, most don't put the comment inside the actual <> delimited message-id id, and those that do look like errors of various kinds - some unbalanced parens - some (null) indicating printf-ing null pointers ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] notmuch-mutt: replace extra command with notmuch-native thread search feature
On Sun, 2023-04-09 at 13:00 -0300, David Bremner wrote: > You might be interested in the s-expression query parser. Part of the > goal is to be less dwim/quirky than the native Xapian query parser, and > to make it easier / safer to construct notmuch queries via programs. That does seem saner to manually construct queries in, but it would still be better to have a query construction library, otherwise programs are going to get it wrong all the time, even within notmuch. > It doesn't do the message-id munging you mention (yet?), This is a big part of the query construction headache, re-implementing the space stripping is easy, the user part removal is harder and I can't find any docs on what else is required here. PS: I don't intend to work on switching notmuch-mutt to sexp. -- bye, pabs https://bonedaddy.net/pabs3/ signature.asc Description: This is a digitally signed message part ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] notmuch-mutt: replace extra command with notmuch-native thread search feature
Paul Wise writes: > So I think that it would be good if there were a library for notmuch > query construction that would take care of all the different quoting > levels, Message-Id munging etc. This could then get language bindings > so that notmuch based tools and MUAs could use them for interactive > graphical query construction. Also maybe Xapian needs or has a way to > construct queries without serialising them to a text string and the > notmuch query construction library I propose could base itself on that. You might be interested in the s-expression query parser. Part of the goal is to be less dwim/quirky than the native Xapian query parser, and to make it easier / safer to construct notmuch queries via programs. It doesn't do the message-id munging you mention (yet?), but quoting is IMHO saner. It does require an extra dependency (sfsexpr). ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] notmuch-mutt: replace extra command with notmuch-native thread search feature
On Sat, 2023-04-08 at 08:42 -0300, David Bremner wrote: > did you test this with a message-id with spaces in it? > the quoting is a delicate. You are correct that the quoting for this was wrong, as it was also wrong in the code before my patch and I just copied that, however... I noticed that when notmuch stores the Message-ID in the Xapian database, it strips the spaces, so messages with spaces in the Message-ID are not found unless the id: search has no spaces also. After switching from testing using id: to subject: and reading the manual page I was able to find the correct quoting for spaces in $mid, but I also added the space removals for the thread query. Then I figured I should test with double quote characters and that blew up in my face too, so I added double quote doubling to workaround that. Then I noticed that you can put parentheses inside a quoted query and that changes the meaning of the query but I couldn't find any way to prevent an arbitrary Message-Id from inserting parentheses into the query. Also notmuch converts Message-Id (test)@hostname to just @hostname in the Xapian database. I think that right now I am not sure that re-implementing all the idiosyncrasies of notmuch queries and Message-Id munging within notmuch-mutt is the way to go, especially since Message-Id fields that intersect with notmuch features are rare. So I think that it would be good if there were a library for notmuch query construction that would take care of all the different quoting levels, Message-Id munging etc. This could then get language bindings so that notmuch based tools and MUAs could use them for interactive graphical query construction. Also maybe Xapian needs or has a way to construct queries without serialising them to a text string and the notmuch query construction library I propose could base itself on that. Anyway, will send an updated patch series including space/quote fixing. -- bye, pabs https://bonedaddy.net/pabs3/ signature.asc Description: This is a digitally signed message part ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Carl Worth writes: > > What it could also do is create some placeholder file like > .NOTMUCH-MUTT-OUTPUT-DIR when it writes its output. And then it could > check for that file's existence before removing a directory. > Sounds reasonable to me. Would someone like to make a patch for notmuch-mutt? d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] notmuch-mutt: replace extra command with notmuch-native thread search feature
Paul Wise writes:
> +search($results_dir, $remove_dups, qq{thread:"{id:$mid}"});
did you test this with a message-id with spaces in it? the quoting is a
delicate.
d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes: > On Tue, Apr 4, 2023 at 12:54 PM David Bremner wrote: >> >> This sounds right. Can we use the detection of missing messages in >> wr_export to reset the appropriate counters? It looks like yes, given >> the call to store_lastmod. [snip] > I would rather go for a solution that is less hacky, and has less > chance of leaving the user in an unrecoverable state. fair enough. Certainly notmuch-git has too many accumulated performance hacks. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Tue, Apr 4, 2023 at 12:54 PM David Bremner wrote: > > Felipe Contreras writes: > > > On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote: > > > Or we could say that after jumping a certain threshold of lastmod we > > delete all the messages and start from scratch, perhaps every 1000 > > revisions. > > > > Or maybe the query could generate a virtual tag if a message was > > deleted since the previous lastmod (e.g. "nm::deleted"). Then it would > > be trivial for the remote helper to tell that to git. > > A complication here is that tags be attached to mail message documents > in the database, so we would need to generate a so called "ghost > message", and clean those up somehow. I thought a little bit more about how I would use git-notmuch, and I don't see the point in tracking messages that have no tags. In my view the whole point of the tool is to backup the tags, and the whole point of a backup is to eventually be able to restore it. But if there's nothing to restore for a specific message, it might very well not exist. So instead of a `nm::deleted` tag, just no tags. I think from the point of view of git-notmuch it shouldn't make a difference. > > I lean towards the threshold, because that way the user doesn't need > > to do anything, and there's no modifications needed in libnotmuch. > > This sounds right. Can we use the detection of missing messages in > wr_export to reset the appropriate counters? It looks like yes, given > the call to store_lastmod. We would need to store them and use that information in the next fetch. Although doable, it seems hacky, and in the past such things have led to problems that are hard to solve due to inconsistent states. For example what happens if in the next fetch we tell git that some files have been removed, but we crash in the middle of it? The next fetch we'll tell git that some files were removed, but git might think they don't exist and fail. I think for that particular problem git was fixed it shouldn't update the files unless the program exists successfully, but I don't know. I would rather go for a solution that is less hacky, and has less chance of leaving the user in an unrecoverable state. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: notmuch-emacs inline image display broken in emacs-29
Hello Alex and David, I also came across this issue: inline image are not displayed only when `notmuch-show-indent-content` is non-nil. However, even when `notmuch-show-indent-content` is `nil` they are displayed but cannot be hidden. As Alex Murray pointed out, the reason is that the function `mm-insert-image` is being changed in Emacs 29 to use `insert-image` instead of `put-image`. `insert-image` inserts images using the `display` property of a whitespace character. The effect is that when `indent-rigidly` is called in `notmuch-show-insert-msg`, whitespace is removed and hence the image is removed. I've filed a bug#62637 on bug-gnu-emacs to discuss this. An easy fix in `mm-insert-image` is use non-whitespace character (the second argument of `insert-image`) so that it is not deleted by `indent-rigidly`. This change actually makes indenting images work in the new version. The other issue is that hiding an overlay created around such an image does not seem to work (see bug#62637 for a discussion). A way to resolve this is to add some whitespace (like a newline) inside the overlay before the image, which seems to fix things. -- Al ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes: > On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote: > Or we could say that after jumping a certain threshold of lastmod we > delete all the messages and start from scratch, perhaps every 1000 > revisions. > > Or maybe the query could generate a virtual tag if a message was > deleted since the previous lastmod (e.g. "nm::deleted"). Then it would > be trivial for the remote helper to tell that to git. A complication here is that tags be attached to mail message documents in the database, so we would need to generate a so called "ghost message", and clean those up somehow. > I lean towards the threshold, because that way the user doesn't need > to do anything, and there's no modifications needed in libnotmuch. This sounds right. Can we use the detection of missing messages in wr_export to reset the appropriate counters? It looks like yes, given the call to store_lastmod. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes: > > I'm not familiar with git-annex, I would need to see an example of > such merging happening. I was confused, git-annex is using the builtin merge strategy "union", which is not eliminating duplicates or sorting, so probably not applicable here. I still have to try some merges between different machines to see what kind of conflicts can arise. > One advantage of using the fast-import format is that it's easy to > change it, or support multiple formats. > > In fact, the format could be specified in the URL, like > `nm::1:$HOME/mail` for the current notmuch-git format, and > `nm::2:$HOME/mail` for the new. This might also be a way to handle the "prefix" setting that nmbug / notmuch-git needs to only sync certain (e.g. notmuch::*) tags ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Selection bug
On Tue, Apr 04 2023, Justus Winter wrote: > My desktop now behaves exactly like my laptop, and it is really > annoying. Surely I'm not the only one with this problem. Can no one > else reproduce this? I haven't tried using the reproduction test, but I did just refresh my inbox and I saw the buggy behavior. So there's a random anecdote which suggests it might not be that hard to reproduce the bug, (perhaps doesn't require any special messages). In my case, it's less annoying because I have more than a screenful of messages in my inbox (for shame!) so the message with the green bar isn't even visible when emacs' point is moved to the message that is actually selected. So if I hadn't been intentionally looking for this behavior I likely would have never noticed. -Carl ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
On Tue, Apr 04 2023, Michael J. Gruber wrote: >> It's really unfortunate if notmuch-mutt makes it that easy to throw away >> your email. >> >> That sounds like a nasty bug that should be fixed in that program. > > To be fair to notmuch-mutt: > ``` > --output-dir DIR > > Store search results as (symlink) messages under maildir DIR. Beware: DIR will > be overwritten. (Default: F<~/.cache/notmuch/mutt/results/>) > ``` > So the default is safe, and the warning is there. It would be a bug > only if notmuch-mutt descended up and deleted a root dir or such. OK. It's definitely acting as documented. So maybe "bug" was a little harsh in my description. It could still be made safer, and I think it would be a good idea to change it. For example, notmuch-mutt currently happily creates a directory that doesn't already exists. That's safe and good. What it could also do is create some placeholder file like .NOTMUCH-MUTT-OUTPUT-DIR when it writes its output. And then it could check for that file's existence before removing a directory. -Carl ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Selection bug
Justus Winter writes:
> David Bremner writes:
>
>> Justus Winter writes:
>>
>>> I think I found a reproducer:
>>>
>>> mkdir -p /tmp/selection-bug/{tmp,new,cur}
>>> echo "[database]
>>> path=/tmp/selection-bug" > /tmp/selection-bug/notmuch-config
>>> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch new
>>> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config emacs -q --eval "(require
>>> 'notmuch)"
>>> # M-x notmuch-hello, search for '*'
>>> notmuch show --format=raw id:87edvlanuh@tethera.net |
>>> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
>>> # refresh search buffer, press down once, i.e. navigate to "End of search
>>> results".
>>> notmuch show --format=raw id:87fsfuuxwn.fsf@thinkbox |
>>> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
>>> notmuch show --format=raw id:8735buw7gf@ngraves.fr |
>>> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
>>> # refresh search buffer, bug: Nicolas' mail is green, mine is opened on
>>> pressing enter.
>>
>> 1) Do you think the particular messages matter? I don't have
>>id:8735buw7gf@ngraves.fr
>
> Uh, you are right. I meant to pick mails form notmuch@, but this one is
> from another list. I actually reduced the example further:
>
> mkdir -p /tmp/selection-bug/{tmp,new,cur}
> echo "[database]
> path=/tmp/selection-bug" > /tmp/selection-bug/notmuch-config
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch new
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config emacs -q --eval "(require
> 'notmuch)"
> # M-x notmuch-hello, search for '*'
> notmuch show --format=raw id:87edvlanuh@tethera.net |
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
> # refresh search buffer, press down once, i.e. navigate to "End of search
> results".
> notmuch show --format=raw id:87fsfuuxwn.fsf@thinkbox |
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
> # refresh search buffer, bug: your mail is green, mine is opened on pressing
> enter.
>
> (I.e., just drop Nicolas' mail.)
>
> The bug can be seen even with just one mail, but then its impact is less
> severe:
>
> mkdir -p /tmp/selection-bug/{tmp,new,cur}
> echo "[database]
> path=/tmp/selection-bug" > /tmp/selection-bug/notmuch-config
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch new
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config emacs -q --eval "(require
> 'notmuch)"
> # M-x notmuch-hello, search for '*', press down once, i.e. navigate past the
> line "End of search results".
> notmuch show --format=raw id:87edvlanuh@tethera.net |
> NOTMUCH_CONFIG=/tmp/selection-bug/notmuch-config notmuch insert
> # refresh search buffer, bug: your mail is green, but in fact "End of search
> results" is selected and is "acted upon" when pressing enter.
>
>> 2) Just to be clear, that script has to be run manually, with the
>>comments corresponding to interactive actions in emacs?
>
> Yes.
>
> As additional data point, I cannot reproduce the problem on my Desktop
> which has the same emacs version but uses elpa-notmuch 0.31.4-2. On
> this system, if I run the reproducer, there is no line with a green
> background, so it is less misleading.
My desktop now behaves exactly like my laptop, and it is really
annoying. Surely I'm not the only one with this problem. Can no one
else reproduce this?
Best,
Justus
signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Am Di., 4. Apr. 2023 um 00:54 Uhr schrieb Carl Worth : > > Ouch. > Yes, we're feeling with you, Fulvio :| > It's really unfortunate if notmuch-mutt makes it that easy to throw away > your email. > > That sounds like a nasty bug that should be fixed in that program. To be fair to notmuch-mutt: ``` --output-dir DIR Store search results as (symlink) messages under maildir DIR. Beware: DIR will be overwritten. (Default: F<~/.cache/notmuch/mutt/results/>) ``` So the default is safe, and the warning is there. It would be a bug only if notmuch-mutt descended up and deleted a root dir or such. Maybe naming the option `scratch-dir` or `cache-dir` or such could have helped, but otoh the name of the default is there. Note that neomutt incorporates notmuch-mutt's functionality, it might be a better choice for this and other reasons. > As for recovering, I suppose there _is_ a fair amount of detail in your > notmuch index from all of the position-indexed terms, (as long as you > haven't run "notmuch new" again since the mail was deleted). > > If you still have a large Xapian database in the notmuch database > directory, it would be theoretically possible to recover a lot of the > email content. But I don't know that anyone has ever written a recovery > tool to help with that process. > That is an interesting question for the insiders, indeed - can e-mail bodies be recovered fully (minus capitalisation, stemming)? In Fulvio's case I would try a file-system dump (if it's not too late) and a recovery tool depending on the fs type. Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Mon, Apr 3, 2023 at 5:46 AM David Bremner wrote: > > David Bremner writes: > > > > > I'm intrigued (and indeed I hadn't really thought about the degree to > > which we were re-inventing git-fast-import and friends); however so far > > my experiments did not get far enough to say anything conclusive. > > > > I did manage to finish, about 70 minutes elapsed. > > Although you'r probably right that a file of tags is the right > representation (it is what git-annex uses also), I think we'd need to > define a custom merge driver to take unions of lists in the same way > that git-annex does. Otherwise merging will be less automagic than it is > now. I'm not familiar with git-annex, I would need to see an example of such merging happening. One advantage of using the fast-import format is that it's easy to change it, or support multiple formats. In fact, the format could be specified in the URL, like `nm::1:$HOME/mail` for the current notmuch-git format, and `nm::2:$HOME/mail` for the new. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Mon, Apr 3, 2023 at 6:37 PM David Bremner wrote:
>
> Felipe Contreras writes:
>
> >
> > That should work to update existing tags, but how are we going to
> > detect if a message has disappeared? Or is that not a thing?
>
> Indeed the same thought had occurred to me not long ago. I remembered
> (belately) that I'd been through some similar thought process with nmbug.
> Messages can and do disappear. So for I guess that optimization not OK,
> at least not without some complications.
>
> > Does "lastmod:0.." get all the revisions? If so, it might make sense
> > to set $lastmod to 0 initially.
> >
> > Then we could unconditionally do:
> >
> > $db.query('lastmod:%d..' % $lastmod, sort: Notmuch::SORT_UNSORTED)
>
> That would work, but as you point out, we'd need to deal with deletions
> somehow. It occurs to me that wr_export also needs to be able to handle
> disappearing message-ids. I suppose like notmuch-restore it can just
> complain and skip any missing ones. It's tempting to try to do some kind
> of lazy cleanup at that point, but I don't really see how that fits with
> the remote-helper protocol.
We could have an external tool, something like `git-notmuch-fsck` or
something that the user has to regularly execute, as `git fsck` was in
the past.
Or we could say that after jumping a certain threshold of lastmod we
delete all the messages and start from scratch, perhaps every 1000
revisions.
Or maybe the query could generate a virtual tag if a message was
deleted since the previous lastmod (e.g. "nm::deleted"). Then it would
be trivial for the remote helper to tell that to git.
I lean towards the threshold, because that way the user doesn't need
to do anything, and there's no modifications needed in libnotmuch.
Cheers.
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes:
>
> That should work to update existing tags, but how are we going to
> detect if a message has disappeared? Or is that not a thing?
Indeed the same thought had occurred to me not long ago. I remembered
(belately) that I'd been through some similar thought process with nmbug.
Messages can and do disappear. So for I guess that optimization not OK,
at least not without some complications.
> Does "lastmod:0.." get all the revisions? If so, it might make sense
> to set $lastmod to 0 initially.
>
> Then we could unconditionally do:
>
> $db.query('lastmod:%d..' % $lastmod, sort: Notmuch::SORT_UNSORTED)
That would work, but as you point out, we'd need to deal with deletions
somehow. It occurs to me that wr_export also needs to be able to handle
disappearing message-ids. I suppose like notmuch-restore it can just
complain and skip any missing ones. It's tempting to try to do some kind
of lazy cleanup at that point, but I don't really see how that fits with
the remote-helper protocol.
d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Ouch. It's really unfortunate if notmuch-mutt makes it that easy to throw away your email. That sounds like a nasty bug that should be fixed in that program. As for recovering, I suppose there _is_ a fair amount of detail in your notmuch index from all of the position-indexed terms, (as long as you haven't run "notmuch new" again since the mail was deleted). If you still have a large Xapian database in the notmuch database directory, it would be theoretically possible to recover a lot of the email content. But I don't know that anyone has ever written a recovery tool to help with that process. -Carl On Mon, Apr 03 2023, Fulvio Pizzigoni wrote: > On Mon, Apr 03, 2023 at 08:39:02PM +0200, Michael J Gruber wrote: >> Am Mo., 3. Apr. 2023 um 20:17 Uhr schrieb Fulvio Pizzigoni >> : >> > >> > >> > >> > >> > >> > >> > Hi Carl e thanks for your prompt answer. >> > >> > As you suggested, I add notmuch@notmuchmail.org email as well. >> > >> > This is what I did: >> > fulvio@linux:~$ notmuch setup >> > Your full name [fulvio]: >> > Your primary email address [my address]: return >> > Additional email address [Press 'Enter' if none]: return >> > Top-level directory of your email archive [/home/fulvio/.mutt]: return >> > Tags to apply to all new messages (separated by spaces) [unread inbox]: >> > return >> > Tags to exclude when searching messages (separated by spaces) [deleted >> > spam]: return >> > fulvio@linux:~$ >> > >> > After this my .mutt directory (~ 4 GB di mail-boxess) appears so: >> > fulvio@linux:~$ ll .mutt >> > totale 12 >> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur >> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new >> > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp >> > fulvio@linux:~$ du -h .mutt >> > 4,0K.mutt/new >> > 4,0K.mutt/tmp >> > 4,0K.mutt/cur >> > 16K .mutt >> > >> > That's all. >> > >> > Terrible! >> > >> > What happened? >> > >> > Thanks in advance. >> > >> > Fulvio > > Hi Michael. > >> >> First of all: Do you have a back-up? > > G, no :-(( > >> >> notmuch itself does not delete mails, as Carl pointed out. In addition >> to notmuch you mentioned notmuch-mutt. Did you run that manually or >> using some mutt config? > > Yes I did this manually: > fulvio@linux:~$ notmuch-mutt -o .mutt/ search > >> It creates a maildir of symlinks with search >> results. In order to do so, it deletes the maildir ... > > I think that's the cause. > > ... And I think it's irreparable. Am I wrong? > > Thanks in advance. > > Fulvio > >> Usually this sits in a cache dir, though. >> >> Michael >> >> > On Mon, Apr 03, 2023 at 09:27:22AM -0700, Carl Worth wrote: >> > > Hi Fulvio, >> > > >> > > I've never used notmuch-mutt. >> > > >> > > But notmuch itself doesn't delete any mail. It's really paranoid that >> > > way, (knowing how valuable mail is). >> > > >> > > I would suggest you write an email to the notmuch@notmuchmail.org >> > > mailing list where you will be able to reach more people likely to have >> > > experience with all of the software you were using. >> > > >> > > And if you could provide more details on the actual steps you used, that >> > > would be useful. For example, you said "configuration process" and "at >> > > the end". But what actual commands were you running on those steps, for >> > > example? >> > > >> > > -Carl >> > > >> > > On Mon, Apr 03 2023, Fulvio Pizzigoni wrote: >> > > > Hi. >> > > > >> > > > I have installed packages notmuch and notmuch-mutt. >> > > > >> > > > During configuration process I have indicated the directory used from >> > > > Mutt for your mail-boxes. >> > > > >> > > > The configuration process have not indicated any allert. >> > > > >> > > > At the end the mail-boxes in that directory was removed; 3 new >> > > > empty sub-directory were created: cur, new, tmp. >> > > > >> > > > Was the data irremediably lost? >> > > > >> > > > Thanks in advance. >> > > > >> > > > Fulvio Pizzigoni >> > ___ >> > notmuch mailing list -- notmuch@notmuchmail.org >> > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
On Mon, Apr 03, 2023 at 08:39:02PM +0200, Michael J Gruber wrote: > Am Mo., 3. Apr. 2023 um 20:17 Uhr schrieb Fulvio Pizzigoni > : > > > > > > > > > > > > > > Hi Carl e thanks for your prompt answer. > > > > As you suggested, I add notmuch@notmuchmail.org email as well. > > > > This is what I did: > > fulvio@linux:~$ notmuch setup > > Your full name [fulvio]: > > Your primary email address [my address]: return > > Additional email address [Press 'Enter' if none]: return > > Top-level directory of your email archive [/home/fulvio/.mutt]: return > > Tags to apply to all new messages (separated by spaces) [unread inbox]: > > return > > Tags to exclude when searching messages (separated by spaces) [deleted > > spam]: return > > fulvio@linux:~$ > > > > After this my .mutt directory (~ 4 GB di mail-boxess) appears so: > > fulvio@linux:~$ ll .mutt > > totale 12 > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp > > fulvio@linux:~$ du -h .mutt > > 4,0K.mutt/new > > 4,0K.mutt/tmp > > 4,0K.mutt/cur > > 16K .mutt > > > > That's all. > > > > Terrible! > > > > What happened? > > > > Thanks in advance. > > > > Fulvio Hi Michael. > > First of all: Do you have a back-up? G, no :-(( > > notmuch itself does not delete mails, as Carl pointed out. In addition > to notmuch you mentioned notmuch-mutt. Did you run that manually or > using some mutt config? Yes I did this manually: fulvio@linux:~$ notmuch-mutt -o .mutt/ search > It creates a maildir of symlinks with search > results. In order to do so, it deletes the maildir ... I think that's the cause. ... And I think it's irreparable. Am I wrong? Thanks in advance. Fulvio > Usually this sits in a cache dir, though. > > Michael > > > On Mon, Apr 03, 2023 at 09:27:22AM -0700, Carl Worth wrote: > > > Hi Fulvio, > > > > > > I've never used notmuch-mutt. > > > > > > But notmuch itself doesn't delete any mail. It's really paranoid that > > > way, (knowing how valuable mail is). > > > > > > I would suggest you write an email to the notmuch@notmuchmail.org > > > mailing list where you will be able to reach more people likely to have > > > experience with all of the software you were using. > > > > > > And if you could provide more details on the actual steps you used, that > > > would be useful. For example, you said "configuration process" and "at > > > the end". But what actual commands were you running on those steps, for > > > example? > > > > > > -Carl > > > > > > On Mon, Apr 03 2023, Fulvio Pizzigoni wrote: > > > > Hi. > > > > > > > > I have installed packages notmuch and notmuch-mutt. > > > > > > > > During configuration process I have indicated the directory used from > > > > Mutt for your mail-boxes. > > > > > > > > The configuration process have not indicated any allert. > > > > > > > > At the end the mail-boxes in that directory was removed; 3 new > > > > empty sub-directory were created: cur, new, tmp. > > > > > > > > Was the data irremediably lost? > > > > > > > > Thanks in advance. > > > > > > > > Fulvio Pizzigoni > > ___ > > notmuch mailing list -- notmuch@notmuchmail.org > > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
On Mon, Apr 03, 2023 at 03:38:21PM -0300, David Bremner wrote: > Fulvio Pizzigoni writes: > > > > After this my .mutt directory (~ 4 GB di mail-boxess) appears so: > > fulvio@linux:~$ ll .mutt > > totale 12 > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new > > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp > > Hi Fulvio; Hi David. > > I don't understand your situation yet, but I have a preliminary > question. > > Is the date on your computer more or less correct? Yes, it is. > You mention the > directories cur, new, and tmp being created by running notmuch setup; It is correct. > on the other hand your listing above shows they already existed in > February. It is the date when I did the setup. Thanks in advance. Fulvio > > d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Mon, Apr 3, 2023 at 2:40 PM David Bremner wrote:
>
> David Bremner writes:
>
> > Indeed that speeds up the initial clone on this machine from 39 minutes
> > (I switched machines) to 30s. I will play with it a bit more, and report
> > back.
>
> It's not a showstopper, but "git pull" takes about 1/2 the wall time
> (about 2/3 of the CPU time) of the original clone, even if there is only
> one tag changed.
Yes, every fetch should take as much time as the original clone.
> Two potential improvements I can think of.
>
> - notmuch-dump.c calls notmuch_query_set_sort (query,
> NOTMUCH_SORT_UNSORTED). I think I managed to do this (diff below),
> but performance gain was negligible.
OK.
> - Since you cache the lastmod value, you should be able to use it in a
> query. This does make a big difference in my experiments. I had to
> remove the 'deleteall' (otherwise only the changed messages are left
> in the git repo). I'm not 100% this is correct, hopefully you see
> quicker than I. In any case the lastmod query is what notmuch-git
> uses.
That should work to update existing tags, but how are we going to
detect if a message has disappeared? Or is that not a thing?
> diff --git a/git-remote-nm b/git-remote-nm
> index c668b38..cabea26 100755
> --- a/git-remote-nm
> +++ b/git-remote-nm
> @@ -148,9 +148,11 @@ def wr_import(ref)
>wr_data("lastmod: %d\n" % ($lastmod || 0))
>wr_l 'from refs/notmuch/master^0' if $lastmod
>
> - wr_l 'deleteall'
> +# wr_l 'deleteall'
>
> - $db.query('').search_messages.each do |msg|
> + $query=$db.query("lastmod:%d.." % ($lastmod || 0) )
Does "lastmod:0.." get all the revisions? If so, it might make sense
to set $lastmod to 0 initially.
Then we could unconditionally do:
$db.query('lastmod:%d..' % $lastmod, sort: Notmuch::SORT_UNSORTED)
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
David Bremner writes:
> Indeed that speeds up the initial clone on this machine from 39 minutes
> (I switched machines) to 30s. I will play with it a bit more, and report
> back.
It's not a showstopper, but "git pull" takes about 1/2 the wall time
(about 2/3 of the CPU time) of the original clone, even if there is only
one tag changed.
Two potential improvements I can think of.
- notmuch-dump.c calls notmuch_query_set_sort (query,
NOTMUCH_SORT_UNSORTED). I think I managed to do this (diff below),
but performance gain was negligible.
- Since you cache the lastmod value, you should be able to use it in a
query. This does make a big difference in my experiments. I had to
remove the 'deleteall' (otherwise only the changed messages are left
in the git repo). I'm not 100% this is correct, hopefully you see
quicker than I. In any case the lastmod query is what notmuch-git
uses.
diff --git a/git-remote-nm b/git-remote-nm
index c668b38..cabea26 100755
--- a/git-remote-nm
+++ b/git-remote-nm
@@ -148,9 +148,11 @@ def wr_import(ref)
wr_data("lastmod: %d\n" % ($lastmod || 0))
wr_l 'from refs/notmuch/master^0' if $lastmod
- wr_l 'deleteall'
+# wr_l 'deleteall'
- $db.query('').search_messages.each do |msg|
+ $query=$db.query("lastmod:%d.." % ($lastmod || 0) )
+ $query.sort=Notmuch::SORT_UNSORTED
+ $query.search_messages.each do |msg|
hash = Blake2b.hex(msg.message_id, Blake2b::Key.none, 2)
dir1, dir2 = hash[..1], hash[2..]
wr_l 'M 644 inline %s/%s/%s/tags' % [dir1, dir2,
encode_filename(msg.message_id)]
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes: > By distributing the files in multiple directories like notmuch-git > does using BLAKE2b, the operation is much faster. > > I've pushed the changes, now there's a dependency, but you can just > `gem install blake2b`. > > I'm able to clone the database of the performance corpus in 5 seconds: > > % git clone --bare nm::$PWD/mail mail.git Indeed that speeds up the initial clone on this machine from 39 minutes (I switched machines) to 30s. I will play with it a bit more, and report back. I had just finished a pretty graph showing nonlinear growth of the old version, but I guess nobody cares now ;) d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Am Mo., 3. Apr. 2023 um 20:17 Uhr schrieb Fulvio Pizzigoni : > > > > > > > Hi Carl e thanks for your prompt answer. > > As you suggested, I add notmuch@notmuchmail.org email as well. > > This is what I did: > fulvio@linux:~$ notmuch setup > Your full name [fulvio]: > Your primary email address [my address]: return > Additional email address [Press 'Enter' if none]: return > Top-level directory of your email archive [/home/fulvio/.mutt]: return > Tags to apply to all new messages (separated by spaces) [unread inbox]: return > Tags to exclude when searching messages (separated by spaces) [deleted spam]: > return > fulvio@linux:~$ > > After this my .mutt directory (~ 4 GB di mail-boxess) appears so: > fulvio@linux:~$ ll .mutt > totale 12 > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp > fulvio@linux:~$ du -h .mutt > 4,0K.mutt/new > 4,0K.mutt/tmp > 4,0K.mutt/cur > 16K .mutt > > That's all. > > Terrible! > > What happened? > > Thanks in advance. > > Fulvio First of all: Do you have a back-up? notmuch itself does not delete mails, as Carl pointed out. In addition to notmuch you mentioned notmuch-mutt. Did you run that manually or using some mutt config? It creates a maildir of symlinks with search results. In order to do so, it deletes the maildir ... Usually this sits in a cache dir, though. Michael > On Mon, Apr 03, 2023 at 09:27:22AM -0700, Carl Worth wrote: > > Hi Fulvio, > > > > I've never used notmuch-mutt. > > > > But notmuch itself doesn't delete any mail. It's really paranoid that > > way, (knowing how valuable mail is). > > > > I would suggest you write an email to the notmuch@notmuchmail.org > > mailing list where you will be able to reach more people likely to have > > experience with all of the software you were using. > > > > And if you could provide more details on the actual steps you used, that > > would be useful. For example, you said "configuration process" and "at > > the end". But what actual commands were you running on those steps, for > > example? > > > > -Carl > > > > On Mon, Apr 03 2023, Fulvio Pizzigoni wrote: > > > Hi. > > > > > > I have installed packages notmuch and notmuch-mutt. > > > > > > During configuration process I have indicated the directory used from > > > Mutt for your mail-boxes. > > > > > > The configuration process have not indicated any allert. > > > > > > At the end the mail-boxes in that directory was removed; 3 new > > > empty sub-directory were created: cur, new, tmp. > > > > > > Was the data irremediably lost? > > > > > > Thanks in advance. > > > > > > Fulvio Pizzigoni > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Fulvio Pizzigoni writes: > > After this my .mutt directory (~ 4 GB di mail-boxess) appears so: > fulvio@linux:~$ ll .mutt > totale 12 > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new > drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp Hi Fulvio; I don't understand your situation yet, but I have a preliminary question. Is the date on your computer more or less correct? You mention the directories cur, new, and tmp being created by running notmuch setup; on the other hand your listing above shows they already existed in February. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Data loss
Hi Carl e thanks for your prompt answer. As you suggested, I add notmuch@notmuchmail.org email as well. This is what I did: fulvio@linux:~$ notmuch setup Your full name [fulvio]: Your primary email address [my address]: return Additional email address [Press 'Enter' if none]: return Top-level directory of your email archive [/home/fulvio/.mutt]: return Tags to apply to all new messages (separated by spaces) [unread inbox]: return Tags to exclude when searching messages (separated by spaces) [deleted spam]: return fulvio@linux:~$ After this my .mutt directory (~ 4 GB di mail-boxess) appears so: fulvio@linux:~$ ll .mutt totale 12 drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 cur drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 new drwxr-xr-x 2 fulvio fulvio 4096 18 feb 20.32 tmp fulvio@linux:~$ du -h .mutt 4,0K.mutt/new 4,0K.mutt/tmp 4,0K.mutt/cur 16K .mutt That's all. Terrible! What happened? Thanks in advance. Fulvio On Mon, Apr 03, 2023 at 09:27:22AM -0700, Carl Worth wrote: > Hi Fulvio, > > I've never used notmuch-mutt. > > But notmuch itself doesn't delete any mail. It's really paranoid that > way, (knowing how valuable mail is). > > I would suggest you write an email to the notmuch@notmuchmail.org > mailing list where you will be able to reach more people likely to have > experience with all of the software you were using. > > And if you could provide more details on the actual steps you used, that > would be useful. For example, you said "configuration process" and "at > the end". But what actual commands were you running on those steps, for > example? > > -Carl > > On Mon, Apr 03 2023, Fulvio Pizzigoni wrote: > > Hi. > > > > I have installed packages notmuch and notmuch-mutt. > > > > During configuration process I have indicated the directory used from > > Mutt for your mail-boxes. > > > > The configuration process have not indicated any allert. > > > > At the end the mail-boxes in that directory was removed; 3 new > > empty sub-directory were created: cur, new, tmp. > > > > Was the data irremediably lost? > > > > Thanks in advance. > > > > Fulvio Pizzigoni ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Mon, Apr 3, 2023 at 4:49 AM David Bremner wrote: > Performance-wise the initial clone seems pretty slow. For my 600k > messages I have been waiting a while now. htop tells me that > git-fast-import has about 45 minutes of CPU time at this point. This > machine is not that fast, but for comparison an initial (i.e. fresh > repo, no caching) "notmuch git commit" takes about 15-20s. I found the problem. If all the files are in the same directory, `git fast-import` spends a lot of time comparing all the paths. By distributing the files in multiple directories like notmuch-git does using BLAKE2b, the operation is much faster. I've pushed the changes, now there's a dependency, but you can just `gem install blake2b`. I'm able to clone the database of the performance corpus in 5 seconds: % git clone --bare nm::$PWD/mail mail.git Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Mon, Apr 3, 2023 at 4:49 AM David Bremner wrote:
>
> Felipe Contreras writes:
>
> > Hi,
> >
> > I noticed you promoted notmuch-git as a user tool to toy around with it.
> >
> > Very quickly I realized that most of what it does is something I've
> > been working on for at least 10 years: making git work with other
> > tools.
> >
> > I presume you haven't heard of git remote-helpers [1], because they do
> > precisely what notmuch-git is trying to do.
> >
> > As a proof of concept I created a remote helper for notmuch [2]. If
> > you have this script (`git-remote-nm`) anywhere in your path, git will
> > interpret URLs prefixed with "nm::" as notmuch transports, and you can
> > do:
> >
> > git clone nm::$HOME/mail
>
> I'm intrigued (and indeed I hadn't really thought about the degree to
> which we were re-inventing git-fast-import and friends); however so far
> my experiments did not get far enough to say anything conclusive.
>
> I tried your script with the bindings from master (554690) but it does
> not seem to like my split configuration, where the database lives in
> ~/.local/share/share/notmuch/default/xapian.
Just clone the xapian database instead of the Maildir:
% git clone nm::$HOME/.local/share/share/notmuch/default/
> Performance-wise the initial clone seems pretty slow. For my 600k
> messages I have been waiting a while now. htop tells me that
> git-fast-import has about 45 minutes of CPU time at this point. This
> machine is not that fast, but for comparison an initial (i.e. fresh
> repo, no caching) "notmuch git commit" takes about 15-20s.
That's weird. In my tests generating the fast-export output is almost
instantaneous, which means `git fast-import` is the one that is slow.
And it seems it starts to get slow after a certain point, so perhaps
it's not optimized to receive many files in one go.
> If you need a larger corpus of messages to play with, the notmuch
> performance suite includes about 400k messages, and running T00-new.sh
> will build a notmuch database that you can clone.
I tried that, the database has 194562 messages, and it takes 1:43
minutes to clone in my machine.
It's weird it takes so long in your machine.
Can you try to hardcode a search query to limit the number of messages?
Just put something in here:
$db.query('').search_messages.each
Cheers.
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
David Bremner writes: > > I'm intrigued (and indeed I hadn't really thought about the degree to > which we were re-inventing git-fast-import and friends); however so far > my experiments did not get far enough to say anything conclusive. > I did manage to finish, about 70 minutes elapsed. Although you'r probably right that a file of tags is the right representation (it is what git-annex uses also), I think we'd need to define a custom merge driver to take unions of lists in the same way that git-annex does. Otherwise merging will be less automagic than it is now. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Felipe Contreras writes: > Hi, > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > Very quickly I realized that most of what it does is something I've > been working on for at least 10 years: making git work with other > tools. > > I presume you haven't heard of git remote-helpers [1], because they do > precisely what notmuch-git is trying to do. > > As a proof of concept I created a remote helper for notmuch [2]. If > you have this script (`git-remote-nm`) anywhere in your path, git will > interpret URLs prefixed with "nm::" as notmuch transports, and you can > do: > > git clone nm::$HOME/mail I'm intrigued (and indeed I hadn't really thought about the degree to which we were re-inventing git-fast-import and friends); however so far my experiments did not get far enough to say anything conclusive. I tried your script with the bindings from master (554690) but it does not seem to like my split configuration, where the database lives in ~/.local/share/share/notmuch/default/xapian. $ git clone nm::/home/bremner/Maildir Cloning into 'Maildir'... /home/bremner/.config/scripts/git-remote-nm:164:in `initialize': failed to read/write file (Notmuch::FileError) from /home/bremner/.config/scripts/git-remote-nm:164:in `new' from /home/bremner/.config/scripts/git-remote-nm:164:in `' If I make a fake .notmuch directory, then it seems to work. I'm not sure if this is an issue with the bindings or with the script. Conceptually there is also the question of how to handle split configurations as a URL. Performance-wise the initial clone seems pretty slow. For my 600k messages I have been waiting a while now. htop tells me that git-fast-import has about 45 minutes of CPU time at this point. This machine is not that fast, but for comparison an initial (i.e. fresh repo, no caching) "notmuch git commit" takes about 15-20s. If you need a larger corpus of messages to play with, the notmuch performance suite includes about 400k messages, and running T00-new.sh will build a notmuch database that you can clone. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/3] ruby: get rid of Tags object
On Sun, Apr 2, 2023 at 5:18 PM David Bremner wrote: > > Felipe Contreras writes: > > > We don't need a Tags enumerable object only for a small number of strings, > > we > > can just get them directly. > > > > This fixes an interaction problem where we might request two tags iterables > > from the same message: > > > > tags_0 = notmuch_message_get_tags(message); > > I have applied this series to master. > > By the way I noticed that the formatting of the old bindings code does > not match the output of uncrustify -c ./devel/uncrustify/cfg $file. > This is not a serious problem, but it does make the review process a bit > noisy (since my clumsy script reformats every file touched by a given > commit). Do you have any objection to my just going through and > reformatting the bindings code with uncrustify at some point? It can be > done as patches also, but it's a bit silly because there can be many > small diffs. I have no problem with changing the format either in one go or as separate patches. *Except* for `sp_not`: `if (! cond)` makes my eyes bleed. For me unary operators act on the thing next to them, so a space before that makes it less clear, not more. Given that I'm the one that has worked on these bindings the most, and probably will be the one that works on them the most in the future, I would appreciate that small compromise on the coding style. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: v2 Index some attachements as text
David Bremner writes: > This obsoletes the series starting with > > id:20220903232839.1473915-2-da...@tethera.net > > Compared to that series this is rebased against master, it has some > more tests (including both positive and negative tests) and it > documents the non-anchoredness of the involved regex search. > I have applied this series to master. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/3] ruby: get rid of Tags object
Felipe Contreras writes: > We don't need a Tags enumerable object only for a small number of strings, we > can just get them directly. > > This fixes an interaction problem where we might request two tags iterables > from the same message: > > tags_0 = notmuch_message_get_tags(message); I have applied this series to master. By the way I noticed that the formatting of the old bindings code does not match the output of uncrustify -c ./devel/uncrustify/cfg $file. This is not a serious problem, but it does make the review process a bit noisy (since my clumsy script reformats every file touched by a given commit). Do you have any objection to my just going through and reformatting the bindings code with uncrustify at some point? It can be done as patches also, but it's a bit silly because there can be many small diffs. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/2] ruby: database open improvements
On Fri, Mar 31, 2023 at 5:07 AM David Bremner wrote: > > Felipe Contreras writes: > > > Essentially we want to do notmuch_database_open_with_config() and load the > > default database. > > > > Applied to master. Thanks for this change, both are things I'd wanted to > fix. Good. Since you chose to pick this series, I just sent another patch I have lying around to reorganize this initializer code. Cheers. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 1/1] lib: replace some uses of Query::MatchAll with a thread-safe alternative
Kevin Boulain writes: > This replaces two instances of Xapian::Query::MatchAll with the > equivalent but thread-safe alternative Xapian::Query(std::string()). > Xapian::Query::MatchAll maintains an internal pointer to a refcounted > Xapian::Internal::QueryTerm. > Applied to master, thanks, and sorry for the delay. By the way I see sfsexp has merged your related PR last week. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/2] ruby: database open improvements
Felipe Contreras writes: > Essentially we want to do notmuch_database_open_with_config() and load the > default database. > Applied to master. Thanks for this change, both are things I'd wanted to fix. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v2 5/5] test: add test for notmuch_message_remove_all_properties_with_prefix
Kevin Boulain writes: > It wasn't covered, though it shares most of its implementation with > notmuch_message_remove_all_properties. I have applied this series to master, with one commit added in the middle commit 336334996750240608d5f29ed5dd8e40a69c4d79 Author: David Bremner Date: Thu Mar 30 07:56:17 2023 -0300 test: reveal notmuch_message_remove_all_properties as broken Close and re-open the database to show that the removal is not committed to the database. I also had to manually deal with a merge conflict for the last patch, so you might want to double check that. In my next life, I will make more liberal use of backup_database / restore_database, to make the tests in this file less order dependent. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v3 1/2] test: uncaught exception when editing properties of a removed message
Kevin Boulain writes: > These two functions don't fail gracefully when editing a removed > message: > BROKEN edit property on removed message without uncaught exception > --- T610-message-property.20.EXPECTED 2023-02-27 11:33:25.792764376 > + > +++ T610-message-property.20.OUTPUT 2023-02-27 11:33:25.793764381 > + > @@ -1,2 +1,3 @@ > == stdout == > == stderr == > +terminate called after throwing an instance of > 'Xapian::DocNotFoundError' > Series applied to master, thanks! d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: sync removed properties to the database
On 2023-03-29 at 08:32 -03, David Bremner wrote: > It would be nice to structure this in terms of a known broken test > (perhaps modify the existing one to reopen the database and dump the > properties) > that is then fixed by patch adding the sync. I have restructured the commits to hopefully make that clearer. I've optimized for the smallest diff but I can rewrite some of the tests if you'd prefer. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v2 1/2] test: uncaught exception when editing properties of a removed message
On 2023-03-29 at 07:42 -03, David Bremner wrote: > Thanks for the speedy update. I hate to do this but I think the message > is now _too_ specific. Goldilocks and the three tests I guess. > > In particular, is there some reason to think that the number 54 is > stable here? I'm worried about the test being flaky if there is some > Xapian related change. Maybe just sed the number away? No worries, it's my fault. I've checked the expected status code and printed a generic error message instead. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
On Wed, Mar 29, 2023 at 3:50 AM Michael J Gruber wrote: > > Am Mi., 29. März 2023 um 10:41 Uhr schrieb Felipe Contreras > : > > > > Hi, > > > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > > > Very quickly I realized that most of what it does is something I've > > been working on for at least 10 years: making git work with other > > tools. > > > > I presume you haven't heard of git remote-helpers [1], because they do > > precisely what notmuch-git is trying to do. > > > > Hi Felipe > > that's an interesting idea for sure. When I came across `notmuch-git` > first I wondered whether it rather should be`git-notmuch`, i.e. a > subcommand to `git`. I admit that - given its preexistence as nmbug - > I was never quite sure what to use it for. Maybe sync tags for mail > stores whose content you sync otherwise? `public-inbox` came to my > mind in this context, too. (I wondered about an nm backend for that, > i.e. a public-inbox backed mailstore for notmuch, without multiple > checkouts.) Yes, I also thought of a public-inbox backend for notmuch, but for that some notion of virtual files should probably be introduced, and I think at the moment the current code of notmuch relies on real files. > So, if we consider the notmuch database (more precisely: the dump > output) as a "remote", then what is the history? I understand that we > can transfer and transform its content in the form of blobs as > specific paths encoding mid etc. Is the history stored by current > `notmuch-git` something secondary (say, like the history of notes refs > in git) which can be discarded? The history is arbitrarily created. Say you have two `git-remote-nm` repositories keeping track of the same notmuch database. Except one does a daily `git fetch`, and the other does it once a month. The former is going to have many more commits, and thus a more granular history. Think of it as a `git fetch` just being a simpler version of some custom `notmuch dump | convert-script | git commit`. > Note that I haven't looked at your code thoroughly yet (I'm not a > rubyist), You don't need to be a rubyist, just copy the script anywhere in your path, and clone your mail database. As long as you never do `git push`, the operations are going to be read-only, but if you want to be extra safe, remove " mode: Notmuch::MODE_READ_WRITE" from the code, and/or copy the mail database somewhere temporary. Do `git fetch` regularly, and you'll see how a history of "origin/master" is being created. > and I'm all for using git tools to do gittish things and > more; I'm just wondering whether fast-import/export cover what current > `notmuch-git` intends to do. They are probably the best tool for > "cloning" an existing nm-db into a git repo of mid-tag associations. > And if all you want is a gittish transport for nm tags then that's > probably perfect! > > `notmuch-git` seems to be about handling both updates (commit etc) You can do the same with `git-notmuch`: just do `git commit`. I do that in the tests to add a tag [1]. > and queries (log etc), Ditto: just do `git log`. If you look at the code of `notmuch-git`, it's just a wrapper for `git log --name-status --no-renames`. > In summary, I think a notmuch-git repo is more than a conversion of > notmuch-dump output (it adds history and commit messages; we have a > "one-sided inverse" only), and the notmuch-git command is more than a > converter between the respective data stores. So is `git-notmuch`: every time you do `git fetch` a commit is created. The history is all there. Cheers. [1] https://github.com/felipec/git-notmuch/blob/cdb2954abf3eb9f2f04f71fd2385a34653f758f5/t/basic.t#L87 -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: sync removed properties to the database
Kevin Boulain writes: > > Yeah, talloc's documentation confirms that when the context is free'd > every child is also free'd. Not quite sure what style is preferred > (explicit or implicit, implicit sure leads to nicer code here). In general implicit de-allocation is fine. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: sync removed properties to the database
Kevin Boulain writes: > On 2023-03-03 at 00:39 +02, Tomi Ollila wrote: >> Somehow testkey1 = testvalue1 disappeared from the test code (which is >> probably expected -- perhaps the commit message of the *change* 1/2 >> tried to point to that ;D) > > Yes, that proves notmuch_message_remove_all_properties is broken without > the patch. The eponymous test gave the impression the property had been > removed (stdout is empty) but I believe this is only due to > _notmuch_message_invalidate_metadata. Now the "dump message properties" > test doesn't list it anymore, which is what I expect. It would be nice to structure this in terms of a known broken test (perhaps modify the existing one to reopen the database and dump the properties) that is then fixed by patch adding the sync. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH v2 1/2] test: uncaught exception when editing properties of a removed message
Kevin Boulain writes: > +== stderr == > +A Xapian exception occurred at message-property.cc:XXX: No termlist for > document 54 > +EOF > +test_expect_equal_file EXPECTED OUTPUT > + > +add_email_corpus Hi Kevin; Thanks for the speedy update. I hate to do this but I think the message is now _too_ specific. Goldilocks and the three tests I guess. In particular, is there some reason to think that the number 54 is stable here? I'm worried about the test being flaky if there is some Xapian related change. Maybe just sed the number away? d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Reimagining notmuch-git/nmbug
Am Mi., 29. März 2023 um 10:41 Uhr schrieb Felipe Contreras : > > Hi, > > I noticed you promoted notmuch-git as a user tool to toy around with it. > > Very quickly I realized that most of what it does is something I've > been working on for at least 10 years: making git work with other > tools. > > I presume you haven't heard of git remote-helpers [1], because they do > precisely what notmuch-git is trying to do. > Hi Felipe that's an interesting idea for sure. When I came across `notmuch-git` first I wondered whether it rather should be`git-notmuch`, i.e. a subcommand to `git`. I admit that - given its preexistence as nmbug - I was never quite sure what to use it for. Maybe sync tags for mail stores whose content you sync otherwise? `public-inbox` came to my mind in this context, too. (I wondered about an nm backend for that, i.e. a public-inbox backed mailstore for notmuch, without multiple checkouts.) So, if we consider the notmuch database (more precisely: the dump output) as a "remote", then what is the history? I understand that we can transfer and transform its content in the form of blobs as specific paths encoding mid etc. Is the history stored by current `notmuch-git` something secondary (say, like the history of notes refs in git) which can be discarded? Note that I haven't looked at your code thoroughly yet (I'm not a rubyist), and I'm all for using git tools to do gittish things and more; I'm just wondering whether fast-import/export cover what current `notmuch-git` intends to do. They are probably the best tool for "cloning" an existing nm-db into a git repo of mid-tag associations. And if all you want is a gittish transport for nm tags then that's probably perfect! `notmuch-git` seems to be about handling both updates (commit etc) and queries (log etc), too, as a wrapper to git commands. Those may be candidates for other git tools, such as aliases, diff helpers, textconv and such. In summary, I think a notmuch-git repo is more than a conversion of notmuch-dump output (it adds history and commit messages; we have a "one-sided inverse" only), and the notmuch-git command is more than a converter between the respective data stores. It smells more like `git-lfs` or other filter-based approaches, storing the real objects outside of the git repo. But I feel I know too little about `notmuch-git`'s purpose so far. Cheers Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/3] ruby: get rid of FileNames object
On Tue, Mar 28, 2023 at 12:47 AM Felipe Contreras
wrote:
> We can replace the Foo object any other kind of Enumerable, and the
> code works just the same:
>
> foo = %w[inbox unread]
I realized this might be too idiomatic of Ruby, it's the same thing as:
foo = [ 'inbox', 'unread' ]
> foo.each { |e| puts e }
> puts 'first: %s' % foo.first
> puts 'count: %s' % foo.count
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/3] ruby: get rid of FileNames object
On Mon, Mar 27, 2023 at 6:13 PM David Bremner wrote:
>
> Felipe Contreras writes:
>
> > We don't need a FileNames enumerable object only for a small number of
> > strings,
> > we can just get them directly.
> >
> > This iterator is meant to be transient and works only once, so we better
> > just
> > iterate it once.
> >
> > This is the same approach I took with the Tags object, I was waiting for
> > feedback on that approach but since there isn't any and there's no reason
> > this
> > shouldn't work, here's the same for Filenames.
>
> Hi Felipe;
>
> I still haven't had a chance to look at the proposed changes, but I did
> wonder what your plan was as far as migration. Usually with the library
> itself we try to provide fairly smooth upgrades.
There's no migration, there's no functional changes from clients' point of view.
Previously the code returned a Notmuch::Tags object, but this class
included the Enumerable [1] module, making it an Enumerable. So
everything you could do with an Enumerable you could do with
Notmuch::Tags, for example: tags.count().
But we didn't implement all these methods, we only had to implement
`each()`. If `each()` works, then everything else works.
For example this class implements an Enumerable:
class Foo
include Enumerable
def each
yield 'inbox'
yield 'unread'
end
end
And then all these work:
foo = Foo.new
foo.each { |e| puts e }
puts 'first: %s' % foo.first
puts 'count: %s' % foo.count
We can replace the Foo object any other kind of Enumerable, and the
code works just the same:
foo = %w[inbox unread]
foo.each { |e| puts e }
puts 'first: %s' % foo.first
puts 'count: %s' % foo.count
So replacing Notmuch::Tags with an array of strings doesn't change
anything because both are Enumerable, and both yield Strings.
Cheers.
[1] https://rubydoc.info/stdlib/core/Enumerable
--
Felipe Contreras
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 0/3] ruby: get rid of FileNames object
Felipe Contreras writes: > We don't need a FileNames enumerable object only for a small number of > strings, > we can just get them directly. > > This iterator is meant to be transient and works only once, so we better just > iterate it once. > > This is the same approach I took with the Tags object, I was waiting for > feedback on that approach but since there isn't any and there's no reason this > shouldn't work, here's the same for Filenames. Hi Felipe; I still haven't had a chance to look at the proposed changes, but I did wonder what your plan was as far as migration. Usually with the library itself we try to provide fairly smooth upgrades. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: catch xapian exceptions
On 2023-03-27 at 08:30 -03, David Bremner wrote: > So you should probably test that the status is not success, and > ideally print the message. You can see some examples in > T560-lib-error.sh, in particular in the created file c_tail which > handles checking the error code and printing the message. Good point, it has shown I was missing a add_email_corpus. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: catch xapian exceptions
Kevin Boulain writes:
> Since libnotmuch exposes a C interface there's no way for clients to
> diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
> index 944e1810..f7cabe4d 100755
> --- a/test/T610-message-property.sh
> +++ b/test/T610-message-property.sh
> @@ -363,7 +363,6 @@ EOF
Overall this looks good, but I think the tests are a little cryptic as written.
> test_expect_equal_file /dev/null OUTPUT
>
> test_begin_subtest "edit property on removed message without uncaught
> exception"
> -test_subtest_known_broken
> cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> EXPECT0(notmuch_database_remove_message (db, notmuch_message_get_filename
> (message)));
> EXPECT0(notmuch_message_remove_property (message, "example", "example"));
> @@ -371,11 +370,11 @@ EOF
> cat <<'EOF' >EXPECTED
> == stdout ==
> == stderr ==
> +line 30: 3
> EOF
> test_expect_equal_file EXPECTED OUTPUT
In general I think it's better if the output of a test does not change
when marked non-broken. More importantly, in this case it means that the
call to notmuch_message_remove_property is not actually returning 0 any
more. So that output is actually the EXPECT0 assertion failing. I agree
it should print something more helpful (and I understand these
assumptions are not documented). So you should probably test that the
status is not success, and ideally print the message. You can see some
examples in T560-lib-error.sh, in particular in the created file c_tail
which handles checking the error code and printing the message.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: BUG: ruby segfault in notmuch_rb_tags_each
On Thu, May 5, 2022 at 9:16 AM David Bremner wrote: > > arcnmx writes: > > > Hi, I've been running into rare crashes via notmuch-vim for a while now and > > finally got around to tracking down the problem. I apologize for the size > > of the reproduction case, but at least it can be reproduced! > > Pinging Felipe to see if he has any ideas. Gmail didn't raise this to my inbox. Great. -- Felipe Contreras ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: BUG: ruby segfault in notmuch_rb_tags_each
On Sat, Apr 30, 2022 at 5:09 PM arcnmx wrote: > > Hi, I've been running into rare crashes via notmuch-vim for a while now and > finally got around to tracking down the problem. I apologize for the size of > the reproduction case, but at least it can be reproduced! Thanks for the reproduction case, I setted up nix and indeed I was able to reproduce the issue, I narrowed it down to a much simpler test-case. The problem is that Notmuch::Tags is mapped to notmuch_tags_t which in the past was a standalone iterator, but later on could only be used once, in particular I think this commit broke the assumption of the Ruby bindings: d19c5de1 (Add the tag list to the unified message metadata pass., 2010-12-09). The code of notmuch-vim stores this tags iterable, traverses it, and then afterwards tries to traverse it again, even after the original parent (notmuch_message_t) might have been destroyed. The crash can be reproduced in C with something like: tags_0 = notmuch_message_get_tags(message); // Store it for later tags_1 = notmuch_message_get_tags(message); // Traverse it I've sent the patches that fix this in the Ruby bindings, but in the meantime you can try this workaround in the Message constructor of notmuch-vim: @tags = msg.tags.map(&:to_s) I'm attaching a simple script that reproduces the issue. Cheers. -- Felipe Contreras simple.rb Description: application/ruby ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Help in Emacs config
Atanas Janackovski writes: > I have justed strated using `notmuch' in Emacs (doom configuration), and I > love > it! However, there are a couple of things I'd like help with. > > 1. I have a number of accounts that I use. How do I configure my config to > save >draft messages based on the "From" header? There currently isn't any automated filing rules in the way that notmuch-fcc-dirs works. It could be done, if there was sufficient interest (and somebody did it!), > 2. From time to time, I want to view `html' content in my browser. Does anyone >have any tips on how I would do this? I have tried the below but this did > not Not sure if this is what you are after, but you can choose ". o" and give the name of a browser. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Help in Emacs config
On Tue, Mar 07 2023, Atanas Janackovski wrote:
> I have justed strated using `notmuch' in Emacs (doom configuration),
> and I love it!
Great! I'm glad it's useful for you.
> 1. I have a number of accounts that I use. How do I configure my
> config to save draft messages based on the "From" header?
I'm not sure what you're looking for here. Do you want drafts saved in
different locations?
At some level, a draft message is just another message, so you can
search for drafts based on the "From" header just like any other
message.
> 2. From time to time, I want to view `html' content in my
> browser. Does anyone have any tips on how I would do this? I have
> tried the below but this did not seem to work:
This seems to work out of the box for me.
>From looking at the contents of my mailcap-mime-data variable, what
sticks out to me is that "text" and then "html" are at different levels
in the structure such as:
("text"
("html"
(viewer . "/usr/bin/firefox '%s'")
(type . "text/html")
("nametemplate" . "%s.html")
(test . t)
("description" . "HTML Text")
(source . user)))
Have fun with notmuch!
-Carl
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: sync removed properties to the database
On 2023-03-03 at 00:39 +02, Tomi Ollila wrote: > Somehow testkey1 = testvalue1 disappeared from the test code (which is > probably expected -- perhaps the commit message of the *change* 1/2 > tried to point to that ;D) Yes, that proves notmuch_message_remove_all_properties is broken without the patch. The eponymous test gave the impression the property had been removed (stdout is empty) but I believe this is only due to _notmuch_message_invalidate_metadata. Now the "dump message properties" test doesn't list it anymore, which is what I expect. I took the liberty to remove all testkey3 properties for the notmuch_message_remove_all_properties_with_prefix test because none of the following tests really require them. > the `message` is (probably?) talloc_free()d at some point, and > at that time the talloc_asprintf'd data (with message as a reference) > will be freed. Yeah, talloc's documentation confirms that when the context is free'd every child is also free'd. Not quite sure what style is preferred (explicit or implicit, implicit sure leads to nicer code here). ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 2/2] lib/message-property: sync removed properties to the database
On Wed, Mar 01 2023, Kevin Boulain wrote:
> _notmuch_message_remove_all_properties wasn't syncing the message back
> to the database but was still invalidating the metadata, giving the
> impression the properties had actually been removed.
>
> Also move the metadata invalidation to _notmuch_message_remove_terms
> to be closer to what's done in _notmuch_message_modify_property and
> _notmuch_message_remove_term.
I don't understand much of this change, but at least code changes are
simple enough to not look broken (cannot know about flow, I could not
find connection).
Somehow testkey1 = testvalue1 disappeared from the test code (which is
probably expected -- perhaps the commit message of the *change* 1/2
tried to point to that ;D)
So, changes looks good, but cannot say if anything works better...
> ---
> Don't we need to talloc_free the talloc_asprintf'd term_prefix?
the `message` is (probably?) talloc_free()d at some point, and
at that time the talloc_asprintf'd data (with message as a reference)
will be freed.
Tomi
>
> lib/message-property.cc | 4 ++-
> lib/message.cc| 2 ++
> test/T610-message-property.sh | 61 ---
> 3 files changed, 33 insertions(+), 34 deletions(-)
>
> diff --git a/lib/message-property.cc b/lib/message-property.cc
> index d5afa30c..0d658038 100644
> --- a/lib/message-property.cc
> +++ b/lib/message-property.cc
> @@ -123,7 +123,6 @@ _notmuch_message_remove_all_properties (notmuch_message_t
> *message, const char *
> if (status)
> return status;
>
> -_notmuch_message_invalidate_metadata (message, "property");
> if (key)
> term_prefix = talloc_asprintf (message, "%s%s%s", _find_prefix
> ("property"), key,
> prefix ? "" : "=");
> @@ -133,6 +132,9 @@ _notmuch_message_remove_all_properties (notmuch_message_t
> *message, const char *
> /* XXX better error reporting ? */
> _notmuch_message_remove_terms (message, term_prefix);
>
> +if (! _notmuch_message_frozen (message))
> + _notmuch_message_sync (message);
> +
> return NOTMUCH_STATUS_SUCCESS;
> }
>
> diff --git a/lib/message.cc b/lib/message.cc
> index 1b1a071a..53f35dd1 100644
> --- a/lib/message.cc
> +++ b/lib/message.cc
> @@ -719,6 +719,8 @@ _notmuch_message_remove_terms (notmuch_message_t
> *message, const char *prefix)
> /* Ignore failure to remove non-existent term. */
> }
> }
> +
> +_notmuch_message_invalidate_metadata (message, "property");
> }
>
>
> diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh
> index 7ebddae3..dd397e16 100755
> --- a/test/T610-message-property.sh
> +++ b/test/T610-message-property.sh
> @@ -89,17 +89,6 @@ testkey2 = NULL
> EOF
> test_expect_equal_file EXPECTED OUTPUT
>
> -test_begin_subtest "notmuch_message_remove_all_properties"
> -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> -EXPECT0(notmuch_message_remove_all_properties (message, NULL));
> -print_properties (message, "", FALSE);
> -EOF
> -cat <<'EOF' >EXPECTED
> -== stdout ==
> -== stderr ==
> -EOF
> -test_expect_equal_file EXPECTED OUTPUT
> -
> test_begin_subtest "testing string map binary search (via message
> properties)"
> cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> {
> @@ -162,6 +151,17 @@ testkey1 = testvalue1
> EOF
> test_expect_equal_file EXPECTED OUTPUT
>
> +test_begin_subtest "notmuch_message_remove_all_properties"
> +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> +EXPECT0(notmuch_message_remove_all_properties (message, NULL));
> +print_properties (message, "", FALSE);
> +EOF
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED OUTPUT
> +
> test_begin_subtest "notmuch_message_properties: multiple values"
> cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
> EXPECT0(notmuch_message_add_property (message, "testkey1", "bob"));
> @@ -173,7 +173,6 @@ cat <<'EOF' >EXPECTED
> == stdout ==
> testkey1 = alice
> testkey1 = bob
> -testkey1 = testvalue1
> testkey1 = testvalue2
> == stderr ==
> EOF
> @@ -186,23 +185,10 @@ EXPECT0(notmuch_message_add_property (message,
> "testkey3", "testvalue3"));
> EXPECT0(notmuch_message_add_property (message, "testkey3", "alice3"));
> print_properties (message, "testkey", FALSE);
> EOF
> -# expected: 4 values for testkey1, 3 values for testkey3
> -# they are not guaranteed to be sorted, so sort them, leaving the first
> -# line '== stdout ==' and the end ('== stderr ==' and whatever error
> -# may have been printed) alone
> -mv OUTPUT unsorted_OUTPUT
> -awk ' NR == 1 { print; next } \
> - NR < 6 { print | "sort"; next } \
> - NR == 6 { close("sort") } \
> - NR < 9 { print | "sort"; next } \
> - NR == 9 { close("sort") } \
> - { print }' unsorted_OUTPUT > OUTPUT
> -rm unsorted_OUTPUT
> cat <<'EOF' >EXPECTED
> == stdout ==
> testkey1 = alice
> testkey1 = bob
> -testkey1 = testvalue1
> testkey1 =
Re: notmuch-emacs inline image display broken in emacs-29
On Sat, 2023-02-25 at 09:48:08 -0400, David Bremner wrote: > David Bremner writes: > >> Alex Murray writes: >> >> I tried replacing indent-rigidly with a simplified version the and that >> leaves the image inserted, but unfortunately doesn't indent it >> properly. If you want to play with it, my half working patch is >> attached. >> > > Actually, the current implementation using indent-rigidly doesn't indent > inlined images properly either, so maybe that is a way forward. I > remember from last time I considered globally replacing our use of > indent-rigidly [1] there were a few issues to be dealt with, but maybe > this is a reasonable approach. > Thanks for the patch David - I tried the one attached to the previous email it but it doesn't seem to fix the issue for me. I also tried the one in [1] and that did seem to help - images in replies (and hence emails that by default are displayed with an indent) show up now, but not for emails that are the first ones in a thread (and hence have no indent by default). (although I notice in a recent conversation in #notmuch that it did appear to work for one user - but not sure which patch they were using). Any ideas where I should start trying to debug this on my side? I am using a pretty recent build of emacs from master with a the latest notmuch-emacs bits from MELPA - https://melpa.org/#/notmuch (and the notmuch binary 0.37 - as packaged in Ubuntu 23.04 with version 0.37-1ubuntu3 FWIW). > [1]: > https://nmbug.notmuchmail.org/nmweb/show/20211214121726.2631714-1-da...@tethera.net > ___ > notmuch mailing list -- notmuch@notmuchmail.org > To unsubscribe send an email to notmuch-le...@notmuchmail.org ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: crash on incoming email
Antoine Beaupré writes: > > Still, one has to wonder: if emacs can just dump core if someone messes > with my eln-cache, that's... bad, no? Yes, it is bad, but the native compilation functionality is new, so hopefully things will get better over time. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: crash on incoming email
On 2023-02-22 16:23:27, David Bremner wrote: > Antoine Beaupré writes: > >> When I have the attached email in my `inbox` and hit ENTER on the email >> (notmuch-search-show-thread here), Emacs crashes completely with: >> >> fév 22 11:57:10 angela emacs[112721]: >> /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d6175746f2d687363726f6c6c_tab_li> >> fév 22 11:57:10 angela emacs[112721]: >> /usr/bin/emacs(+0x1b811b)[0x55d1080b011b] >> fév 22 11:57:10 angela emacs[112721]: >> /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d666f726d61742d74656d706c617465_> >> fév 22 11:57:10 angela emacs[112721]: >> /usr/bin/emacs(+0x1b811b)[0x55d1080b011b] >> fév 22 11:57:10 angela emacs[112721]: >> /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d666f726d6174_tab_line_format_0+> > > I can't duplicate this here (at least not with emacs -q). I suspect we > have pretty similar Debian environments, but maybe try moving your > .emacs.d/eln-cache out of the way. I keep forgetting those tricks, sorry. I didn't do anything (other than restarting emacs and rebooting and I can'T reproduce this issue anymore... So I guess whenever i'll see those backtraces next, i'll flush the cache and retry. Still, one has to wonder: if emacs can just dump core if someone messes with my eln-cache, that's... bad, no? (I guess you can do pretty much *anything* if you get to mess with that folder anyways, which also makes me quite uncomfortable... but you have a lisp machine or you do not, i guess...) a. -- Lorsque l'on range des objets dans des tiroirs, et que l'on a plus d'objets que de tiroirs, alors un tiroir au moins contient deux objets. - Lejeune-Dirichlet, Peter Gustav ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 1/1] lib/notmuch: update example
Kevin Boulain writes: > Likely missed in 86cbd215e, when notmuch_query_search_messages_st was > renamed to notmuch_query_search_messages. > --- Applied to master, thanks. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [DRAFT 1/1] lib: replace some uses of Query::MatchAll with a thread-safe alternative
Kevin Boulain writes:
> This is what can be expected from the tests when they fail:
>== stderr ==
> +==
> +WARNING: ThreadSanitizer: data race (pid=207931)
> + Read of size 1 at 0x7b1001a0 by thread T2:
> +#0 memcpy (libtsan.so.2+0x62506)
> +#1 void std::__cxx11::basic_string,
> std::allocator >::_M_construct(char*, char*,
> std::forward_iterator_tag) [clone .isra.0] (libxapian.so.30+0x872b3)
> +
> + Previous write of size 8 at 0x7b1001a0 by thread T1:
> +#0 operator new(unsigned long) (libtsan.so.2+0x8ba83)
> +#1 Xapian::Query::Query(std::__cxx11::basic_string std::char_traits, std::allocator > const&, unsigned int, unsigned
> int) (libxapian.so.30+0x855cd)
> ...
I don't know what the difference between our environments is, but these
tests are failing in glib for me.
+==
+WARNING: ThreadSanitizer: data race (pid=392122)
+ Atomic read of size 1 at 0x7b1025c0 by thread T2:
+#0 pthread_rwlock_rdlock
../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1406
(libtsan.so.2+0x447f1)
+#1 g_rw_lock_reader_lock (libglib-2.0.so.0+0xa8b84)
+
+ Previous write of size 8 at 0x7b1025c0 by thread T1:
+#0 malloc
../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:647
(libtsan.so.2+0x3ebb8)
+#1 (libglib-2.0.so.0+0xa880a)
+
...
FWIW I have libglib2.0-0 version 2.74.5-1 on Debian.
> +printf "C compiler supports address sanitizer... "
s/address/thread/
> +test_cmdline="${CC} ${CFLAGS} ${CPPFLAGS} -fsanitize=thread minimal.c
> ${LDFLAGS} -o minimal"
> +if ${test_cmdline} >/dev/null 2>&1 && ./minimal
> @@ -186,7 +186,7 @@ _notmuch_query_string_to_xapian_query (notmuch_database_t
> *notmuch,
> {
> try {
> if (query_string == "" || query_string == "*") {
> - output = Xapian::Query::MatchAll;
> + output = Xapian::Query(std::string());
I think this probably deserves a brief comment like "use thread-safe
alternative to Query::MatchAll.
That is why I was thinking a preprocessor macro / inline-function might
be appropriate, to only need that comment in one place.
> +test_begin_subtest "create"
> +test_C ${MAIL_DIR} ${MAIL_DIR}-2 < +#include
> +#include
> +#include
> +#include
I guess we will find out if any changes are needed to test harness to
portably support pthreads. BSD and macOS are the usual places that have
slightly different expectations with respect to include files and
libraries; I don't know if that applies here.
d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [DRAFT 0/1] lib: replace some uses of Query::MatchAll with a thread-safe alternative
Kevin Boulain writes: > > Thoughts? Should clients using the library gate every function call to > Notmuch behind a lock instead? Ideally that should not be necessary. On the other hand, it may take some time to fix issues with concurrent access. > I can also start a discussion in sfsexp's issue tracker. Yes please. I guess in worst case we can implment come kind of concurrency control to sfsexp (wrap calls in libnotmuch inside some mutex), although that sounds pretty far from ideal. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH 1/1] lib/notmuch: update example
Kevin Boulain writes:
> Likely missed in 86cbd215e, when notmuch_query_search_messages_st was
> renamed to notmuch_query_search_messages.
> ---
> lib/notmuch.h | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/lib/notmuch.h b/lib/notmuch.h
> index ce375c04..73e246e0 100644
> --- a/lib/notmuch.h
> +++ b/lib/notmuch.h
> @@ -1172,7 +1172,8 @@ notmuch_query_search_threads_st (notmuch_query_t
> *query, notmuch_threads_t **out
> *
> * query = notmuch_query_create (database, query_string);
> *
> - * for (messages = notmuch_query_search_messages (query);
> + * notmuch_query_search_messages (query, );
> + * for (;
> * notmuch_messages_valid (messages);
> * notmuch_messages_move_to_next (messages))
> * {
Thanks for the documentation update, but I don't think we should
encourage people to ignore the status code. Perhaps something like
if (notmuch_query_search_messages (query, ))
return EXIT_FAILURE;
for (;
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: notmuch-emacs inline image display broken in emacs-29
David Bremner writes: > Alex Murray writes: > > I tried replacing indent-rigidly with a simplified version the and that > leaves the image inserted, but unfortunately doesn't indent it > properly. If you want to play with it, my half working patch is > attached. > Actually, the current implementation using indent-rigidly doesn't indent inlined images properly either, so maybe that is a way forward. I remember from last time I considered globally replacing our use of indent-rigidly [1] there were a few issues to be dealt with, but maybe this is a reasonable approach. [1]: https://nmbug.notmuchmail.org/nmweb/show/20211214121726.2631714-1-da...@tethera.net ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: notmuch-emacs inline image display broken in emacs-29
Alex Murray writes: > Hi folks, > > I have noticed that inline image display doesn't work anymore in > emacs-29 / git master and have tracked it down to a change to the way > mm-inline-image inserts an image into the buffer. > > In > https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=d2186160a9e978960c0f96bc3b4fc65b5affc170 > mm-inline-image was changed to use insert-image instead of > put-image. This inserts the image into the buffer using a single space > as the string contents with a display property set as the image contents > so that the single space is not shown itself but instead the image is > shown in its place. > > I had thought the culprit then was the use of > notmuch-wash-elide-blank-lines within the default value of > notmuch-show-insert-text/plain-hook - assuming that since the line > containing the image is now just a single space, it then gets deleted > from the buffer and the image is never shown. However, even after > setting notmuch-show-insert-text/plain-hook to nil, images are still not > shown inline by notmuch. > > However, if I locally redefine mm-inline-image to use say a period "." > as the string argument to insert-image then the image appears as > expected. > > Any thoughts on what may be happening here and how best to resolve this? > The problem seems to be the call to indent-rigidly in notmuch-show-lazy-part. The image is actually inserted OK, but then apparently deleted by indent-rigidly (probably because it looks like whitespace). I tried replacing indent-rigidly with a simplified version the and that leaves the image inserted, but unfortunately doesn't indent it properly. If you want to play with it, my half working patch is attached. >From 484f78e11d7f520b76b30b7d92aec15ff71f215f Mon Sep 17 00:00:00 2001 From: David Bremner Date: Sat, 25 Feb 2023 09:12:57 -0400 Subject: [PATCH] WIP: replace use of indent-rigidly in notmuch-show-lazy --- emacs/notmuch-show.el | 21 - 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index 36cce619..522ba1fd 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -324,6 +324,25 @@ position of the message in the thread." ;;; Utilities +;; force all indentation to be spaces at BOL +;; Based on apply-on-rectangle, from rect.el +(defun notmuch--indent-rigidly (start end count) + (cond + ((zerop count) t) + ((< count 0) (indent-rigidly start end count)) + (t +(save-excursion + (let ((startpt (progn (goto-char start) (line-beginning-position))) + (endpt (progn (goto-char end) (line-end-position))) + (spaces (spaces-string count))) + (goto-char startpt) + (while + (progn + (insert spaces) + (cl-incf endpt count) + (and (zerop (forward-line 1)) (bolp) + (<= (point) endpt) + (defmacro with-current-notmuch-show-message ( body) "Evaluate body with current buffer set to the text of current message." `(save-excursion @@ -1024,7 +1043,7 @@ will return nil if the CID is unknown or cannot be retrieved." (narrow-to-region part-beg part-end) (delete-region part-beg part-end) (apply #'notmuch-show-insert-bodypart-internal part-args) - (indent-rigidly part-beg + (notmuch--indent-rigidly part-beg part-end (* notmuch-show-indent-messages-width depth))) (goto-char part-end) -- 2.39.1 ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: emacs/notmuch hangs opening email
David Bremner writes: > Jon Fineman writes: > >> When I try and open one specific email, I get a message of >> "Fontifying..." and then the mouse pointer turns into a watch. >> >> If I C-G it breaks out and shows the email message with a red warning >> at the top of "[some mark read tag changes may have failed]", but >> since reading the message was interrupted some/many key mappings >> aren't set, etc. >> > > Is the message by chance SMIME signed? That can be fixed on the gpgsm > level. Otherwise, the only thing I can think of is to M-x edebug-defun > on notmuch-show--build-buffer, and step through it to see where it gets > stuck. > > d Thanks. Well this email didn't help (but I did learn a bit about edebug), but your reply to another email about emacs crashing with incoming email did. It prompted me to bisect my init.el and I discovered that the following line was causing the issue, even though I wasn't using w3m to view the message. (setq mm-text-html-renderer 'w3m) If I change the above to use gnus-w3m it works. Not sure I have the ability to figure out what is going on beyond that though. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: crash on incoming email
Antoine Beaupré writes: > When I have the attached email in my `inbox` and hit ENTER on the email > (notmuch-search-show-thread here), Emacs crashes completely with: > > fév 22 11:57:10 angela emacs[112721]: > /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d6175746f2d687363726f6c6c_tab_li> > fév 22 11:57:10 angela emacs[112721]: > /usr/bin/emacs(+0x1b811b)[0x55d1080b011b] > fév 22 11:57:10 angela emacs[112721]: > /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d666f726d61742d74656d706c617465_> > fév 22 11:57:10 angela emacs[112721]: > /usr/bin/emacs(+0x1b811b)[0x55d1080b011b] > fév 22 11:57:10 angela emacs[112721]: > /home/anarcat/.emacs.d/eln-cache/28.2-8532cd27/tab-line-e55f541b-b1fd08d4.eln(F7461622d6c696e652d666f726d6174_tab_line_format_0+> I can't duplicate this here (at least not with emacs -q). I suspect we have pretty similar Debian environments, but maybe try moving your .emacs.d/eln-cache out of the way. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Thread safety?
Kevin Boulain writes: > The documentation of MatchNothing says it's thread-safe. > > If you'd like, I'm happy to submit a patch (likely very small, given the > limited number of occurrences) after I review a bit more the > documentation and the code (I'm just starting with Notmuch and Xapian so > that's probably the extent of what I can do). Sounds good. With apologies for the imposing table of contents, please also have a quick look at https://notmuchmail.org/contributing/ Probably the relevant bit is https://notmuchmail.org/contributing/#index5h2 I suspect a (deterministic) regression test is not really feasible. ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Thread safety?
On 2023-02-21 at 07:59 -04, David Bremner wrote: > Thanks for tracking this down. I did try some experiments a while ago > with multi-threaded indexing (which is where the locking you mentioned > arose), but overall it isn't well tested (by me). To be honest I struggled a bit, notably on GLib which emitted false positives under -fsanitize so I resorted to debugging that one manually (but maybe I missed something obvious). I only discovered the issue because the test framework I'm using runs the tests in different threads. > It seems reasonable to try to provide the same level of thread safety > as Xapian does. That's good to hear. > So I guess we should go ahead and replace all of the MatchAll objects. > I'm guessing this might apply to Xapian::Query::MatchNothing as well. > Probably preprocessor macros wrapping Xapian::Query(std::string()) and > whatever the equivalent for MatchNothing is would make this easier to > read. The documentation of MatchNothing says it's thread-safe. If you'd like, I'm happy to submit a patch (likely very small, given the limited number of occurrences) after I review a bit more the documentation and the code (I'm just starting with Notmuch and Xapian so that's probably the extent of what I can do). ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] emacs/tree: use two argument form of setq-local
David Bremner writes: > Apparently the macro setq-local only takes two arguments in Emacs 26.1 > --- > emacs/notmuch-tree.el | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el > index 14775d59..b58fa6a6 100644 > --- a/emacs/notmuch-tree.el > +++ b/emacs/notmuch-tree.el > @@ -1451,11 +1451,11 @@ notmuch-tree buffers, just set >(unless (derived-mode-p 'notmuch-tree-mode) > (user-error "notmuch-tree-outline-mode is only meaningful for notmuch > trees!")) >(if notmuch-tree-outline-mode > - (progn (setq-local outline-regexp "^[^\n]+" > - outline-level #'notmuch-tree-outline--level) > + (progn (setq-local outline-regexp "^[^\n]+") > + (setq-local outline-level #'notmuch-tree-outline--level) >(notmuch-tree-outline--set-visibility)) > -(setq-local outline-regexp (default-value 'outline-regexp) > - outline-level (default-value 'outline-level > +(setq-local outline-regexp (default-value 'outline-regexp)) > +(setq-local outline-level (default-value 'outline-level Applied to master. d ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: [PATCH] emacs/tree: use two argument form of setq-local
Am Di., 21. Feb. 2023 um 12:49 Uhr schrieb David Bremner : > > Apparently the macro setq-local only takes two arguments in Emacs 26.1 > --- > emacs/notmuch-tree.el | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el > index 14775d59..b58fa6a6 100644 > --- a/emacs/notmuch-tree.el > +++ b/emacs/notmuch-tree.el > @@ -1451,11 +1451,11 @@ notmuch-tree buffers, just set >(unless (derived-mode-p 'notmuch-tree-mode) > (user-error "notmuch-tree-outline-mode is only meaningful for notmuch > trees!")) >(if notmuch-tree-outline-mode > - (progn (setq-local outline-regexp "^[^\n]+" > -outline-level #'notmuch-tree-outline--level) > + (progn (setq-local outline-regexp "^[^\n]+") > +(setq-local outline-level #'notmuch-tree-outline--level) > (notmuch-tree-outline--set-visibility)) > -(setq-local outline-regexp (default-value 'outline-regexp) > - outline-level (default-value 'outline-level > +(setq-local outline-regexp (default-value 'outline-regexp)) > +(setq-localoutline-level (default-value 'outline-level > Thanks, notmuch.git master plus this patch builds happily again on EPEL 8 (and 9, and ...). Michael ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
