Re: performance problems with notmuch new

2020-04-29 Thread Eric Wong
David Bremner wrote: > Franz Fellner writes: > > mail takes at least 10 seconds, sometimes even more. It can go into > > minutes when I get lots of mail (~30...). When I run it after a > > reboot I can have breakfast while notmuch starts up... This is all on > > spinning rust. I thought of

Re: performance problems with notmuch new

2020-04-27 Thread David Bremner
Don Zickus writes: > On Fri, Apr 24, 2020 at 08:07:27PM -0300, David Bremner wrote: >> Don Zickus writes: >> >> > >> > Of course I am using this in a strange context. I am deleting emails >> > locally and then running 'notmuch new' to clean up the database. >> > >> [snip] >> > So maybe my use

Re: performance problems with notmuch new

2020-04-27 Thread Don Zickus
On Fri, Apr 24, 2020 at 08:07:27PM -0300, David Bremner wrote: > Don Zickus writes: > > > > > Of course I am using this in a strange context. I am deleting emails > > locally and then running 'notmuch new' to clean up the database. > > > [snip] > > So maybe my use case is unusual and the

Re: performance problems with notmuch new

2020-04-24 Thread David Bremner
Franz Fellner writes: > > On Thu Apr 23 00:21:30 2020, Olly Betts wrote: >> Then I'd try compacting the database (I think there's a "notmuch >> compact" subcommand to do this). > And there we go. Cured the issues. Dropped the very first indexing > from several minutes to 1.5 seconds on the

Re: performance problems with notmuch new

2020-04-24 Thread David Bremner
Don Zickus writes: > > Of course I am using this in a strange context. I am deleting emails > locally and then running 'notmuch new' to clean up the database. > [snip] > So maybe my use case is unusual and the slowness is expected. I'd say it's a known performance bug in notmuch that it

Re: performance problems with notmuch new

2020-04-24 Thread Don Zickus
On Fri, Apr 24, 2020 at 07:36:23AM -0300, David Bremner wrote: > Don Zickus writes: > > > > > The only thing I can think of is my fstrim service runs 1x / week on Monday > > at midnight and maybe that helped clean things up?? Perhaps I should > > increase that frequency or run it manually when

Re: performance problems with notmuch new

2020-04-24 Thread Franz Fellner
On Thu Apr 23 00:21:30 2020, Olly Betts wrote: > First question: what version of Xapian are you using? On my laptop it's 1.4.15 (arch linux) and the desktop runs 1.4.14 (Gentoo linux) > And second thing to check, are you committing each message separately? No, I sync with mbsync which

Re: performance problems with notmuch new

2020-04-24 Thread David Bremner
Don Zickus writes: > > The only thing I can think of is my fstrim service runs 1x / week on Monday > at midnight and maybe that helped clean things up?? Perhaps I should > increase that frequency or run it manually when things go bad. > Is notmuch new still slow on your mail store, or did that

Re: performance problems with notmuch new

2020-04-22 Thread Olly Betts
On Mon, Apr 20, 2020 at 11:36:36AM -0300, David Bremner wrote: > Franz Fellner writes: > > > I also suffer from bad performance of notmuch new. I used notmuch > > some years ago and notmuch new always felt instantanious. Had to stop > > using it because internet was too slow to sync my mails

Re: performance problems with notmuch new

2020-04-22 Thread Franz Fellner
On Mon Apr 20 11:36:36 2020, David Bremner wrote: > What kind of filesystem do you have on your spinning rust? It's ext4 on both HDDs. ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch

Re: performance problems with notmuch new

2020-04-20 Thread Don Zickus
On Mon, Apr 20, 2020 at 11:31:09AM -0300, David Bremner wrote: > Don Zickus writes: > > > > > Hmm, for me --small was 35s and --medium was 32 minutes. This is on a > > i7-9750H / nvme. I would expect numbers similar to yours. > > > > In addition to the breakdown of numbers that I posted, it

Re: performance problems with notmuch new

2020-04-20 Thread Don Zickus
On Wed, Apr 15, 2020 at 05:29:13PM -0300, David Bremner wrote: > Don Zickus writes: > > >> runs in about 30s here (i7 4770 / SSD). Replacing --small with --medium > >> takes about 10M (so a superlinear slowdown in wall clock time, since > >> that represents a 10x scale-up in the corpus size.).

Re: performance problems with notmuch new

2020-04-20 Thread Kim Minh Kaplan
Franz Fellner writes: > I also suffer from bad performance of notmuch new. > I used notmuch some years ago and notmuch new always felt instantanious. > Had to stop using it because internet was too slow to sync my mails :/ > Now (with better internet and a completely new setup using mbsync)

Re: performance problems with notmuch new

2020-04-20 Thread David Bremner
Franz Fellner writes: > I also suffer from bad performance of notmuch new. I used notmuch > some years ago and notmuch new always felt instantanious. Had to stop > using it because internet was too slow to sync my mails :/ Now (with > better internet and a completely new setup using mbsync)

Re: performance problems with notmuch new

2020-04-20 Thread David Bremner
Don Zickus writes: > > Hmm, for me --small was 35s and --medium was 32 minutes. This is on a > i7-9750H / nvme. I would expect numbers similar to yours. > In addition to the breakdown of numbers that I posted, it would be potentially useful to know if it is I/O bound or CPU bound, and what

Re: performance problems with notmuch new

2020-04-18 Thread Franz Fellner
I also suffer from bad performance of notmuch new. I used notmuch some years ago and notmuch new always felt instantanious. Had to stop using it because internet was too slow to sync my mails :/ Now (with better internet and a completely new setup using mbsync) indexing one mail takes at least 10

Re: performance problems with notmuch new

2020-04-15 Thread David Bremner
Don Zickus writes: >> runs in about 30s here (i7 4770 / SSD). Replacing --small with --medium >> takes about 10M (so a superlinear slowdown in wall clock time, since >> that represents a 10x scale-up in the corpus size.). > > Hmm, for me --small was 35s and --medium was 32 minutes. This is on

Re: performance problems with notmuch new

2020-04-15 Thread Don Zickus
On Wed, Apr 15, 2020 at 01:01:52PM -0300, David Bremner wrote: > Don Zickus writes: > > > > > Tips for debugging this? > > > > I ran the notmuch performance/time-test, but after 15 minutes of waiting for > > the initial notmuch new to finish, I gave up and aborted. > > You can try one of the

Re: performance problems with notmuch new

2020-04-15 Thread David Bremner
Don Zickus writes: > > Tips for debugging this? > > I ran the notmuch performance/time-test, but after 15 minutes of waiting for > the initial notmuch new to finish, I gave up and aborted. You can try one of the smaller corpus sizes $ make OPTIONS=--small time-test runs in about 30s here (i7

performance problems with notmuch new

2020-04-15 Thread Don Zickus
Hi, I have noticed my 'notmuch new' command seems awfully slow, maybe 10-20 emails / second on large batches. It goes quick for the first few hundred (maybe close to 100/second), then quickly slows down to about 10/second after processing the first 500 or so. I am guessing that isn't an