[Nmh-workers] Large MH directories

2017-04-08 Thread Thomas Levine
I am considering converting all of my thirteen (?) years of email (600,000 messages) from of Maildir to MH format in order that I may easily read them and reply to them in nmh. I would prefer for them to be organized in about seven categories, with one directory per category, so some of these direc

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ralph Corderoy
Hi Thomas, > some of these directories will have lots of files. > > Is this a bad idea? I'm mainly concerned about speed: Other than like > pick and scan on the whole folder, will anything else get very slow? Well, `scan .' will also read all entries in the folder's directory even though it knows

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ken Hornstein
>Is this a bad idea? I'm mainly concerned about speed: Other than like >pick and scan on the whole folder, will anything else get very slow? I'm >using Fast File System on OpenBSD. We've been asked about this before; unfortunately, with the current (n)mh design, it's just going to suck. A bit of

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Tom Lane
Ken Hornstein writes: >> Is this a bad idea? I'm mainly concerned about speed: Other than like >> pick and scan on the whole folder, will anything else get very slow? I'm >> using Fast File System on OpenBSD. > We've been asked about this before; unfortunately, with the current (n)mh > design, it

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ralph Corderoy
Hi Ken, > next(1)/prev(1) would NOT, because it would need to know the next > message number. One thought, next(1) could speculatively stat(2) `43' and fallback to readdir(3) if it's not there. It could do N stat()s, e.g. three, before giving up. -- Cheers, Ralph. https://plus.google.com/+Ralp

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ken Hornstein
>It may or may not matter that I use exmh not direct nmh commands. >exmh seems to cache the scan output, but then again it adds a lot of >its own overhead. AFAIK, exmh does NOT use any native nmh commands for message display or sequence manipulation. Yes, it does cache scan output, but I believe

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ken Hornstein
>> next(1)/prev(1) would NOT, because it would need to know the next >> message number. > >One thought, next(1) could speculatively stat(2) `43' and fallback to >readdir(3) if it's not there. It could do N stat()s, e.g. three, before >giving up. That is true. Buuuttt ... at the cost of added com

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Paul Vixie
i think that rcvstore would also become very slow, unless we're caching the next message number rather than discovering it every time. huge mboxes are one of the reasons i eventually expect to have to abandon Maildir. what i wish i had is a set of MH-like utilities that hid the backend storage fro

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ken Hornstein
>i think that rcvstore would also become very slow, unless we're caching >the next message number rather than discovering it every time. Yeah, and we'd have to be super careful about that. >huge mboxes are one of the reasons i eventually expect to have to >abandon Maildir. what i wish i had is a

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Paul Vixie
Ken Hornstein wrote: > ... > > Right, I think that people forget Maildir was not designed as a mail > _folder_ but instead as a mail _drop_ (a place where you store mail to > retrieve it). Yes, Dovecot uses it as a mail store, but I believe it does > a lot of caching. One of my plans for redoi

Re: [Nmh-workers] Large MH directories

2017-04-08 Thread Ken Hornstein
>wouldn't that make us pretty much like the GNU MH-like thing that's >already in existence? (should we parrot any of their API's, in hopes of >creating a future generation of portable shell-level e-mail toolery?) Well ... sort of? AFAICT, the MH part of mailutils was implemented to make MH-E work