100,000 messages, and counting.
Hi, mutt! My inbox has now reached the grand total of 100,000 messages (_exactly_ 100,000, coincidentally enough). This is partly a result of me being subscribed to too many mailing lists, and partly of me not getting around to clearing things out. (Yes, I know I could do clever things to split incoming messages amongst several mailboxes, but I don't _want_ to.) And the point of this? To start mutt from cold takes just ~12 seconds, this being on a 4 year old Athlon II machine. Just 12 seconds to read a Gig of mailbox, thread it, and display it. This is amazing performance! Thanks to all the maintainers, and keep it up! -- Alan Mackenzie (Nuremberg, Germany).
Re: 100,000 messages, and counting.
On 17Feb14 18:50 +, Alan Mackenzie wrote: > My inbox has now reached the grand total of 100,000 messages (_exactly_ > 100,000, coincidentally enough). This is partly a result of me being > subscribed to too many mailing lists, and partly of me not getting > around to clearing things out. mbox or maildir? > (Yes, I know I could do clever things to split incoming messages amongst > several mailboxes, but I don't _want_ to.) I also have just two mailboxes. inbox and trash! That's enough. For sorting/splitting there is mairix (or plain mutt searching) :) I started to use trash as my 2nd level mailbox, because startup of mutt took too long. Because deleting mails rewrote my entire mbox, I switched over to maildir, so no rewriting of the mbox file is necessary. Cheers, Bastian
Re: 100,000 messages, and counting.
Hi, Bastian. On Mon, Feb 17, 2014 at 08:42:16PM +0100, bastian-muttu...@t6l.de wrote: > On 17Feb14 18:50 +, Alan Mackenzie wrote: > > My inbox has now reached the grand total of 100,000 messages (_exactly_ > > 100,000, coincidentally enough). This is partly a result of me being > > subscribed to too many mailing lists, and partly of me not getting > > around to clearing things out. > mbox or maildir? mbox. I've never used maildir, but it strikes me that it would be slower, not counting the initial 12s loading time. > > (Yes, I know I could do clever things to split incoming messages amongst > > several mailboxes, but I don't _want_ to.) > I also have just two mailboxes. > inbox and trash! That's enough. > For sorting/splitting there is mairix (or plain mutt searching) :) "Plain", indeed! I find mutt searching very sophisticated. > I started to use trash as my 2nd level mailbox, because startup of mutt took > too > long. > Because deleting mails rewrote my entire mbox, I switched over to maildir, so > no rewriting of the > mbox file is necessary. > Cheers, > Bastian -- Alan Mackenzie (Nuremberg, Germany).
Re: 100,000 messages, and counting.
Split between mailboxes and logrotate your mailboxes too(with a custom script that checks size and does stuff to it) I only got inbox of 5000, 150MB mailbox after 10+ years in mutt, but spam is 500MB and "root" is 50k so I never open it, takes too long to open, spam takes many seconds too Maildir is really the best thing there is for large mailboxes, it only opens headers usually, no need to read the whole file when opening the mail box I got maybe 30-40 mboxes, one for each mailing list, inbox, auction sites(trading),sent,spam,root,work inbox,work inbox2 etc. Mutt will watch those that you specify are important... no need to read all mailinglists everyday(too much info) make mutt watch those mboxes which are important and alert you so you can change to them with 'c' If you rotate your mailboxes you can write a custom script that greps all rotated mailboxes too for finding your stuff .. or you could just use Maildir, mbox is not good for large mboxes If you don't use maildir you can keep upgrading servers/disks every year to make it fast enough to open your mailboxes but maildir only fetches headers, so much faster, it even caches headers, reducing load time a lot I use procmail to forward into different mailboxes, then I make mutt watch those Many inboxes, many emails in the same client, that is what mutt does well (with send-hooks/folder-hooks) Then I combine all my own emails into one inbox, but work emails/trading stuff etc. into separate inboxes I was switching to Maildir due to this problem you mention(mbox is not good for large files unless you got plenty ram/fast disks) unfortunately my inbox file must have been corrupted, so it could not be converted... due to disk crashes so now I am still with mbox 5 years later, even slower than before :) - Original Message - From: "Alan Mackenzie" To: Sent: Monday, February 17, 2014 8:58 PM Subject: Re: 100,000 messages, and counting. Hi, Bastian. On Mon, Feb 17, 2014 at 08:42:16PM +0100, bastian-muttu...@t6l.de wrote: On 17Feb14 18:50 +, Alan Mackenzie wrote: > My inbox has now reached the grand total of 100,000 messages (_exactly_ > 100,000, coincidentally enough). This is partly a result of me being > subscribed to too many mailing lists, and partly of me not getting > around to clearing things out. mbox or maildir? mbox. I've never used maildir, but it strikes me that it would be slower, not counting the initial 12s loading time. > (Yes, I know I could do clever things to split incoming messages > amongst > several mailboxes, but I don't _want_ to.) I also have just two mailboxes. inbox and trash! That's enough. For sorting/splitting there is mairix (or plain mutt searching) :) "Plain", indeed! I find mutt searching very sophisticated. I started to use trash as my 2nd level mailbox, because startup of mutt took too long. Because deleting mails rewrote my entire mbox, I switched over to maildir, so no rewriting of the mbox file is necessary. Cheers, Bastian -- Alan Mackenzie (Nuremberg, Germany).
Re: 100,000 messages, and counting.
https://svn.rlogin.dk/dotfiles/muttrc https://svn.rlogin.dk/dotfiles/procmailrc Some inspriation for many mboxes perhaps Spamassassin,procmail for every mbox,priority mails etc. Mutt watching each inbox, only alerting on new mails in those inboxes that I care about Send hooks / alternate identities etc. in muttrc (so by pressing 8 I change to 'trading' mbox, and my email changes when I reply to certain mails - ebay i.e., as they only allow you to reply from your ebay email) by pressing 'v' when I send I can chose between 20+ emails I got and it will use msmtp to send with, from those new emails after chosing Had one mbox for facebook too, but unsubscribed from that.. facebook mbox... who wants that stuff in their regular inbox Same goes for anything that has 'order','bestellung','invoice' in title, goes into my 'trading' inbox so I don't have to have it floating in my inbox will get an announcement in mutt that I can press 'c' to change to trading inbox and view the mail Procmail is also nice for creating a 'p5911' priority title for your emails if you receive important emails, then they will never be sent into spam by spamassassin if they have that in the title Spamassassin feeds on my spam once a day, 500MB and learns from it, so very few spam make it into my inbox, maybe 1 a day max Procmail is very helpful with mutt, wouldn't be without it.. - Original Message - From: "Michael Ole Olsen" To: ; "Alan Mackenzie" Sent: Monday, February 17, 2014 9:22 PM Subject: Re: 100,000 messages, and counting. Split between mailboxes and logrotate your mailboxes too(with a custom script that checks size and does stuff to it) I only got inbox of 5000, 150MB mailbox after 10+ years in mutt, but spam is 500MB and "root" is 50k so I never open it, takes too long to open, spam takes many seconds too Maildir is really the best thing there is for large mailboxes, it only opens headers usually, no need to read the whole file when opening the mail box I got maybe 30-40 mboxes, one for each mailing list, inbox, auction sites(trading),sent,spam,root,work inbox,work inbox2 etc. Mutt will watch those that you specify are important... no need to read all mailinglists everyday(too much info) make mutt watch those mboxes which are important and alert you so you can change to them with 'c' If you rotate your mailboxes you can write a custom script that greps all rotated mailboxes too for finding your stuff .. or you could just use Maildir, mbox is not good for large mboxes If you don't use maildir you can keep upgrading servers/disks every year to make it fast enough to open your mailboxes but maildir only fetches headers, so much faster, it even caches headers, reducing load time a lot I use procmail to forward into different mailboxes, then I make mutt watch those Many inboxes, many emails in the same client, that is what mutt does well (with send-hooks/folder-hooks) Then I combine all my own emails into one inbox, but work emails/trading stuff etc. into separate inboxes I was switching to Maildir due to this problem you mention(mbox is not good for large files unless you got plenty ram/fast disks) unfortunately my inbox file must have been corrupted, so it could not be converted... due to disk crashes so now I am still with mbox 5 years later, even slower than before :) - Original Message - From: "Alan Mackenzie" To: Sent: Monday, February 17, 2014 8:58 PM Subject: Re: 100,000 messages, and counting. Hi, Bastian. On Mon, Feb 17, 2014 at 08:42:16PM +0100, bastian-muttu...@t6l.de wrote: On 17Feb14 18:50 +, Alan Mackenzie wrote: > My inbox has now reached the grand total of 100,000 messages > (_exactly_ > 100,000, coincidentally enough). This is partly a result of me being > subscribed to too many mailing lists, and partly of me not getting > around to clearing things out. mbox or maildir? mbox. I've never used maildir, but it strikes me that it would be slower, not counting the initial 12s loading time. > (Yes, I know I could do clever things to split incoming messages > amongst > several mailboxes, but I don't _want_ to.) I also have just two mailboxes. inbox and trash! That's enough. For sorting/splitting there is mairix (or plain mutt searching) :) "Plain", indeed! I find mutt searching very sophisticated. I started to use trash as my 2nd level mailbox, because startup of mutt took too long. Because deleting mails rewrote my entire mbox, I switched over to maildir, so no rewriting of the mbox file is necessary. Cheers, Bastian -- Alan Mackenzie (Nuremberg, Germany).
Re: 100,000 messages, and counting.
On 17Feb2014 19:58, Alan Mackenzie wrote: > On Mon, Feb 17, 2014 at 08:42:16PM +0100, bastian-muttu...@t6l.de wrote: > > On 17Feb14 18:50 +, Alan Mackenzie wrote: > > > My inbox has now reached the grand total of 100,000 messages (_exactly_ > > > 100,000, coincidentally enough). This is partly a result of me being > > > subscribed to too many mailing lists, and partly of me not getting > > > around to clearing things out. > > > mbox or maildir? > > mbox. I've never used maildir, but it strikes me that it would be > slower, not counting the initial 12s loading time. It depends. I'm running a mix at present, and still deciding. I moved to totally maildir several years ago and recently was prusuaded by the SO to use mbox for some things. Still testing. For me with the header cache (and on an SSD), maildir is faster than mbox on open: - when an mobox is modified, mutt must still read the entire thing to update the header cache; a maildir need only be consulted for the new messages individually - as pointed out, deletes require a whole mbox rewrite - I fear race conditions when adding to the mail folder (my mail filer does not lock, and even if it did there is the thorny issue of deciding which lock mechanism other mail utilities are using); mutt is pretty good about detecting outside mbox mods, but then it refuses to update the mbox (fair enough too) and tosses my deletes Maildir is slower: - opening a maildir which does not have a header cache - rsyncing the folder to somewhere else, especially if that place uses a physical disc with headers to step etc - maildirs are more expensive in disc space terms - less efficient use of the filesyste, per message Currently I use mbox for: - all my archive folders (I have 'd' mapped to archive) - my inbox (considering switching back to maildir) - many of my other folders It is a trade off, but with the heder cache, maildirs tend to be very faster indeed. Cheers, -- Cameron Simpson Fear the government that fears your computer. - Jon Drukman
Re: 100,000 messages, and counting.
* Cameron Simpson [02-17-14 18:23]: [...] > Currently I use mbox for: > > - all my archive folders (I have 'd' mapped to archive) > > - my inbox (considering switching back to maildir) > > - many of my other folders > > It is a trade off, but with the heder cache, maildirs tend to be > very faster indeed. I have tons of mail, all in mbox. Header caching works for mbox also :^) I find mbox plenty fast from non ssd drive over nfs with header caching. I have 7 or 8 with > 1 msgs but keep my working instance of mutt alive in a tmux, similar to screen, session. But still even over nfs I seldom see a wait > 5 or 6 seconds for a *new* instance of mutt. For better local control, I frequently even run a *local* instance of mutt with ~/.muttrc adjusted to access the remote nfs mail files. -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri http://wahoo.no-ip.orgPhoto Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535@ http://linuxcounter.net
Re: 100,000 messages, and counting.
* Alan Mackenzie [2014-02-17 21:04]: > > mbox or maildir? > > mbox. I've never used maildir, but it strikes me that it would be > slower, not counting the initial 12s loading time. I'm currently at 294000 and use Maildir, but with Dovecot/IMAP in between. It's very fast, and I've only run into two minor issues: New mail isn't reported instantly (I often have to open an unread message (since last mutt restart) to trigger new mail check), and a delay/hang for a few seconds when mutt checks for new mail (usually when I 'q' a message I've just read). Used mbox before, but apart from the corruption issues with mbox, deleting a message and rewriting the mailbox would take ages and it was hopeless to manage. I also used AOX (SQL based, IMAP interface) for a long time, but there was simply too many bugs and quirks so I had to let it go. I use screen with multiple instances of Mutt, to avoid having to change/re-open different mailboxes all the time. That also has the advantage I can just screen -r from any computer to bring up the e-mail environment. Oh, and I also have Thunderbird open on a couple of machines, in case I get a really annoying HTML mail or some weird attachments Mutt won't handle. > "Plain", indeed! I find mutt searching very sophisticated. Indeed. I wish Mutt would come in a GUI version, though, so that HTML mail etc. could be displayed properly. Or that someone made a "Mutt plugin" to Thunderbird or some other GUI e-mail client, so that you could use the same keybindings as Mutt (and use "/" to quickly find an e-mail you're looking for, instead of having to search). bastian-muttu...@t6l.de wrote: > > I started to use trash as my 2nd level mailbox, because startup of mutt > > took too > > long. Now that's weird... -- Vegard Svanberg [*Takapa@IRC (EFnet)]
Re: 100,000 messages, and counting.
Am Tue, Feb 18, 2014 at 08:42:53AM +0100, Vegard Svanberg schrieb: > I use screen with multiple instances of Mutt, to avoid having to > change/re-open different mailboxes all the time. That also has the > advantage I can just screen -r from any computer to bring up the e-mail > environment. a little bit OT, but could you please tell me how you have configured screen and mutt? Are you starting mutt with mutt -R? many thanks in advance! -- :: Igor Sosa Mayor :: joseleopoldo1...@gmail.com :: :: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ :: :: jabberid: rogorido ::::
Re: 100,000 messages, and counting.
* Igor Sosa Mayor [02-18-14 07:34]: > Am Tue, Feb 18, 2014 at 08:42:53AM +0100, Vegard Svanberg schrieb: > > I use screen with multiple instances of Mutt, to avoid having to > > change/re-open different mailboxes all the time. That also has the > > advantage I can just screen -r from any computer to bring up the e-mail > > environment. > > a little bit OT, but could you please tell me how you have configured > screen and mutt? Are you starting mutt with mutt -R? I use tmux instead of screen, but no need or personal preference to start mutt in "read-only" mode. I am the only user of mutt on my system so I don't worry about another session changes happening while I am using mutt. The only thing I notice is when I enter a tmux/screen session from a *live* instance of mutt, mutt sometimes complains of changes from the separate instance accessing the same email or mbox file. The same tmux/screen session of mutt accessed from different stations will display the same, they are the same instance of mutt. -- (paka)Patrick Shanahan Plainfield, Indiana, USA @ptilopteri http://en.opensuse.orgopenSUSE Community Memberfacebook/ptilopteri http://wahoo.no-ip.orgPhoto Album: http://wahoo.no-ip.org/gallery2 Registered Linux User #207535@ http://linuxcounter.net
Re: 100,000 messages, and counting.
* Igor Sosa Mayor [2014-02-18 13:33]: > a little bit OT, but could you please tell me how you have configured > screen and mutt? Are you starting mutt with mutt -R? No configuration, really, I just start screen and start each mutt instance in a new window. No concurrency problems as each mutt handles a different folder (and, as mentioned, I use IMAP with Dovecot anyway). -- Vegard Svanberg [*Takapa@IRC (EFnet)]
Re: 100,000 messages, and counting.
Am Tue, Feb 18, 2014 at 07:52:07AM -0500, Patrick Shanahan schrieb: > I use tmux instead of screen, but no need or personal preference to start > mutt in "read-only" mode. I am the only user of mutt on my system so I > don't worry about another session changes happening while I am using mutt. aha... interesting. I'm also the only user and I also use tmux... I will give it a try. Thanks! -- :: Igor Sosa Mayor :: joseleopoldo1...@gmail.com :: :: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ :: :: jabberid: rogorido ::::
Re: 100,000 messages, and counting.
Am Tue, Feb 18, 2014 at 03:00:19PM +0100, Vegard Svanberg schrieb: > No configuration, really, I just start screen and start each mutt > instance in a new window. No concurrency problems as each mutt handles a > different folder (and, as mentioned, I use IMAP with Dovecot anyway). thanks. I never thought this is so easy... -- :: Igor Sosa Mayor :: joseleopoldo1...@gmail.com :: :: GnuPG: 0x1C1E2890 :: http://www.gnupg.org/ :: :: jabberid: rogorido ::::
Re: 100,000 messages, and counting., procmail mailinglist to new inbox recipe
Most importantly, there is a nice procmail recipe in that procmailrc that
creates list inboxes automatically
as soon as you sign up for a mailing list, procmail will create it as a new
inbox for you automatically... pretty cool
:0:
* ^((List-Id|X-(Mailing-)?List):(.*[<]\/[^>]*))
{
LISTID=$MATCH
:0:
* LISTID ?? ^\/[^@\.]*
lists/$MATCH
}one inbox for each mailig list, switch to them with 'c?'
- Original Message -
From: "Michael Ole Olsen"
To: "Michael Ole Olsen" ; ; "Alan
Mackenzie"
Sent: Monday, February 17, 2014 9:43 PM
Subject: Re: 100,000 messages, and counting.
https://svn.rlogin.dk/dotfiles/muttrc
https://svn.rlogin.dk/dotfiles/procmailrc
Some inspriation for many mboxes perhaps
Spamassassin,procmail for every mbox,priority mails etc.
Mutt watching each inbox, only alerting on new mails in those inboxes that
I care about
Send hooks / alternate identities etc. in muttrc (so by pressing 8 I
change to 'trading' mbox, and my email changes when I reply to certain
mails - ebay i.e., as they only allow you to reply from your ebay email)
by pressing 'v' when I send I can chose between 20+ emails I got and it
will use msmtp to send with, from those new emails after chosing
Had one mbox for facebook too, but unsubscribed from that..
facebook mbox... who wants that stuff in their regular inbox
Same goes for anything that has 'order','bestellung','invoice' in title,
goes into my 'trading' inbox so I don't have to have it floating in my
inbox
will get an announcement in mutt that I can press 'c' to change to trading
inbox and view the mail
Procmail is also nice for creating a 'p5911' priority title for your
emails
if you receive important emails, then they will never be sent into spam by
spamassassin if they have that in the title
Spamassassin feeds on my spam once a day, 500MB and learns from it, so
very few spam make it into my inbox, maybe 1 a day max
Procmail is very helpful with mutt, wouldn't be without it..
- Original Message -
From: "Michael Ole Olsen"
To: ; "Alan Mackenzie"
Sent: Monday, February 17, 2014 9:22 PM
Subject: Re: 100,000 messages, and counting.
Split between mailboxes and logrotate your mailboxes too(with a custom
script that checks size and does stuff to it)
I only got inbox of 5000, 150MB mailbox after 10+ years in mutt, but spam
is 500MB and "root" is 50k so I never open it, takes too long to open,
spam takes many seconds too
Maildir is really the best thing there is for large mailboxes, it only
opens headers usually, no need to read the whole file when opening the
mail box
I got maybe 30-40 mboxes, one for each mailing list, inbox, auction
sites(trading),sent,spam,root,work inbox,work inbox2 etc.
Mutt will watch those that you specify are important... no need to read
all mailinglists everyday(too much info)
make mutt watch those mboxes which are important and alert you so you can
change to them with 'c'
If you rotate your mailboxes you can write a custom script that greps all
rotated mailboxes too for finding your stuff
.. or you could just use Maildir, mbox is not good for large mboxes
If you don't use maildir you can keep upgrading servers/disks every year
to make it fast enough to open your mailboxes
but maildir only fetches headers, so much faster, it even caches headers,
reducing load time a lot
I use procmail to forward into different mailboxes, then I make mutt
watch those
Many inboxes, many emails in the same client, that is what mutt does well
(with send-hooks/folder-hooks)
Then I combine all my own emails into one inbox, but work emails/trading
stuff etc. into separate inboxes
I was switching to Maildir due to this problem you mention(mbox is not
good for large files unless you got plenty ram/fast disks)
unfortunately my inbox file must have been corrupted, so it could not be
converted... due to disk crashes
so now I am still with mbox 5 years later, even slower than before :)
- Original Message -
From: "Alan Mackenzie"
To:
Sent: Monday, February 17, 2014 8:58 PM
Subject: Re: 100,000 messages, and counting.
Hi, Bastian.
On Mon, Feb 17, 2014 at 08:42:16PM +0100, bastian-muttu...@t6l.de wrote:
On 17Feb14 18:50 +, Alan Mackenzie wrote:
> My inbox has now reached the grand total of 100,000 messages
> (_exactly_
> 100,000, coincidentally enough). This is partly a result of me
> being
> subscribed to too many mailing lists, and partly of me not getting
> around to clearing things out.
mbox or maildir?
mbox. I've never used maildir, but it strikes me that it would be
slower, not counting the initial 12s loading time.
> (Yes, I know I could do clever things to split incoming messages
> amongst
> several mailboxes, but I d
