Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-06-01 Thread Scott Silva
on 5-30-2009 2:25 PM Roy Sigurd Karlsbakk spake the following:
 On 30. mai. 2009, at 00.03, Scott Silva wrote:
 
 # e2fsck -D -f /dev/hdXY

 Note: This should work with both ext2 and ext3 filesystems. Depending
 on the
 size of your filesystem, this could take a long time. Perhaps you
 should go
 get some coffee...
 
 
 AFAIK an updatedb (as in locate/slocate), or perhaps just a find /
 should access all directories and make and index in each of them. That
 way you won't need to umount the fs.
 
If you look at the quoted text, an fsck is required on pre-existing
directories. They don't have the proper extents or index structures if created
before directory_index is turned on. No other program is going to update the
directory indexes. You can read those directories as many times as you want,
it won't change it.




signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-30 Thread Roy Sigurd Karlsbakk

On 30. mai. 2009, at 00.03, Scott Silva wrote:


# e2fsck -D -f /dev/hdXY

Note: This should work with both ext2 and ext3 filesystems.  
Depending on the
size of your filesystem, this could take a long time. Perhaps you  
should go

get some coffee...



AFAIK an updatedb (as in locate/slocate), or perhaps just a find /  
should access all directories and make and index in each of them. That  
way you won't need to umount the fs.


roy
--
Roy Sigurd Karlsbakk
(+47) 97542685 / 98013356
r...@karlsbakk.net
http://blogg.karlsbakk.net/
--
I all pedagogikk er det essensielt at pensum presenteres  
intelligibelt. Det er et elementært imperativ for alle pedagoger å  
unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de  
fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.




Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-30 Thread tomas
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Sat, May 30, 2009 at 11:25:58PM +0200, Roy Sigurd Karlsbakk wrote:
 On 30. mai. 2009, at 00.03, Scott Silva wrote:
[...]
 AFAIK an updatedb (as in locate/slocate) [...]

This won't help with the original poster's problem (which is in helping
the kernel to cope with huge directories). The OP's problem is at the
kernel level, locate is an application (albeit a nice one).

Regards
- -- tomás
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFKIgD1Bcgs9XrR2kYRAutsAJ46nRL6gV1qCejXvL2jORYK0jGcWgCfeTsk
iox6dAFCXBRbuUpuMQQL4r8=
=EsWv
-END PGP SIGNATURE-


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-29 Thread Bernd Petrovitsch
On Thu, 2009-05-28 at 21:28 -0500, Kyle Wheeler wrote:
 On Friday, May 29 at 09:46 AM, quoth Curtis Maloney:
  This is certainly one advantage dbox and maildir have -- not being 
  limited to the FS file size limit per folder.
 
 That's not *entirely* accurate. Certainly no single message can exceed 
 the 2GB limit even with maildir, and the other issue that begins to 
 come up is the impact/effect of large numbers of files. Depending on 
 the filesystem (I'm assuming ext2?), there's probably a hard limit on 

FC4 had ext3 (unless my memory is totally mistaken).

 the number of files per directory, and almost certainly there's a big 

Subdirectories, yes (because the link count in the inode is of quite
finite size).
But there never was TTBOMK a limit on the number of files (!=
Directories) in ext2 (except the trivial one: The directory is as large
as the largest file. But that applies probably to all filesystems -
though more recent ones allow for much larger files).
Let alone ext3.

 performance penalty for that many files. To get good performance with 
 Maildir and really large folders, you need a filesystem that can 
 handle large numbersof files. Ext3 has directory hashing, ReiserFS is 

Make sure you have the dir_index option set on that filesystem (which
is probably set per default anyways these days. Otherwise you can change
it on the fly with `tune2fs`).
The back of my head suggests that one has to recreate the directory
after changing that option (read: `mkdir new; mv old/* new; rmdir old;
mv new old`. Solving the command line too long problem is left to the
reader;-).

 good... I believe XFS and several others have tackled the problem as 
 well (I don't know about FFS).
 
 That said... eGADS - a real life FC4 in the wild?!?! According to 
 fedoraproject.org:
 
 For 20030101-20050607 there are a potential 863 CVE named
 vulnerabilities that could have affected FC4 packages. 759 (88%)
 of those are fixed because FC4 includes an upstream version that
 includes a fix, 10 (1%) are still outstanding, and 94 (11%) are
 fixed with a backported patch.
 
 That would make me a little nervous that's just the issues over 
 the course of two years, ending in 2005 (FOUR years ago).

I don't know what/how others do but many servers are not really in the
wild but behind more recent firewalls and/or loadbalancers and/or
similar equipment (like running database servers behind webservers).

And running FC4 doesn't mean that that certain/some/several/many
packages aren't replaced by more recent ones - for whatever reason
(security, performance, newer version, newer drivers in the
kernel, ...).

For a pristine FC4 with lots of services directly at the Internet: I
totally agree with you.

Apart from the basic question if one shouldn't better run a more
conservative distribution (like RHEL) in the first place where the
support cycle is much longer.

Bernd
-- 
Firmix Software GmbH   http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
  Embedded Linux Development and Services




Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-29 Thread Stefan Klatt
 
Hi Scott,

 Are you saying that you can gzip a mailbox, and Dovecot will then 
 somehow uncompress that and send the data back out to the Desktop 
 Client as a normal looking IMAP box?
 
 I have never heard of this tactic, and I am not finding any references 
 to it on google, though I am probably just not understanding it and 
 looking at the wrong search terms.
I use this to save/backup older emails. 
zipped mbox = Fast, small, only one file to handle, not changeable = optimal
for backup

Stefan




Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-29 Thread Eduardo M KALINOWSKI

On Qui, 28 Mai 2009, Curtis Maloney wrote:

But what sort of user puts 4GB of mail into a _single_ folder??


Those that never delete the emails nor move them into folders.  
Everything they ever received is in the INBOX.


And yes, this happens a lot.


--
Eduardo M KALINOWSKI
edua...@kalinowski.com.br



Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-29 Thread Scott Silva
on 5-28-2009 4:46 PM Curtis Maloney spake the following:
 robforrest wrote:
 What is happening is that as users log in (via thunderbird), they are
 shown
 the contents of their mailboxes.  However, when they try and access a
 given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several
 users
 have inboxes in excess of 4GB.
 
 FC4 is quite old... can the kernel and filesystem you're using properly
 handle files over 4GB?
 
 This is certainly one advantage dbox and maildir have -- not being
 limited to the FS file size limit per folder.
 
 But what sort of user puts 4GB of mail into a _single_ folder??
 
(l)users!

;-P





signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-29 Thread Scott Silva
on 5-29-2009 1:42 AM Bernd Petrovitsch spake the following:
 On Thu, 2009-05-28 at 21:28 -0500, Kyle Wheeler wrote:
 On Friday, May 29 at 09:46 AM, quoth Curtis Maloney:
 This is certainly one advantage dbox and maildir have -- not being 
 limited to the FS file size limit per folder.
 That's not *entirely* accurate. Certainly no single message can exceed 
 the 2GB limit even with maildir, and the other issue that begins to 
 come up is the impact/effect of large numbers of files. Depending on 
 the filesystem (I'm assuming ext2?), there's probably a hard limit on 
 
 FC4 had ext3 (unless my memory is totally mistaken).
 
 the number of files per directory, and almost certainly there's a big 
 
 Subdirectories, yes (because the link count in the inode is of quite
 finite size).
 But there never was TTBOMK a limit on the number of files (!=
 Directories) in ext2 (except the trivial one: The directory is as large
 as the largest file. But that applies probably to all filesystems -
 though more recent ones allow for much larger files).
 Let alone ext3.
 
 performance penalty for that many files. To get good performance with 
 Maildir and really large folders, you need a filesystem that can 
 handle large numbersof files. Ext3 has directory hashing, ReiserFS is 
 
 Make sure you have the dir_index option set on that filesystem (which
 is probably set per default anyways these days. Otherwise you can change
 it on the fly with `tune2fs`).

I don't think you can do it on the fly. Any directories created before the
option was set will not have directory indexing until a fsck is done.

quote

 Using Directory Indexing

This feature improves file access in large directories or directories
containing many files by using hashed binary trees to store the directory
information. It's perfectly safe to use, and it provides a fairly substantial
improvement in most cases; so it's a good idea to enable it:

# tune2fs -O dir_index /dev/hdXY

This will only take effect with directories created on that filesystem after
tune2fs is run. In order to apply this to currently existing directories, we
must run the e2fsck utility to optimize and reindex the directories on the
filesystem:

# e2fsck -D -f /dev/hdXY

Note: This should work with both ext2 and ext3 filesystems. Depending on the
size of your filesystem, this could take a long time. Perhaps you should go
get some coffee...

/quote




 The back of my head suggests that one has to recreate the directory
 after changing that option (read: `mkdir new; mv old/* new; rmdir old;
 mv new old`. Solving the command line too long problem is left to the
 reader;-).
 




signature.asc
Description: OpenPGP digital signature


[Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread robforrest

Hi all,

I'm new here and would very much appreciate any help you can give me.

We are running a rather outdated mail server that until recently has been
running beautifully.  Under the pretense of it is ain't broke, don't fix
it it hasn't been updated so is running Fedora Core 4 and dovecot v0.99.14.

What is happening is that as users log in (via thunderbird), they are shown
the contents of their mailboxes.  However, when they try and access a given
email, they get no response even if they wait upwards of 10 minutes.  I
believe that the problem has arisen from their huge inboxes, several users
have inboxes in excess of 4GB.

Do any of you know how I could split these large mbox files down in to
manageable chunks of 1GB?

Please be aware that I only have remote access to the server over a modest
internet connection.

Thanks again for any suggestions.

Rob
-- 
View this message in context: 
http://www.nabble.com/Users-with-large-%284GB%29-inboxes-crippling-dovecot-tp23758036p23758036.html
Sent from the Dovecot mailing list archive at Nabble.com.



Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Frank Elsner
On Thu, 28 May 2009 02:07:01 -0700 (PDT) robforrest wrote:
 
 Hi all,
 
 I'm new here and would very much appreciate any help you can give me.
 
 We are running a rather outdated mail server that until recently has been
 running beautifully.  Under the pretense of it is ain't broke, don't fix
 it it hasn't been updated so is running Fedora Core 4 and dovecot v0.99.14.
 
 What is happening is that as users log in (via thunderbird), they are shown
 the contents of their mailboxes.  However, when they try and access a given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several users
 have inboxes in excess of 4GB.
 
 Do any of you know how I could split these large mbox files down in to
 manageable chunks of 1GB?

I don't know. I know that maildir is far better for this situation.
You should 
- Upgrade dovecot
- Convert to maildir format

For the conversion to maildir refer to the discussion on this list.


--Frank Elsner


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Silva
on 5-28-2009 2:07 AM robforrest spake the following:
 Hi all,
 
 I'm new here and would very much appreciate any help you can give me.
 
 We are running a rather outdated mail server that until recently has been
 running beautifully.  Under the pretense of it is ain't broke, don't fix
 it it hasn't been updated so is running Fedora Core 4 and dovecot v0.99.14.
 
It's broke, you need to fix it. This system is an accident waiting to happen
(or a bot waiting to be owned).

 What is happening is that as users log in (via thunderbird), they are shown
 the contents of their mailboxes.  However, when they try and access a given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several users
 have inboxes in excess of 4GB.
 
 Do any of you know how I could split these large mbox files down in to
 manageable chunks of 1GB?
 
 Please be aware that I only have remote access to the server over a modest
 internet connection.
 
 Thanks again for any suggestions.
 
 Rob
Why not try a current version of dovecot. 1.0 would help a lot, and 1.1 would
be even better. 0.99.14 has been improved so much, you are just not going to
fix it easily.

You can get more current rpms of dovecot from atrpms.net, but you might have
to either try a RHEL4 version (loosely based on Core 3) or rebuild a src rpm
on a core 4 machine. That is one of the problems with running core on a
server. If you let it get too far out of service, you have more trouble
updating it.




signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Timo Sirainen
On Thu, 2009-05-28 at 02:07 -0700, robforrest wrote:
 What is happening is that as users log in (via thunderbird), they are shown
 the contents of their mailboxes.  However, when they try and access a given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several users
 have inboxes in excess of 4GB.

mbox_very_dirty_syncs = yes would probably make these mailboxes usable
without having to split anything or convert to Maildir. That requires
Dovecot v1.0+ of course.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Silva
on 5-28-2009 9:36 AM Timo Sirainen spake the following:
 On Thu, 2009-05-28 at 02:07 -0700, robforrest wrote:
 What is happening is that as users log in (via thunderbird), they are shown
 the contents of their mailboxes.  However, when they try and access a given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several users
 have inboxes in excess of 4GB.
 
 mbox_very_dirty_syncs = yes would probably make these mailboxes usable
 without having to split anything or convert to Maildir. That requires
 Dovecot v1.0+ of course.
 
The indexing improvements in 1.0 would probably help a lot also. I have
several large mboxes and also some gzipped log storage boxes and 1.1 is fairly
speedy with them. A full re-index in thunderbird takes right at a minute with
a 6gb gzipped box I just tested.



signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Haneda

On May 28, 2009, at 10:39 AM, Scott Silva wrote:


on 5-28-2009 9:36 AM Timo Sirainen spake the following:

On Thu, 2009-05-28 at 02:07 -0700, robforrest wrote:
What is happening is that as users log in (via thunderbird), they  
are shown
the contents of their mailboxes.  However, when they try and  
access a given
email, they get no response even if they wait upwards of 10  
minutes.  I
believe that the problem has arisen from their huge inboxes,  
several users

have inboxes in excess of 4GB.


mbox_very_dirty_syncs = yes would probably make these mailboxes  
usable

without having to split anything or convert to Maildir. That requires
Dovecot v1.0+ of course.

The indexing improvements in 1.0 would probably help a lot also. I  
have
several large mboxes and also some gzipped log storage boxes and 1.1  
is fairly
speedy with them. A full re-index in thunderbird takes right at a  
minute with

a 6gb gzipped box I just tested.



Can you state what the size of your several large mboxes is?  I have  
been curious about this.  My current email server forces me to manage  
my IMAP boxes not based on size, though I believe there is a less than  
2GB limit, but on message count.  Too many messages kills me, the  
storage is one file for each mailbox.


As I am migrating to Dovecot, it would be nice to know what I am in  
for, and if I can simply stop managing this?  My users are going to  
use the server like it is gmail, and never file a darn thing.


One thing I wanted to do, was look at the inbox, and if it is over  
xMB's, rename it to inbox.date which will force the user to deal with  
it.  However, if there is no need to do this, and Dovecot can handle  
it, then I would prefer to skip the intrusion on my users.


What happens in Dovecot if one inbox is massive and in fact hurting  
performance.  Does that performance hit trickle down to all users, or  
just that one user?


Thank you, and looking forward to being 100% Doevcot here in a few  
weeks.

--
Scott * If you contact me off list replace talklists@ with scott@ *



Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Silva
on 5-28-2009 12:24 PM Scott Haneda spake the following:
 On May 28, 2009, at 10:39 AM, Scott Silva wrote:
 
 on 5-28-2009 9:36 AM Timo Sirainen spake the following:
 On Thu, 2009-05-28 at 02:07 -0700, robforrest wrote:
 What is happening is that as users log in (via thunderbird), they
 are shown
 the contents of their mailboxes.  However, when they try and access
 a given
 email, they get no response even if they wait upwards of 10 minutes.  I
 believe that the problem has arisen from their huge inboxes, several
 users
 have inboxes in excess of 4GB.

 mbox_very_dirty_syncs = yes would probably make these mailboxes usable
 without having to split anything or convert to Maildir. That requires
 Dovecot v1.0+ of course.

 The indexing improvements in 1.0 would probably help a lot also. I have
 several large mboxes and also some gzipped log storage boxes and 1.1
 is fairly
 speedy with them. A full re-index in thunderbird takes right at a
 minute with
 a 6gb gzipped box I just tested.
 
 
 Can you state what the size of your several large mboxes is?  I have
 been curious about this.  My current email server forces me to manage my
 IMAP boxes not based on size, though I believe there is a less than 2GB
 limit, but on message count.  Too many messages kills me, the storage is
 one file for each mailbox.
 
 As I am migrating to Dovecot, it would be nice to know what I am in for,
 and if I can simply stop managing this?  My users are going to use the
 server like it is gmail, and never file a darn thing.
 
 One thing I wanted to do, was look at the inbox, and if it is over
 xMB's, rename it to inbox.date which will force the user to deal with
 it.  However, if there is no need to do this, and Dovecot can handle it,
 then I would prefer to skip the intrusion on my users.
 
 What happens in Dovecot if one inbox is massive and in fact hurting
 performance.  Does that performance hit trickle down to all users, or
 just that one user?
 
 Thank you, and looking forward to being 100% Doevcot here in a few weeks.
My largest is about 5+ GB gzipped (I thought it was 6gb until I just checked),
but I am on a 64 bit server and don't have a 2 gb filesize limit. I have
several users with 2 to 3 GB inboxes on IMAP, and don't get any complaints. I
have been considering going to maildir, and scripting some stuff like moving
messages older than 30 days out of the inbox, and also purging any messages
marked as deleted, and also older stuff in the trash. Maildir is much easier
with this since it is one file per message instead of one file per folder.

If one user does manage to really hose a box, the re-indexing that dovecot can
slow the system for a little while, but it clears up pretty fast. Only rarely
have I had to intervene and kill a process.

The one place where dovecot might be less than stellar is with pop3 access
from outlook if they leave mail on the server. It can get confused sometimes
and re-download everything, but this is Outlook's problem, not dovecot's.



signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Haneda

On May 28, 2009, at 1:12 PM, Scott Silva wrote:

Can you state what the size of your several large mboxes is?  I  
have
been curious about this.  My current email server forces me to  
manage my
IMAP boxes not based on size, though I believe there is a less than  
2GB
limit, but on message count.  Too many messages kills me, the  
storage is

one file for each mailbox.

As I am migrating to Dovecot, it would be nice to know what I am in  
for,
and if I can simply stop managing this?  My users are going to use  
the

server like it is gmail, and never file a darn thing.

One thing I wanted to do, was look at the inbox, and if it is over
xMB's, rename it to inbox.date which will force the user to deal with
it.  However, if there is no need to do this, and Dovecot can  
handle it,

then I would prefer to skip the intrusion on my users.

What happens in Dovecot if one inbox is massive and in fact hurting
performance.  Does that performance hit trickle down to all users, or
just that one user?

Thank you, and looking forward to being 100% Doevcot here in a few  
weeks.


Thanks for the reply..

My largest is about 5+ GB gzipped (I thought it was 6gb until I just  
checked),
but I am on a 64 bit server and don't have a 2 gb filesize limit. I  
have
several users with 2 to 3 GB inboxes on IMAP, and don't get any  
complaints. I


Are you saying that you can gzip a mailbox, and Dovecot will then  
somehow uncompress that and send the data back out to the Desktop  
Client as a normal looking IMAP box?


I have never heard of this tactic, and I am not finding any references  
to it on google, though I am probably just not understanding it and  
looking at the wrong search terms.


have been considering going to maildir, and scripting some stuff  
like moving
messages older than 30 days out of the inbox, and also purging any  
messages
marked as deleted, and also older stuff in the trash. Maildir is  
much easier
with this since it is one file per message instead of one file per  
folder.


If one user does manage to really hose a box, the re-indexing that  
dovecot can
slow the system for a little while, but it clears up pretty fast.  
Only rarely

have I had to intervene and kill a process.

The one place where dovecot might be less than stellar is with pop3  
access
from outlook if they leave mail on the server. It can get confused  
sometimes
and re-download everything, but this is Outlook's problem, not  
dovecot's.



I used to keep an eye on these clients, and set a never leave mail on  
server setting on my old server.  Not sure how to do this in Dovecot,  
though if I have my way, everyone is going to be on IMAP, though they  
are free to use it like POP if they want.


Thanks again for your reply, impressive stats to me, considering the  
stats of where I am coming from.

--
Scott * If you contact me off list replace talklists@ with scott@ *



Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Timo Sirainen
On Thu, 2009-05-28 at 13:35 -0700, Scott Haneda wrote:
  My largest is about 5+ GB gzipped (I thought it was 6gb until I just  
  checked),
  but I am on a 64 bit server and don't have a 2 gb filesize limit. I  
  have
  several users with 2 to 3 GB inboxes on IMAP, and don't get any  
  complaints. I
 
 Are you saying that you can gzip a mailbox, and Dovecot will then  
 somehow uncompress that and send the data back out to the Desktop  
 Client as a normal looking IMAP box?
 
 I have never heard of this tactic, and I am not finding any references  
 to it on google, though I am probably just not understanding it and  
 looking at the wrong search terms.

http://wiki.dovecot.org/Plugins/Zlib



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Matt
On Thu, May 28, 2009 at 3:47 PM, Timo Sirainen t...@iki.fi wrote:

 On Thu, 2009-05-28 at 13:35 -0700, Scott Haneda wrote:
   My largest is about 5+ GB gzipped (I thought it was 6gb until I just
   checked),
   but I am on a 64 bit server and don't have a 2 gb filesize limit. I
   have
   several users with 2 to 3 GB inboxes on IMAP, and don't get any
   complaints. I
 
  Are you saying that you can gzip a mailbox, and Dovecot will then
  somehow uncompress that and send the data back out to the Desktop
  Client as a normal looking IMAP box?
 
  I have never heard of this tactic, and I am not finding any references
  to it on google, though I am probably just not understanding it and
  looking at the wrong search terms.

 http://wiki.dovecot.org/Plugins/Zlib


What advantage is compression rather then disk space on server?

Matt


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Timo Sirainen
On Thu, 2009-05-28 at 16:00 -0500, Matt wrote:
  http://wiki.dovecot.org/Plugins/Zlib
 
 
 What advantage is compression rather then disk space on server?

For mbox, there's no other advantage than less disk space usage. Instead
there are disadvantages, because the mbox is then read-only.

For Maildir, it might also make disk I/O usage slightly smaller but
probably not much unless you have large mails.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Scott Silva
snip
 
 
 I used to keep an eye on these clients, and set a never leave mail on
 server setting on my old server.  Not sure how to do this in Dovecot,
 though if I have my way, everyone is going to be on IMAP, though they
 are free to use it like POP if they want.
 
 Thanks again for your reply, impressive stats to me, considering the
 stats of where I am coming from.

With many kudos to Timo, dovecot gets better with each release. When I went
from UWImap to 0.99 the improvements were great. From 0.99 to 1.0, while not
as dramatic, was even better. 1.0 to 1.1 even better. The improvements aren't
as big from release to release, but they are noticeable. I would never go back
to UWImap for any reason.



signature.asc
Description: OpenPGP digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Curtis Maloney

robforrest wrote:

What is happening is that as users log in (via thunderbird), they are shown
the contents of their mailboxes.  However, when they try and access a given
email, they get no response even if they wait upwards of 10 minutes.  I
believe that the problem has arisen from their huge inboxes, several users
have inboxes in excess of 4GB.


FC4 is quite old... can the kernel and filesystem you're using properly 
handle files over 4GB?


This is certainly one advantage dbox and maildir have -- not being limited 
to the FS file size limit per folder.


But what sort of user puts 4GB of mail into a _single_ folder??

--
Curtis Maloney
cmalo...@cardgate.net



Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread dovecot

At 07:46 PM 5/28/2009, Curtis Maloney wrote:
FC4 is quite old... can the kernel and filesystem you're using 
properly handle files over 4GB?


This is certainly one advantage dbox and maildir have -- not being 
limited to the FS file size limit per folder.


But what sort of user puts 4GB of mail into a _single_ folder??


One that's been using the mail server since FC4? :-)

Rick




Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Kyle Wheeler

On Friday, May 29 at 09:46 AM, quoth Curtis Maloney:
This is certainly one advantage dbox and maildir have -- not being 
limited to the FS file size limit per folder.


That's not *entirely* accurate. Certainly no single message can exceed 
the 2GB limit even with maildir, and the other issue that begins to 
come up is the impact/effect of large numbers of files. Depending on 
the filesystem (I'm assuming ext2?), there's probably a hard limit on 
the number of files per directory, and almost certainly there's a big 
performance penalty for that many files. To get good performance with 
Maildir and really large folders, you need a filesystem that can 
handle large numbersof files. Ext3 has directory hashing, ReiserFS is 
good... I believe XFS and several others have tackled the problem as 
well (I don't know about FFS).


That said... eGADS - a real life FC4 in the wild?!?! According to 
fedoraproject.org:


   For 20030101-20050607 there are a potential 863 CVE named
   vulnerabilities that could have affected FC4 packages. 759 (88%)
   of those are fixed because FC4 includes an upstream version that
   includes a fix, 10 (1%) are still outstanding, and 94 (11%) are
   fixed with a backported patch.

That would make me a little nervous that's just the issues over 
the course of two years, ending in 2005 (FOUR years ago).


~Kyle
--
I would therefore like to posit that computing’s central challenge, 
how not to make a mess of it, has not yet been met.

   -- Edsger Dijkstra


signature.asc
Description: Digital signature


Re: [Dovecot] Users with large (4GB) inboxes crippling dovecot

2009-05-28 Thread Timo Sirainen


On May 28, 2009, at 10:28 PM, Kyle Wheeler wrote:


On Friday, May 29 at 09:46 AM, quoth Curtis Maloney:
This is certainly one advantage dbox and maildir have -- not being  
limited to the FS file size limit per folder.


That's not *entirely* accurate. Certainly no single message can  
exceed the 2GB limit even with maildir,


Dovecot is built with large file support, so it can handle 4GB mails  
and 4GB mboxes just fine.