Re: [Dovecot] mbox file too large error with deliver
On Mon, 2008-11-24 at 19:13 +0200, Gokdeniz Karadag wrote: > >> > fatal: main.cf configuration error: mailbox_size_limit is smaller > >> > than message_size_limit > >> > > >> > A limit of 2147483648 - 1 worked fine. > >> > > >> > There are two distinct bugs probably, one in postfix and one in > >> > dovecot. > > > > Postfix clearly is using signed 32bit integer here, which could be > > considered a bug. But why do you think there's a bug in Dovecot? Note > > that Dovecot never even sees this value. It's the kernel that enforces > > it, Dovecot simply reacts to a write() failure. > > The logs stating the "File too large" errors belong to dovecot's deliver > process. While I was searching for a solution, I read that deliver honors > mailbox_size_limit setting of postfix, I'm not sure if that is correct or not. It's kind of correct, but not really. What happens is: 1. Postfix reads the mailbox_size_limit setting. 2. Based on the setting Postfix calls setrlimit(RLIMIT_FSIZE) function to tell kernel the maximum file size for the process and child processes. This is similar to running "ulimit -f mailbox_size_limit" on shell before running deliver. 3. Postfix executes deliver. deliver is in no way aware of the mailbox_size_limit setting or that there is any kind of a limit. It simply tries to write to a file, which fails with EFBIG. > If postfix sets file size limit with ulimit before execing deliver, OR uses a > similar process which is out of deliver's control, then there is nothing > concerning dovecot. If someone can clarify the issue I would be happy. Yes, this is the case. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mbox file too large error with deliver
> Timo Sirainen <[EMAIL PROTECTED]> wrote: >> > Setting mailbox_size_limit to 0 made deliveries possible. a limit of >> > 20 (<2GB) also did not show the errors. Thank you ! :) >> > >> > My issue is fixed but deliver might have a bug, so I will explain >> > things, sorry for not being clear previously. >> > >> > Old mail server was 32 bit(an old version of debian) and used >> > procmail for deliveries, that setup worked with the given limit of >> > 5GB and mailbox sizes above 1GB. >> > >> > After migrating to an 64 bit server (debian etch) and dovecot's >> > deliver, the stated delivery errors appeared for mailboxes above 1 >> > GB. As I stated at the beginning of my reply, setting >> > mailbox_size_limit below 2GB made deliveries possible. I tried with >> > edge values, mailbox_size_limit=2147483648 (exactly 2GB) would give >> > the following error in postfix logs: >> > >> > fatal: main.cf configuration error: mailbox_size_limit is smaller >> > than message_size_limit >> > >> > A limit of 2147483648 - 1 worked fine. >> > >> > There are two distinct bugs probably, one in postfix and one in >> > dovecot. > > Postfix clearly is using signed 32bit integer here, which could be > considered a bug. But why do you think there's a bug in Dovecot? Note > that Dovecot never even sees this value. It's the kernel that enforces > it, Dovecot simply reacts to a write() failure. The logs stating the "File too large" errors belong to dovecot's deliver process. While I was searching for a solution, I read that deliver honors mailbox_size_limit setting of postfix, I'm not sure if that is correct or not. Note that the destination mailboxes are smaller than 2GB here, but still gets "File too large" error. The machine is 64-bit, so no fs related kernel limits on file size. I have not dived into postfix and/or dovecot internals, but my guess is , dovecot gets the mailbox size limit and behaves accordingly. If that is the case, the problem can be fixed in dovecot's deliver code. If postfix sets file size limit with ulimit before execing deliver, OR uses a similar process which is out of deliver's control, then there is nothing concerning dovecot. If someone can clarify the issue I would be happy. -- Gokdeniz Karadag
Re: [Dovecot] mbox file too large error with deliver
On Sun, 2008-11-23 at 23:55 +0200, Gokdeniz Karadag wrote: > The deliver binary cannot deliver to large mailboxes. I cannot state an exact > size limit but mbox mailboxes above 1GB seem to be affected. > > I have these in in postfix/main.cf > mailbox_command = /usr/lib/dovecot/deliver > mailbox_size_limit = 512000 So this is 5 GB. I don't know how exactly the limits work with 32 bit systems, but if you get the error with >1 GB files then my guess is that this 5 GB gets wrapped to 1 GB (5 mod 4 = 1). > user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): write() failed with > mbox > file /var/mail/dummy: File too large Kernel gives this error when it reaches the maximum file size limit. If you don't want a limit, just set mailbox_size_limit = 0. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mbox file too large error with deliver
Gokdeniz Karadag wrote: Hi, I am using dovecot on debian, with version 1.0.rc15-2etch4. Along with postfix. The deliver binary cannot deliver to large mailboxes. I cannot state an exact size limit but mbox mailboxes above 1GB seem to be affected. I have these in in postfix/main.cf mailbox_command = /usr/lib/dovecot/deliver mailbox_size_limit = 512000 The user.log file shows these (I believe if no logging is specified, deliver binary tries to log with "logger" and they go to user ?) user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): write() failed with mbox file /var/mail/dummy: File too large user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): msgid=<[EMAIL PROTECTED]>: save failed to INBOX user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): sieve runtime error: Keep: Generic Error The /var/mail/dummy file exists and is 1.9GB, so my filesystem certainly supports files >1G What may be the problem ? The limit you're probably thinking of is 2G (not 1G), if it applies to your particular OS. On a 32 bit system without large file support treating sizes as a signed integer, 2^31 bytes is 2 gigs. ~Seth
Re: [Dovecot] mbox file too large error with deliver
On 11/23/2008, Gokdeniz Karadag ([EMAIL PROTECTED]) wrote: > dovecot -n output follows : > > > log_timestamp: %Y-%m-%d %H:%M:%S Either this is an old version or you snipped the version from the top of the dovecot -n output... What version of dovecot? Also, OS/version info may help too... -- Best regards, Charles
[Dovecot] mbox file too large error with deliver
Hi, I am using dovecot on debian, with version 1.0.rc15-2etch4. Along with postfix. The deliver binary cannot deliver to large mailboxes. I cannot state an exact size limit but mbox mailboxes above 1GB seem to be affected. I have these in in postfix/main.cf mailbox_command = /usr/lib/dovecot/deliver mailbox_size_limit = 512000 The user.log file shows these (I believe if no logging is specified, deliver binary tries to log with "logger" and they go to user ?) user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): write() failed with mbox file /var/mail/dummy: File too large user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): msgid=<[EMAIL PROTECTED]>: save failed to INBOX user.log.3.gz:Nov 8 21:23:01 guvercin deliver(dummy): sieve runtime error: Keep: Generic Error The /var/mail/dummy file exists and is 1.9GB, so my filesystem certainly supports files >1G What may be the problem ? dovecot -n output follows : log_timestamp: %Y-%m-%d %H:%M:%S syslog_facility: local1 protocols: imaps pop3s ssl_cert_file(default): /etc/ssl/certs/imap-cert.pem ssl_cert_file(imap): /etc/ssl/certs/imap-cert.pem ssl_cert_file(pop3): /etc/ssl/certs/pop-cert.pem ssl_key_file(default): /etc/ssl/private/imap-key.pem ssl_key_file(imap): /etc/ssl/private/imap-key.pem ssl_key_file(pop3): /etc/ssl/private/pop-key.pem login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u:INDEX=/var/spool/dovecot/indexes/%u mbox_read_locks: fcntl dotlock mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep imap_client_workarounds(pop3): outlook-idle pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xv%08Xu pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master:
Re: [Dovecot] mbox File too large error
Nicolas KOWALSKI petole.dyndns.org> writes: > I have a rather large (~400MB) INBOX file in mbox format. When using > deliver, it complains in the logs: > > Dec 2 16:47:13 petole deliver(niko): write() failed with mbox file /home/niko/mail/INBOX: File too large This has nothing to do with dovecot: I just forgot to set a decent value to "mailbox_size_limit" in my postfix configuration. Sorry for the noise, -- Nicolas
[Dovecot] mbox File too large error
Hello, I have a rather large (~400MB) INBOX file in mbox format. When using deliver, it complains in the logs: Dec 2 16:47:13 petole deliver(niko): write() failed with mbox file /home/niko/mail/INBOX: File too large I do not see anything related in the configuration. And ulimit shows nothing special: $ ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited max nice(-e) 0 file size (blocks, -f) unlimited pending signals (-i) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size(512 bytes, -p) 8 POSIX message queues (bytes, -q) unlimited max rt priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) unlimited virtual memory (kbytes, -v) unlimited file locks (-x) unlimited This is with dovecot 1.0.8, sieve plugin 1.0.2: # dovecot -n # 1.0.8: /usr/local/etc/dovecot.conf login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login mail_location: mbox:~/mail:INBOX=~/mail/INBOX mbox_write_locks: fcntl auth default: passdb: driver: pam userdb: driver: passwd What can I do to be able to deliver to large mbox files ? Thanks, -- Nicolas
