[Dovecot] Dovecot Erros in Logs
Different Error today: Aug 31 08:01:05 IMAP(user-folder): Panic: pool_data_stack_realloc(): stack frame changed Aug 31 08:01:05 IMAP(user-folder): Error: Raw backtrace: imap [0x49d0c8] - imap [0x49db63] - imap [0x49d386] - imap [0x4a79bb] - imap [0x49b3f5] - imap(buffer_write+0x72) [0x49b732] - imap(buffer_append_c+0x18) [0x49b848] - imap(mail_namespace_get_vname+0x4b) [0x46374b] - imap(mailbox_list_subscriptions_fill+0xf9) [0x42d559] - imap(maildir_list_iter_init+0x592) [0x42d202] - imap [0x41c953] - imap(cmd_list_full+0x261) [0x41cd01] - imap(cmd_lsub+0xe) [0x41d27e] - imap [0x4203f3] - imap [0x4203b8] - imap(client_handle_input+0x161) [0x420681] - imap(client_input+0x5f) [0x420f2f] - imap(io_loop_handler_run+0x109) [0x4a5949] - imap(io_loop_run+0x28) [0x4a4c98] - imap(main+0x712) [0x428b32] - /lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x3170c1c40b] - imap [0x41940a] It seems to only happen to this one user. I cannot see anything odd about this user's folder. Dave
[Dovecot] Dovecot Erros in Logs
I got lots of errors that look like this: Error: write(dnotify pipe) failed: Bad file descriptor I am running dovecot-1.2.4-0_99 on RHEL4 dovecot -n: # 1.2.4: /etc/dovecot.conf # OS: Linux 2.6.9-89.0.3.ELsmp x86_64 Red Hat Enterprise Linux AS release 4 (Nahant Update 8) ext3 log_path: /var/log/dovecot info_log_path: /var/log/dovecot-info login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_process_per_connection: no login_process_size: 1024 login_processes_count: 6 login_max_processes_count: 1024 login_max_connections: 1024 max_mail_processes: 5 verbose_proctitle: yes first_valid_uid: 50 mail_uid: 93 mail_gid: 12 mail_location: maildir:/var/spool/maildirs/%d/%n/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_process_size: 1024 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmas...@example.com auth default: cache_ttl: 43200 cache_negative_ttl: 0 verbose: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf plugin: quota: maildir Any suggestions? Dave
Re: [Dovecot] Dovecot Erros in Logs
I had heard/read something online about using inotify instead of dnotify. Any idea what that is about? I have not heard any complaints, nor do I know why this happens. I just notice this and exactly this in the logs. Dave Quoting Timo Sirainen t...@iki.fi: On Fri, 2009-08-28 at 11:47 -0400, David Cunningham wrote: I got lots of errors that look like this: Error: write(dnotify pipe) failed: Bad file descriptor Hmm. Are they all exactly this, or also something else? I don't really see in the code how that could happen. Anyway you could disable using dnotify with configure --with-notify=none
Re: [Dovecot] Auth Issues - Urgent - Help!
No one else with opinions on this? Dave Quoting David Cunningham [EMAIL PROTECTED]: Yes, i telnet to port 143 and enter everything manually. Dave Quoting Charles Marcus [EMAIL PROTECTED]: On 11/19/2008 10:17 PM, David Cunningham wrote: Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled, new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered. Is this something wrong with my configuraiton, or the auth code itself? Maybe it is the mail client doing the caching... have you tested this on the command line? -- Best regards, Charles
Re: [Dovecot] Auth Issues - Urgent - Help!
I think the last thing you say is exactly what is happening to me. I think the user is updating the password, but a slight delay in my LDAP replication is causing them to try the new password before it is actually the new password. Yes, I was refering to auth_cache_negative_ttl=0. I didn't realize that was user not found only. Is there any way to force the cache to check the password for anything that was not previously cached as being the correct password? Dave Quoting Timo Sirainen [EMAIL PROTECTED]: On Wed, 2008-11-19 at 22:17 -0500, David Cunningham wrote: Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled, Do you mean auth_cache_negative_ttl=0 by this? It only affects user not found caching. new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered. Is this something wrong with my configuraiton, or the auth code itself? The way it should work is that: 1) User logs in with password X which succeeds. 2) Password is changed to Y. 3) User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup, sees that the password was changed and caches it. But this can also happen: 1) User logs in with password X which succeeds. 2) Password is changed to Y. 3) User logs in with password X, which succeeds. Or: 1) User logs in with password X which succeeds. 2) User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth succeeded, so it'll do an auth lookup and sees that the password wasn't changed. 3) Password is changed to Y. 4) User logs in with password Y. Dovecot sees that X != Y, but it sees that the previous auth failed, so it doesn't bother doing another lookup. Can you consistently make Dovecot behave differently as described above?
Re: [Dovecot] Auth Issues - Urgent - Help!
Wowa, that's easy enough. I will do that the next time that I upgrade
in a few weeks.
Dave
Quoting Timo Sirainen [EMAIL PROTECTED]:
On Fri, 2008-11-21 at 14:50 -0500, David Cunningham wrote:
Is there any way to force the cache to
check the password for anything that was not previously cached as
being the correct password?
Nope. Hmm. Perhaps there should be a different TTL for that. I don't
really like adding new settings though.
For now you can at least do it by modifying sources:
src/auth/passdb-cache.c:
if (ret == 0 node-last_success) {
Change it to:
if (ret == 0) {
Re: [Dovecot] Auth Issues - Urgent - Help!
Yes, i telnet to port 143 and enter everything manually. Dave Quoting Charles Marcus [EMAIL PROTECTED]: On 11/19/2008 10:17 PM, David Cunningham wrote: Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled, new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered. Is this something wrong with my configuraiton, or the auth code itself? Maybe it is the mail client doing the caching... have you tested this on the command line? -- Best regards, Charles
Re: [Dovecot] Auth Issues - Urgent - Help!
Well, most of my issues are gone with adding auth cache. However, I am having an issue. Sometimes, even though cache incorrect passwords is disabled, new passwords do not work. It would seem that once a user logs in with one password successfully the cache does not automatically retry if the user tries a different passwords. I would think that the auth cache should check to see if the password changed on the ldap server if something other than the cached password is entered. Is this something wrong with my configuraiton, or the auth code itself? Thanks, Dave Quoting Timo Sirainen [EMAIL PROTECTED]: On Wed, 2008-10-08 at 08:01 -0400, David Cunningham wrote: After a few hours of running, I get tons of the following errors in my logs: dovecot: Oct 08 07:41:50 Error: auth(default): ldap([EMAIL PROTECTED],x.x.x.x): Request queue is full BTW. I improved this error message slightly to also tell how many seconds old data is in the queue. http://hg.dovecot.org/dovecot-1.1/rev/0329dc4df5ed I guess you're using auth binds? If you weren't, I think it wouldn't be possible to fill the queue.
[Dovecot] Auth Issues - Urgent - Help!
After a few hours of running, I get tons of the following errors in my logs: dovecot: Oct 08 07:41:50 Error: auth(default): ldap([EMAIL PROTECTED],x.x.x.x): Request queue is full I removed the username and IP, obviously. Any idea how to stop this? I have about 5 Thousand users using horde that login ever 1-5 minutes to refresh their page. I assume it is a setting, but I am confused as to why it doesn't happen almost right away. It seems to take some time to build up. Please help! This is taking my webmail system down hourly.
Re: [Dovecot] Auth Issues - Urgent - Help!
Here is my dovecot -n: # 1.1.3: /etc/dovecot.conf log_path: /var/log/dovecot info_log_path: /var/log/dovecot-info login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: MECnet Mail System, Authorized Use Only, Please Log In. login_process_per_connection: no login_process_size: 1024 login_max_processes_count: 1024 login_max_connections: 1024 max_mail_processes: 5 verbose_proctitle: yes first_valid_uid: 50 mail_uid: 93 mail_gid: 12 mail_location: maildir:/var/spool/maildirs/%d/%n/Maildir mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_process_size: 1024 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: verbose: yes passdb: driver: ldap args: /etc/dovecot-ldap.conf plugin: quota: maildir My buddy and I found #define DB_LDAP_MAX_QUEUE_SIZE 1024 in the db-ldap.h file in the source. We believe we hitting this threshold for some reason. So, we are looking to increase this to 8192. However, when trying to build the RPM source from atrpms, I get this: # rpmbuild -ba dovecot.spec error: line 1: Unknown tag: %bcond_without inotify Any help? Dave Quoting Jurvis LaSalle [EMAIL PROTECTED]: On Oct 8, 2008, at 8:01 AM, David Cunningham wrote: After a few hours of running, I get tons of the following errors in my logs: dovecot: Oct 08 07:41:50 Error: auth(default): ldap([EMAIL PROTECTED],x.x.x.x): Request queue is full I removed the username and IP, obviously. Any idea how to stop this? I have about 5 Thousand users using horde that login ever 1-5 minutes to refresh their page. I assume it is a setting, but I am confused as to why it doesn't happen almost right away. It seems to take some time to build up. Please help! This is taking my webmail system down hourly. dovecot -n? Hunch is login_max_processes_count is too low. http://wiki.dovecot.org/LoginProcess hth, JL
Re: [Dovecot] Auth Issues - Urgent - Help!
Unfortantely, it just happened again! sigh I am going to implement my increased queue change and see what happens. Dave Quoting Jurvis LaSalle [EMAIL PROTECTED]: On Oct 8, 2008, at 8:01 AM, David Cunningham wrote: After a few hours of running, I get tons of the following errors in my logs: dovecot: Oct 08 07:41:50 Error: auth(default): ldap([EMAIL PROTECTED],x.x.x.x): Request queue is full I removed the username and IP, obviously. Any idea how to stop this? I have about 5 Thousand users using horde that login ever 1-5 minutes to refresh their page. I assume it is a setting, but I am confused as to why it doesn't happen almost right away. It seems to take some time to build up. Please help! This is taking my webmail system down hourly. dovecot -n? Hunch is login_max_processes_count is too low. http://wiki.dovecot.org/LoginProcess hth, JL
Re: [Dovecot] Auth Issues - Urgent - Help!
Thank you, I will try the caching.
Dave
Quoting Timo Sirainen [EMAIL PROTECTED]:
On Wed, 2008-10-08 at 10:48 -0400, David Cunningham wrote:
I agree. In fact, I may have found a DNS issue that may have been
causing login sessions to hang and thus reach max too quickly. The
last few hours have been stable. So, I am keeping my fingers crossed.
I have also recompiled dovecot and changed the setting in db-ldap.h
that reads:
#define DB_LDAP_MAX_QUEUE_SIZE 1024
to
#define DB_LDAP_MAX_QUEUE_SIZE 8192
If you're getting more than 1024 requests queued, something's wrong or
you have hundreds or logins per second. Which one is it? (5000 users
logging in once per minute is still only 83/sec)
The queue keeps increasing if the LDAP server isn't replying to old
requests. So have you looked at the LDAP server side if it's running too
slow?
Anyway two things you could do here:
1) Enable auth cache with large enough size so Dovecot doesn't consult
LDAP server nearly as much.
2) Increase the number of auth processes (auth { .. count=5 }), so that
you'll use more connections and hopefully the LDAP server likes that
better than one connection sending lots of requests. Or maybe not.
Re: [Dovecot] Auth Issues - Urgent - Help!
Simply changing auth_cache_size to a non-zero number enables caching, correct?
How big is too big?
Where does it cache it?
Here is what I set:
auth_cache_size = 1048576
I was hoping for 1GB worth of cache.
I have 16GB of memory on the system, so memory is not an issue if it
stores it in memory as opposed to disk.
Dave
Quoting David Cunningham [EMAIL PROTECTED]:
Thank you, I will try the caching.
Dave
Quoting Timo Sirainen [EMAIL PROTECTED]:
On Wed, 2008-10-08 at 10:48 -0400, David Cunningham wrote:
I agree. In fact, I may have found a DNS issue that may have been
causing login sessions to hang and thus reach max too quickly. The
last few hours have been stable. So, I am keeping my fingers crossed.
I have also recompiled dovecot and changed the setting in db-ldap.h
that reads:
#define DB_LDAP_MAX_QUEUE_SIZE 1024
to
#define DB_LDAP_MAX_QUEUE_SIZE 8192
If you're getting more than 1024 requests queued, something's wrong or
you have hundreds or logins per second. Which one is it? (5000 users
logging in once per minute is still only 83/sec)
The queue keeps increasing if the LDAP server isn't replying to old
requests. So have you looked at the LDAP server side if it's running too
slow?
Anyway two things you could do here:
1) Enable auth cache with large enough size so Dovecot doesn't consult
LDAP server nearly as much.
2) Increase the number of auth processes (auth { .. count=5 }), so that
you'll use more connections and hopefully the LDAP server likes that
better than one connection sending lots of requests. Or maybe not.
