[Dovecot] Trim trailing whitespace from username
Recently we changed Postfix to use Dovecot for our SASL authentication
and we ran into trouble with some of our clients having extraneous
spaces at the end of their usernames. The quick fix was to add a space
to username_chars. The slightly longer fix was a pretty simple patch to
Dovecot. I put the trimming in auth_request_fix_username. I didn't think
it warranted a full strfuncs function.
If there is a better way to do this I'm all ears. I don't really like
patching with my own code, even if I did essentially steal if from the
kernel's strstrip().
diff -u dovecot-1.1.rc5/src/auth/auth-request.c
dovecot-1.1.rc5-patched/src/auth/auth-request.c
--- dovecot-1.1.rc5/src/auth/auth-request.c 2008-05-04
15:01:52.0 -0700
+++ dovecot-1.1.rc5-patched/src/auth/auth-request.c 2008-05-16
00:44:15.0 -0700
@@ -22,6 +22,7 @@
#include stdlib.h
#include sys/stat.h
+#include ctype.h
struct auth_request *
auth_request_new(struct auth *auth, const struct mech_module *mech,
@@ -750,6 +751,7 @@
{
unsigned char *p;
char *user;
+ size_t size;
if (strchr(username, '@') == NULL
request-auth-default_realm != NULL) {
@@ -759,6 +761,16 @@
user = p_strdup(request-pool, username);
}
+ /* Trim trailing whitespace from the username */
+ size = strlen((unsigned char*)user);
+ if(size) {
+ p = user + size - 1;
+ while (p != user isspace(*p))
+ p--;
+ *(p + 1) = '\0';
+ p = NULL;
+ }
+
for (p = (unsigned char *)user; *p != '\0'; p++) {
if (request-auth-username_translation[*p 0xff] != 0)
*p = request-auth-username_translation[*p
0xff];
Re: [Dovecot] Trim trailing whitespace from username
On Fri, 2008-05-16 at 00:48 -0700, David Jonas wrote: Recently we changed Postfix to use Dovecot for our SASL authentication and we ran into trouble with some of our clients having extraneous spaces at the end of their usernames. The quick fix was to add a space to username_chars. The slightly longer fix was a pretty simple patch to Dovecot. I put the trimming in auth_request_fix_username. I didn't think it warranted a full strfuncs function. If there is a better way to do this I'm all ears. I don't really like patching with my own code, even if I did essentially steal if from the kernel's strstrip(). How about this: http://hg.dovecot.org/dovecot-1.1/rev/15ddb7513e2d Then you can use auth_username_format = %Tu signature.asc Description: This is a digitally signed message part
[Dovecot] Looking for suggestions: How to strip attachments from mails
Hi, I know this is not 100% on-topic, but I'll try anyway... I'm looking for a way to strip attachments from incoming mails on the server. The attachments should be stored on a file server (where users have access via SAMBA) and only a link to the file (UNC path) should remain in the mail body. This is to prevent the mail storage from filling up with a large amount of big mails where at the same time the 'payload' (attached file) is not conveniently accessible by the users. The current setup is postfix / dovecot deliver / dovecot (with sieve plugin). I'm considering piping the mails through something like mimeStrip.pl (http://freshmeat.net/projects/mimestrip.pl/), but therefore I'd probably need to bring procmail/maildrop into the game, right? Or are there other (better) ways to accomplish this? Patrick. -- STAR Software (Shanghai) Co., Ltd.http://www.star-group.net/ Phone:+86 (21) 5427 7799 x 826 Fax: +86 (21) 6485 0071 PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 signature.asc Description: This is a digitally signed message part.
Re: [Dovecot] mysql index control allow nets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 timo thanks again (and again). but ... :( i attach my logs with mail_debug=yes i also attach my dovecot-sql.conf and my table too. I am reading from http://wiki.dovecot.org/UserDatabase that variable mail is overriding the default mail_location so i tried to pass all my values at mail with delimiter : for INDEX and CONTROL. Timo Sirainen wrote: On May 15, 2008, at 5:07 PM, Evaggelos Balaskas wrote: there is not index nor control folder nor user folder created under /var/mail Evaggelos Balaskas - http://ebalaskas.gr Unix System Engineer Informatics Engineer Technological Education -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFILX/PWIK+Pe9twhoRAhYcAJ0WT+7onaRO+R5jArhpzrqHUrHKkACcCxDw iiYRJQW2I9jFfSKhtUlK4us= =GsYd -END PGP SIGNATURE- dovecot: May 16 15:25:59 Info: IMAP([EMAIL PROTECTED]): Connection closed bytes=0/0 dovecot: May 16 15:26:01 Info: auth(default): new auth connection: pid=6618 dovecot: May 16 15:26:06 Info: auth(default): client in: AUTH 1 PLAIN service=imapsecured lip=127.0.0.1 rip=127.0.0.1 lport=1143 rport=36937 resp=AGViYWxAZWJhbGFza2FzLmdyAHRlc3Q= dovecot: May 16 15:26:06 Info: auth(default): client out: OK1 [EMAIL PROTECTED] dovecot: May 16 15:26:06 Info: auth-worker(default): sql([EMAIL PROTECTED],127.0.0.1): query: SELECT concat(userid, '@', domain) AS user, password, nets AS allow_nets FROM users WHERE userid = 'ebal' AND domain = 'ebalaskas.gr' dovecot: May 16 15:26:06 Info: auth-worker(default): auth([EMAIL PROTECTED],127.0.0.1): allow_nets: Matching for network 127.0.0.1 dovecot: May 16 15:26:06 Info: auth(default): master in: REQUEST3 65121 dovecot: May 16 15:26:06 Info: auth-worker(default): sql([EMAIL PROTECTED],127.0.0.1): SELECT concat(userid, '@', domain) AS user, home, username, uid, gid, concat( mail_type, ':', home, 'ebalaskas.gr/ebal:INDEX=', home, `index`, '/ebalaskas.gr/ebal:CONTROL=', home, control , '/ebalaskas.gr/ebal' ) AS mail FROM users WHERE userid = 'ebal' AND domain = 'ebalaskas.gr' dovecot: May 16 15:26:06 Info: auth(default): master out: USER 3 [EMAIL PROTECTED] home=/var/mail/ username=Evaggelos Balaskas uid=1108 gid=8 mail=maildir:/var/mail/ebalaskas.gr/ebal:INDEX=/var/mail/.index/ebalaskas.gr/ebal:CONTROL=/var/mail/.control/ebalaskas.gr/ebal dovecot: May 16 15:26:06 Info: imap-login: Login: user=[EMAIL PROTECTED], method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured dovecot: May 16 15:26:06 Info: IMAP([EMAIL PROTECTED]): Effective uid=1108, gid=8, home=/var/mail/ dovecot: May 16 15:26:06 Info: IMAP([EMAIL PROTECTED]): maildir: data=/var/mail/ebalaskas.gr/ebal:INDEX=/var/mail/.index/ebalaskas.gr/ebal:CONTROL=/var/mail/.control/ebalaskas.gr/ebal dovecot: May 16 15:26:06 Info: IMAP([EMAIL PROTECTED]): maildir++: root=/var/mail/ebalaskas.gr/ebal, index=/var/mail/.index/ebalaskas.gr/ebal, control=/var/mail/.control/ebalaskas.gr/ebal, inbox=/var/mail/ebalaskas.gr/ebal driver = mysql connect = host=localhost dbname=dovecot user=root password_query = SELECT concat(userid, '@', domain) AS user, \ password, nets AS allow_nets FROM users WHERE userid = '%n' AND domain = '%d' user_query = SELECT concat(userid, '@', domain) AS user, home, username, uid, gid, \ concat( mail_type, ':', home, '%d/%n:INDEX=', home, `index`, '/%d/%n:CONTROL=', home, control , '/%d/%n' ) AS mail \ FROM users WHERE userid = '%n' AND domain = '%d' CREATE TABLE IF NOT EXISTS `users` ( `userid` varchar(128) NOT NULL, `domain` varchar(128) NOT NULL, `password` varchar(64) NOT NULL, `username` varchar(100) NOT NULL, `mail_type` varchar(10) NOT NULL, `home` varchar(255) NOT NULL, `sieve` varchar(255) NOT NULL, `index` varchar(100) NOT NULL, `control` varchar(100) NOT NULL, `nets` varchar(100) NOT NULL, `uid` int(11) NOT NULL, `gid` int(11) NOT NULL, PRIMARY KEY (`userid`,`domain`) ) ENGINE=MyISAM;
[Dovecot] Sieve plugins 1.0.3 and 1.1.5 released
http://dovecot.org/releases/sieve/dovecot-sieve-1.0.3.tar.gz http://dovecot.org/releases/sieve/dovecot-sieve-1.0.3.tar.gz.sig http://dovecot.org/releases/sieve/dovecot-sieve-1.1.5.tar.gz http://dovecot.org/releases/sieve/dovecot-sieve-1.1.5.tar.gz.sig I've usually released new Sieve plugins at the same time as new Dovecot versions, but since I wasn't planning on releasing v1.0.14 anytime soon and it looks like v1.0.2 release was quite a while ago, I decided to release these today. Changes: - Improved vacation reply checks, log discarded vacation replies (in v1.0.3, these were already in v1.1.0) - Expand ~ to home directory in sieve path - If compiled script has the exact same mtime as the source script, assume the script hasn't changed (changed check to =). - Unfold multiline headers before comparing them. - Improved error logging. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Looking for suggestions: How to strip attachments from mails
I'm looking for a way to strip attachments from incoming mails on the server. http://detach.optimism.cc/ works in line with procmail or similiar. It is not a standalone server; but instead acts as a filter. I use it in front of my mailing lists, so that attachements are not sent out. I couple this with demime to further reduce the remaining text to plain text ( http://scifi.squawk.com/demime.html - uh oh, host not responding.. and I don't see an alternate location for it).
[Dovecot] imap-login processes
First, thanks for this great product. We migrated from Courier last week with only small bumps along the way. We have a few hundred active users with a mix of pop3 and imap clients. The server is running on sparc Solaris 9. We're seeing a problem now with the number of imap-login processes rising slowly to it's limit and then (understandably) not letting any more logins happen. Legitimate users seem to have no problem logging in until the limit is hit. Another odd thing is that this is not a problem for pop3-login processes, just imap... I've tried changing some obvious things including the passdb from pam to passwd and back again, but here's the current dovecot -n output: # 1.0.13: /opt/dovecot/etc/dovecot.conf protocols: imap imaps pop3 pop3s ssl_cert_file: /opt/dovecot/etc/ssl/certs/dovecot.pem ssl_key_file: /opt/dovecot/etc/ssl/certs/key.pem disable_plaintext_auth: no login_dir: /opt/dovecot/var/run/dovecot/login login_executable(default): /opt/dovecot/libexec/dovecot/imap-login login_executable(imap): /opt/dovecot/libexec/dovecot/imap-login login_executable(pop3): /opt/dovecot/libexec/dovecot/pop3-login login_processes_count: 12 login_max_processes_count: 512 verbose_proctitle: yes mail_location: maildir:~/Maildir mmap_disable: yes mail_executable(default): /opt/dovecot/libexec/dovecot/imap mail_executable(imap): /opt/dovecot/libexec/dovecot/imap mail_executable(pop3): /opt/dovecot/libexec/dovecot/pop3 mail_plugin_dir(default): /opt/dovecot/lib/dovecot/imap mail_plugin_dir(imap): /opt/dovecot/lib/dovecot/imap mail_plugin_dir(pop3): /opt/dovecot/lib/dovecot/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv namespace: type: private prefix: INBOX. inbox: yes auth default: verbose: yes passdb: driver: pam userdb: driver: passwd Any ideas or advice? Thanks, Bryan Polk Unix Systems Administrator Communication and Multimedia Services FAMU-FSU College of Engineering [EMAIL PROTECTED]
Re: [Dovecot] imap-login processes
On Fri, 2008-05-16 at 11:38 -0400, Bryan Polk wrote: First, thanks for this great product. We migrated from Courier last week with only small bumps along the way. We have a few hundred active users with a mix of pop3 and imap clients. The server is running on sparc Solaris 9. We're seeing a problem now with the number of imap-login processes rising slowly to it's limit and then (understandably) not letting any more logins happen. Legitimate users seem to have no problem logging in until the limit is hit. Another odd thing is that this is not a problem for pop3-login processes, just imap... How many imap processes do you have at that time? Each SSL connection uses up one imap-login process. One sure way to fix this would be to change to high-performance mode as described by http://wiki.dovecot.org/LoginProcess signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Trim trailing whitespace from username
Timo Sirainen wrote: On Fri, 2008-05-16 at 00:48 -0700, David Jonas wrote: Recently we changed Postfix to use Dovecot for our SASL authentication and we ran into trouble with some of our clients having extraneous spaces at the end of their usernames. The quick fix was to add a space to username_chars. The slightly longer fix was a pretty simple patch to Dovecot. I put the trimming in auth_request_fix_username. I didn't think it warranted a full strfuncs function. If there is a better way to do this I'm all ears. I don't really like patching with my own code, even if I did essentially steal if from the kernel's strstrip(). How about this: http://hg.dovecot.org/dovecot-1.1/rev/15ddb7513e2d Then you can use auth_username_format = %Tu Ah, a much better place to put it. Applied cleaningly, seems to be working well. Thanks! I've added it to the wiki, http://wiki.dovecot.org/Variables
Re: [Dovecot] imap-login processes
On Fri, 16 May 2008, Timo Sirainen wrote: How many imap processes do you have at that time? Each SSL connection uses up one imap-login process. One sure way to fix this would be to change to high-performance mode as described by http://wiki.dovecot.org/LoginProcess Currently 74 imap processes, 336 imap-login processes. I tried switching to high-performance mode yesterday.. It seemed to end up with sort of the same results, though harder to diagnose. Each process I guess ended up hitting the login_process_size = 64 max and then stopped responding. So imap would work for some people and not others. I'd rather have it be broken for all or none, so I switched it back instead of tweaking the performance options. I think there's something in our environment causing this but I have no idea what.. -bryan
Re: [Dovecot] imap-login processes
On May 16, 2008, at 6:56 PM, Bryan Polk wrote: On Fri, 16 May 2008, Timo Sirainen wrote: How many imap processes do you have at that time? Each SSL connection uses up one imap-login process. One sure way to fix this would be to change to high-performance mode as described by http://wiki.dovecot.org/LoginProcess Currently 74 imap processes, 336 imap-login processes. It would help to know what these extra processes are doing. Unfortunately there's no simple way to do that.. Maybe writing a script that trusses the processes for a few seconds and then seeing what it shows? Another way would be to try if LINUX_PROCTITLE_HACK works also with Solaris. You can remove the comments around #define from src/lib/ process-title.c and then set verbose_proctitle=yes. This should first be tested though, because if it doesn't work it could break badly. If it does work, you should at least see IP addresses for each process that has a connected client and also TLS if SSL/TLS is being used. It could probably also include username for SSL proxies. I tried switching to high-performance mode yesterday.. It seemed to end up with sort of the same results, though harder to diagnose. Each process I guess ended up hitting the login_process_size = 64 max and then stopped responding. If it hits that limit, it gets killed by signal 9 by the kernel (and that gets logged). It's also a good idea to then increase it to something like 256. But I can't see why it would stop responding then. PGP.sig Description: This is a digitally signed message part
[Dovecot] Panic: IMAP(...): Trying to close mailbox INBOX with open transactions
I have been seeing this crash, which has been reported before but apparently not yet resolved. As with the previous reporters, I do not know how to reproduce it reliably. Dovecot version: 1.1.rc5 Operating system: Mac OS X 10.5.2 CPU architecture: x86 File system: HFS+ Activity: From the backtrace it appears the client logged out while in the middle of some operation. Backtrace: 0 libSystem.B.dylib 0x946210ea __kill + 10 1 libSystem.B.dylib 0x946983f2 raise + 26 2 libSystem.B.dylib 0x946a79af abort + 73 3 imap 0x000743e2 i_internal_fatal_handler + 102 4 imap0x00073e03 i_fatal + 0 5 imap0x0004614c mailbox_close + 48 6 imap0x53c4 cmd_logout + 74 7 imap0x7f14 client_command_input + 41 8 imap0x80d9 client_command_input + 494 9 imap0x81c5 client_handle_input + 218 10 imap0x835d client_input + 176 11 imap0x0007c001 io_loop_handler_run + 350 12 imap0x0007af06 io_loop_run + 44 13 imap0x00010732 main + 1972 14 imap0x1e76 start + 54
Re: [Dovecot] imap-login processes
I tried switching to high-performance mode yesterday.. It seemed to end up with sort of the same results, though harder to diagnose. Each process I guess ended up hitting the login_process_size = 64 max and then stopped responding. We had this same problem, and switching to high-performance mode helps. However, finding out which user or application is logging in, and controlling the end-user is the only way to fix it. In our case it was a multi-threaded application that used IMAP (excessively). Monitoring the maillog should help point you in the right direction. Dozens of these per minute should throw up a red-flag: May 16 17:36:16 hostname imap-login: Login: user [:::IP_Address] Joe
Re: [Dovecot] imap-login processes
On May 16, 2008, at 8:39 PM, Joe Allesi wrote: I tried switching to high-performance mode yesterday.. It seemed to end up with sort of the same results, though harder to diagnose. Each process I guess ended up hitting the login_process_size = 64 max and then stopped responding. We had this same problem, and switching to high-performance mode helps. However, finding out which user or application is logging in, and controlling the end-user is the only way to fix it. In our case it was a multi-threaded application that used IMAP (excessively). Monitoring the maillog should help point you in the right direction. Dozens of these per minute should throw up a red-flag: May 16 17:36:16 hostname imap-login: Login: user [:::IP_Address] v1.1 probably helps with this, since it limits the number of simultaneous connections from a user+IP combination. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Panic: IMAP(...): Trying to close mailbox INBOX with open transactions
On May 16, 2008, at 8:29 PM, Mike Abbott wrote: I have been seeing this crash, which has been reported before but apparently not yet resolved. As with the previous reporters, I do not know how to reproduce it reliably. Could you also post your dovecot -n output? PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] imap-login processes
On Fri, 16 May 2008, Joe Allesi wrote: We had this same problem, and switching to high-performance mode helps. However, finding out which user or application is logging in, and controlling the end-user is the only way to fix it. In our case it was a multi-threaded application that used IMAP (excessively). Monitoring the maillog should help point you in the right direction. Dozens of these per minute should throw up a red-flag: Looking back through the log for today we only have about 7-35 imap-logins happening per minute, from an assortment of users. There doesn't appear to be one user that's doing more than others. One thing I did notice was entries like this: imap-login: Login: user=faizalmi, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Is there a reason the rip/lip would say 127.0.0.1 for some small number of users and not others? It would help to know what these extra processes are doing. Unfortunately there's no simple way to do that.. Maybe writing a script that trusses the processes for a few seconds and then seeing what it shows? To truss each imap-login I would need to write the script to execute truss imap-login and put that in place of imap-login in the config file? I think I might try the source code re-compile first.. -bryan
Re: [Dovecot] imap-login processes
On Fri, 16 May 2008 14:47:05 -0400 (EDT) Bryan Polk wrote: On Fri, 16 May 2008, Joe Allesi wrote: We had this same problem, and switching to high-performance mode helps. However, finding out which user or application is logging in, and controlling the end-user is the only way to fix it. In our case it was a multi-threaded application that used IMAP (excessively). Monitoring the maillog should help point you in the right direction. Dozens of these per minute should throw up a red-flag: Looking back through the log for today we only have about 7-35 imap-logins happening per minute, from an assortment of users. There doesn't appear to be one user that's doing more than others. One thing I did notice was entries like this: imap-login: Login: user=faizalmi, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Is there a reason the rip/lip would say 127.0.0.1 for some small number of users and not others? Do you have a Web-Interface like SquirrelMail or Horde on the same host? Our SquirrelMail shows up in this fashion. --Frank Elsner
Re: [Dovecot] imap-login processes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I hope something like this could help you : # Authentication Cache auth_cache_size = 10240 auth_cache_ttl = 18000 Bryan Polk wrote: On Fri, 16 May 2008, Frank Elsner wrote: Do you have a Web-Interface like SquirrelMail or Horde on the same host? Our SquirrelMail shows up in this fashion. Oh, yeah that would be it. Thanks :) Evaggelos Balaskas - http://ebalaskas.gr Unix System Engineer Informatics Engineer Technological Education -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFILfJ0WIK+Pe9twhoRAtP4AJ9tH5h7M+kPPCDQFgt3wiF0lvGBPwCgz3gj WFU4bDg/Y0SPWlBQ1Md78aI= =BcMN -END PGP SIGNATURE-
Re: [Dovecot] mysql index control allow nets
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 everything is perfect. Evaggelos Balaskas wrote: timo thanks again (and again). but ... :( i attach my logs with mail_debug=yes i also attach my dovecot-sql.conf and my table too. I am reading from http://wiki.dovecot.org/UserDatabase that variable mail is overriding the default mail_location so i tried to pass all my values at mail with delimiter : for INDEX and CONTROL. Timo Sirainen wrote: On May 15, 2008, at 5:07 PM, Evaggelos Balaskas wrote: there is not index nor control folder nor user folder created under /var/mail Evaggelos Balaskas - http://ebalaskas.gr Unix System Engineer Informatics Engineer Technological Education -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFILgBTWIK+Pe9twhoRArrvAJ4+0wFK9chOQaZDzTm+f7O5NhieKACfSgLw 1vU6sXK6kuIw49l0ENV1Is0= =ecEE -END PGP SIGNATURE-
