Re: [Dovecot] How to run Dovecot as IMAP Proxy?
> > You will not get any support for 0.99 here, unless Timo can provide it. > It is *that* old, and your decision to use it - regardless of your > claimed reason - is baffling. I admit I am running a pretty old version and frankly speaking I was not hoping for any support. Let RH earn their money we paid as support. As you would have noticed I was looking for help in running Dovecot as a IMAP Proxy without the need to setup SQL etc. I only went into details to give clarity. I really appreciate all the responses and suggestions. >> We use recent Dovecot from atrpms.net with RHEL 5.3. Userbase is 25.000. >> Dovecot runs on three machines - as master (proxy) and with two slaves. >> The system is stable. We hardly need to touch it. I would be very much interested in knowing the details of your setup as I am suffering here with only 6000+ users. And that too with a config, which I frankly think is an overkill for our requirement. However maybe I should start a separate thread for that? Thanks and Regards V S Rao.
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
* Charles Marcus : > On 5/22/2009, V S Rao (viriy...@yahoo.com) wrote: > > BTW, just to confirm once again, are there any known issues with Dovecot & > > imapproxyd? > > You will not get any support for 0.99 here, unless Timo can provide it. > It is *that* old, and your decision to use it - regardless of your > claimed reason - is baffling. We use recent Dovecot from atrpms.net with RHEL 5.3. Userbase is 25.000. Dovecot runs on three machines - as master (proxy) and with two slaves. The system is stable. We hardly need to touch it. p...@rick -- state of mind Digitale Kommunikation http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht MünchenPartnerschaftsregister PR 563
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On 5/22/2009, V S Rao (viriy...@yahoo.com) wrote: > BTW, just to confirm once again, are there any known issues with Dovecot & > imapproxyd? You will not get any support for 0.99 here, unless Timo can provide it. It is *that* old, and your decision to use it - regardless of your claimed reason - is baffling. The only experience I had with it - what, 5 years ago? - was its broken mbox problem... we had to move to courier because of it. -- Best regards, Charles
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On Thu, 2009-05-21 at 21:26 -0700, V S Rao wrote: > BTW, just to confirm once again, are there any known issues with Dovecot & > imapproxyd? Because I find it strange that there should be around 8000 > established connections between Squirrelmail and imapproxyd & around 400 > connections between imapproxyd and Dovecot while on the mail server I see > only a couple of connections (typically around 40). Or is it some > configuration in Dovecot which is causing this kind of behavior, as I did not > notice this while running uw-imap. > missed your OP post here (anti spam must have ate it, i think because of very-long-lines) but I read the archives, do you mean up-imapproxy from http://www.imapproxy.org ? none, I've been using those them together for a very long time. I suspect from reading the archives you don't mean it since it doesnt use any user databases, perhaps you need to have a look at it. If you are using it. then you need to first make sure you are using current version, and not some 10 year old version included in a debian or redhat install. suggest using up-imapproxy-1.2.7rc2.tar.gz thats been very stable for some time and I think Dave will be issuing a stable 1.7.2 in the near future, we use it on production boxes without a single glitch, but if your pedantic or have company politics that forbid RC/Betas, 1.2.6 will still do the job fine.
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
So I guess I will take the problem to imapproxy group and find out if I can get a clue to the issue I am facing. BTW, just to confirm once again, are there any known issues with Dovecot & imapproxyd? Because I find it strange that there should be around 8000 established connections between Squirrelmail and imapproxyd & around 400 connections between imapproxyd and Dovecot while on the mail server I see only a couple of connections (typically around 40). Or is it some configuration in Dovecot which is causing this kind of behavior, as I did not notice this while running uw-imap. Regards V S Rao
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
You can also use pimpstat to monitor the effectiveness of imapproxy. We had imapproxy running running in front of squirrelmail for a post.office IMAP server (based on uw-imap). We eventually went into the code for imapproxy and tweaked some of the SELECT caching and even tweaked some of the squirrelmail imap code a bit to be more efficient with the proxy. Now we are using Dovecot without imapproxy in front of squirrelmail and there are no problems. -Original Message- From: dovecot-bounces+jkrejci=usinternet@dovecot.org [mailto:dovecot-bounces+jkrejci=usinternet@dovecot.org] On Behalf Of Seth Mattinen Sent: Thursday, May 21, 2009 4:36 PM To: V S Rao Cc: Dovecot Mailing List Subject: Re: [Dovecot] How to run Dovecot as IMAP Proxy? V S Rao wrote: > Thanks for the many responses and views. > > I have taken RH support for my mail server only and so have to ensure they support me. So am going with their recommended version which is 0.99.x. > > Now on the webmail I have migrated to RHEL 5.3 and the dovecot being used there is 1.0.7.x. So that should support proxying, right? > > BTW, originally I did not go to Redhat. I moved from uw-imap to Dovecot 1.1.14 on the mail server for POP3. The problem of POP3 timeouts continued and I could not find any reason why POP3 was timing out when 400+ concurrent IMAP sessions were working fine. Moreover other network services such as SMTP, Telnet, SSH etc were working fine. Had the box installed behind a IPS box to see if there were any DoS attacks, but had to rule that out. As a last resort, paid RH for support and they made me downgrade Dovecot from 1.1.14 to 0.99.x. Surprisingly the moment I downgraded dovecot, the problem shifted to IMAP & remained ever since. So naturally they told me uw-imap was the problem and made me move to Dovecot 0.99.x for IMAP as well, but no improvement. > > But the surprising thing, as I mentioned in my original post, IMAP works fine on command line, with clients such as Outlook, THunderbird etc., but Webmail is very slow that its almost not available. > > BTW webmail was also running on RHEL 4.0. Seeing that the problem existed in webmail alone I upgraded webmail to RHEL 5.3 with all the latest RH supported packages on a new hardware. > > Any ideas to help me debug this issue would be of great help as I am stuck on this issue for almost a month now. > > Oh, btw regarding stats, I don't have a measure, but when webmail was working the login was well within 5 seconds. Now it takes forever (more than a min) or timeouts most of the time. Even after login, any request such as opening a mail or changing to a mail folder was well within 5 seconds. Now again it takes forever or timeouts most of the time. > Use imapproxyd (UP-ImapProxy) if you want a caching proxy for webmail. It specifically deals with the "webmail constantly logging in" issue. >From the Debian man page: "ImapProxy was written to compensate for webmail clients that are unable to maintain persistent connections to an IMAP server. Most webmail clients need to log in to an IMAP server for nearly every single transaction; This behaviour can cause tragic performance problems on the IMAP server. ImapProxy tries to deal with this problem by leaving server connections open for a short time after a webmail client logs out. When the webmail client connects again, ImapProxy will determine if there is a cached connection available and reuse it if possible." Dovecot proxy quite simply won't do what it looks like you want to do, and is irrelevant on your webmail server. ~Seth
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
V S Rao wrote: > Thanks for the many responses and views. > > I have taken RH support for my mail server only and so have to ensure they > support me. So am going with their recommended version which is 0.99.x. > > Now on the webmail I have migrated to RHEL 5.3 and the dovecot being used > there is 1.0.7.x. So that should support proxying, right? > > BTW, originally I did not go to Redhat. I moved from uw-imap to Dovecot > 1.1.14 on the mail server for POP3. The problem of POP3 timeouts continued > and I could not find any reason why POP3 was timing out when 400+ concurrent > IMAP sessions were working fine. Moreover other network services such as > SMTP, Telnet, SSH etc were working fine. Had the box installed behind a IPS > box to see if there were any DoS attacks, but had to rule that out. As a last > resort, paid RH for support and they made me downgrade Dovecot from 1.1.14 to > 0.99.x. Surprisingly the moment I downgraded dovecot, the problem shifted to > IMAP & remained ever since. So naturally they told me uw-imap was the problem > and made me move to Dovecot 0.99.x for IMAP as well, but no improvement. > > But the surprising thing, as I mentioned in my original post, IMAP works fine > on command line, with clients such as Outlook, THunderbird etc., but Webmail > is very slow that its almost not available. > > BTW webmail was also running on RHEL 4.0. Seeing that the problem existed in > webmail alone I upgraded webmail to RHEL 5.3 with all the latest RH supported > packages on a new hardware. > > Any ideas to help me debug this issue would be of great help as I am stuck on > this issue for almost a month now. > > Oh, btw regarding stats, I don't have a measure, but when webmail was working > the login was well within 5 seconds. Now it takes forever (more than a min) > or timeouts most of the time. Even after login, any request such as opening a > mail or changing to a mail folder was well within 5 seconds. Now again it > takes forever or timeouts most of the time. > Use imapproxyd (UP-ImapProxy) if you want a caching proxy for webmail. It specifically deals with the "webmail constantly logging in" issue. >From the Debian man page: "ImapProxy was written to compensate for webmail clients that are unable to maintain persistent connections to an IMAP server. Most webmail clients need to log in to an IMAP server for nearly every single transaction; This behaviour can cause tragic performance problems on the IMAP server. ImapProxy tries to deal with this problem by leaving server connections open for a short time after a webmail client logs out. When the webmail client connects again, ImapProxy will determine if there is a cached connection available and reuse it if possible." Dovecot proxy quite simply won't do what it looks like you want to do, and is irrelevant on your webmail server. ~Seth
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On Thu, 2009-05-21 at 14:08 -0700, V S Rao wrote: > Now on the webmail I have migrated to RHEL 5.3 and the dovecot being > used there is 1.0.7.x. So that should support proxying, right? Yes, but I forgot this before: Dovecot's proxying doesn't do any connection caching. So there's no benefit in using it in front of a webmail. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
Thanks for the many responses and views. I have taken RH support for my mail server only and so have to ensure they support me. So am going with their recommended version which is 0.99.x. Now on the webmail I have migrated to RHEL 5.3 and the dovecot being used there is 1.0.7.x. So that should support proxying, right? BTW, originally I did not go to Redhat. I moved from uw-imap to Dovecot 1.1.14 on the mail server for POP3. The problem of POP3 timeouts continued and I could not find any reason why POP3 was timing out when 400+ concurrent IMAP sessions were working fine. Moreover other network services such as SMTP, Telnet, SSH etc were working fine. Had the box installed behind a IPS box to see if there were any DoS attacks, but had to rule that out. As a last resort, paid RH for support and they made me downgrade Dovecot from 1.1.14 to 0.99.x. Surprisingly the moment I downgraded dovecot, the problem shifted to IMAP & remained ever since. So naturally they told me uw-imap was the problem and made me move to Dovecot 0.99.x for IMAP as well, but no improvement. But the surprising thing, as I mentioned in my original post, IMAP works fine on command line, with clients such as Outlook, THunderbird etc., but Webmail is very slow that its almost not available. BTW webmail was also running on RHEL 4.0. Seeing that the problem existed in webmail alone I upgraded webmail to RHEL 5.3 with all the latest RH supported packages on a new hardware. Any ideas to help me debug this issue would be of great help as I am stuck on this issue for almost a month now. Oh, btw regarding stats, I don't have a measure, but when webmail was working the login was well within 5 seconds. Now it takes forever (more than a min) or timeouts most of the time. Even after login, any request such as opening a mail or changing to a mail folder was well within 5 seconds. Now again it takes forever or timeouts most of the time. Thanks and Regards V S Rao - Original Message - From: "Timo Sirainen" To: "Dovecot Mailing List" Sent: Friday, May 22, 2009 12:32:24 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [Dovecot] How to run Dovecot as IMAP Proxy? On Thu, 2009-05-21 at 11:47 -0700, Seth Mattinen wrote: > Yes, that too. My point was not to judge Dovecot's performance based on > a 0.99 version. ;) I've successfully used several versions of dovecot > 1.1.x with imapproxyd and IMAP-only webmail clients. Do you have any numbers about how much faster it works with imapproxyd vs. without?
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
Timo Sirainen wrote: > On Thu, 2009-05-21 at 11:47 -0700, Seth Mattinen wrote: >> Yes, that too. My point was not to judge Dovecot's performance based on >> a 0.99 version. ;) I've successfully used several versions of dovecot >> 1.1.x with imapproxyd and IMAP-only webmail clients. > > Do you have any numbers about how much faster it works with imapproxyd > vs. without? > I've never run it without. I just did it because I'd rather collapse the 10 distinct logins a typical webmail client will make for every click into a single login. ~Seth
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On Thu, 2009-05-21 at 11:47 -0700, Seth Mattinen wrote: > Yes, that too. My point was not to judge Dovecot's performance based on > a 0.99 version. ;) I've successfully used several versions of dovecot > 1.1.x with imapproxyd and IMAP-only webmail clients. Do you have any numbers about how much faster it works with imapproxyd vs. without? signature.asc Description: This is a digitally signed message part
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
Scott Silva wrote: > on 5-21-2009 11:33 AM Seth Mattinen spake the following: >> V S Rao wrote: >>> Hi Timo, >>> >>> Thanks for the response. Apologize, but my responses are going to be a bit >>> lengthy. >>> I have migrated from uw-imap to Dovecot for POP3 & IMAP service. I run webmail using squirrelmail. When running uw-imap I used to run up.imapproxyd on the webmail server for faster responses. After migrating to Dovecot, I find that up.imapproxyd does not work well with dovecot. Why not? >>> Here are my observations. I have around 6000+ mailboxes and roughly the >>> same number of users. Earlier the mail server (running sendmail + uw-imap >>> for POP3 & IMAP) would have around 80 pop3 connections (peak) and around >>> 300 IMAP connections, concurrent. >>> (ps -aef | grep imap | wc -l or ps -aef | grep pop3 | wc -l). There have >>> been cases where I have observed upto 500 concurrent IMAP sessions. >>> >>> The IMAP connections are from a webmail server running Apache with >>> Squirrelmail. The observation was that response was slow & so based on the >>> suggestion on Squirrelmail for performance improvement we have installed >>> up-imapproxyd on the squirrelmail machine. There was a significant >>> improvement in the response times for the users, because of caching. >>> >>> After that I encountered some strange problems of POP3 timing out for users >>> (earlier I did post that problem in this forum). I opened a ticket with >>> Redhat and naturally they refused to support me with uw-imap running. So >>> switched to Dovecot 0.99.x (I run the server on RHEL 4.0 and that is the >>> max version supported by Redhat for that version). Ever since I did that >>> POP3 works fine but now webmail is almost not available to the users. >>> People usually get "connection dropped by IMAP server". However the IMAP >>> server seems to work fine. Checked through manual "telnet >>> 143 & also through other client such as outlook >>> & Thunderbird. >>> >> >> I realize you're using 0.99 because it's "supported" by RedHat, but in >> reality it's absolutely ancient history. There have been far too many >> performance enhancements/fixes between then and now to even begin to list. >> >> ~Seth >> > Besides, he didn't go to RedHat for support anyway. He came here. And here > says start with at least the latest 1.0 version, or maybe even 1.1. You can > get it from atrpms.net if you want an rpm. > Yes, that too. My point was not to judge Dovecot's performance based on a 0.99 version. ;) I've successfully used several versions of dovecot 1.1.x with imapproxyd and IMAP-only webmail clients. ~Seth
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
on 5-21-2009 11:33 AM Seth Mattinen spake the following: > V S Rao wrote: >> Hi Timo, >> >> Thanks for the response. Apologize, but my responses are going to be a bit >> lengthy. >> >>> I have migrated from uw-imap to Dovecot for POP3 & IMAP service. I run >>> webmail using squirrelmail. When running uw-imap I used to run >>> up.imapproxyd on the webmail server for faster responses. After >>> migrating to Dovecot, I find that up.imapproxyd does not work well >>> with dovecot. >>> Why not? >> Here are my observations. I have around 6000+ mailboxes and roughly the same >> number of users. Earlier the mail server (running sendmail + uw-imap for >> POP3 & IMAP) would have around 80 pop3 connections (peak) and around 300 >> IMAP connections, concurrent. >> (ps -aef | grep imap | wc -l or ps -aef | grep pop3 | wc -l). There have >> been cases where I have observed upto 500 concurrent IMAP sessions. >> >> The IMAP connections are from a webmail server running Apache with >> Squirrelmail. The observation was that response was slow & so based on the >> suggestion on Squirrelmail for performance improvement we have installed >> up-imapproxyd on the squirrelmail machine. There was a significant >> improvement in the response times for the users, because of caching. >> >> After that I encountered some strange problems of POP3 timing out for users >> (earlier I did post that problem in this forum). I opened a ticket with >> Redhat and naturally they refused to support me with uw-imap running. So >> switched to Dovecot 0.99.x (I run the server on RHEL 4.0 and that is the max >> version supported by Redhat for that version). Ever since I did that POP3 >> works fine but now webmail is almost not available to the users. People >> usually get "connection dropped by IMAP server". However the IMAP server >> seems to work fine. Checked through manual "telnet >> 143 & also through other client such as outlook & >> Thunderbird. >> > > > I realize you're using 0.99 because it's "supported" by RedHat, but in > reality it's absolutely ancient history. There have been far too many > performance enhancements/fixes between then and now to even begin to list. > > ~Seth > Besides, he didn't go to RedHat for support anyway. He came here. And here says start with at least the latest 1.0 version, or maybe even 1.1. You can get it from atrpms.net if you want an rpm. signature.asc Description: OpenPGP digital signature
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
V S Rao wrote: > Hi Timo, > > Thanks for the response. Apologize, but my responses are going to be a bit > lengthy. > >> I have migrated from uw-imap to Dovecot for POP3 & IMAP service. I run >> webmail using squirrelmail. When running uw-imap I used to run >> up.imapproxyd on the webmail server for faster responses. After >> migrating to Dovecot, I find that up.imapproxyd does not work well >> with dovecot. > >> Why not? > > Here are my observations. I have around 6000+ mailboxes and roughly the same > number of users. Earlier the mail server (running sendmail + uw-imap for POP3 > & IMAP) would have around 80 pop3 connections (peak) and around 300 IMAP > connections, concurrent. > (ps -aef | grep imap | wc -l or ps -aef | grep pop3 | wc -l). There have been > cases where I have observed upto 500 concurrent IMAP sessions. > > The IMAP connections are from a webmail server running Apache with > Squirrelmail. The observation was that response was slow & so based on the > suggestion on Squirrelmail for performance improvement we have installed > up-imapproxyd on the squirrelmail machine. There was a significant > improvement in the response times for the users, because of caching. > > After that I encountered some strange problems of POP3 timing out for users > (earlier I did post that problem in this forum). I opened a ticket with > Redhat and naturally they refused to support me with uw-imap running. So > switched to Dovecot 0.99.x (I run the server on RHEL 4.0 and that is the max > version supported by Redhat for that version). Ever since I did that POP3 > works fine but now webmail is almost not available to the users. People > usually get "connection dropped by IMAP server". However the IMAP server > seems to work fine. Checked through manual "telnet > 143 & also through other client such as outlook & Thunderbird. > I realize you're using 0.99 because it's "supported" by RedHat, but in reality it's absolutely ancient history. There have been far too many performance enhancements/fixes between then and now to even begin to list. ~Seth
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On Thu, 2009-05-21 at 11:10 -0700, V S Rao wrote: > So switched to Dovecot 0.99.x (I run the server on RHEL 4.0 and that > is the max version supported by Redhat for that version). Oh, no.. I had had hoped people would have stopped running v0.99 by now :) That's probably the cause of all your problems. Also did you plan on using it for proxying? I don't think 0.99 supports proxying. With v1.0+ your webmail would probably work just fine without even proxying. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
Hi Timo, Thanks for the response. Apologize, but my responses are going to be a bit lengthy. > I have migrated from uw-imap to Dovecot for POP3 & IMAP service. I run > webmail using squirrelmail. When running uw-imap I used to run > up.imapproxyd on the webmail server for faster responses. After > migrating to Dovecot, I find that up.imapproxyd does not work well > with dovecot. > Why not? Here are my observations. I have around 6000+ mailboxes and roughly the same number of users. Earlier the mail server (running sendmail + uw-imap for POP3 & IMAP) would have around 80 pop3 connections (peak) and around 300 IMAP connections, concurrent. (ps -aef | grep imap | wc -l or ps -aef | grep pop3 | wc -l). There have been cases where I have observed upto 500 concurrent IMAP sessions. The IMAP connections are from a webmail server running Apache with Squirrelmail. The observation was that response was slow & so based on the suggestion on Squirrelmail for performance improvement we have installed up-imapproxyd on the squirrelmail machine. There was a significant improvement in the response times for the users, because of caching. After that I encountered some strange problems of POP3 timing out for users (earlier I did post that problem in this forum). I opened a ticket with Redhat and naturally they refused to support me with uw-imap running. So switched to Dovecot 0.99.x (I run the server on RHEL 4.0 and that is the max version supported by Redhat for that version). Ever since I did that POP3 works fine but now webmail is almost not available to the users. People usually get "connection dropped by IMAP server". However the IMAP server seems to work fine. Checked through manual "telnet 143 & also through other client such as outlook & Thunderbird. I noticed that while the number of established connections for the webmail to mail server was shown as around 40 to 50 on the mail server, the same is shown as around 700 on webmail server. { netstat -an | grep :143 | grep ESTABLISHED | wc -l }. imapproxyd is configured to listen on port 9143 on the localhost and communicates to the mail server on std port 143. What was really bizzare is that when I check for the number of connections on port 9143 it shows around 8000! netstat -an | grep :9143 | grep ESTABLISHED | wc -l So I disabled the imapproxyd on the webmail & now at least the 50 connections are consistent. However the response is extremely slow and so was contemplating on the idea of trying out using another IMAP Proxy server and hence the question. > I don't understand Databases and don't want to install mysql etc for a > simple thing like IMAP PRoxy. > Do you just want it to proxy everyone to a single destination or to > different servers based on username? If you want everyone to the same > destination, why do you even want to use a proxy? I want to proxy all user to the same server. The idea being as explained above benefit from caching. > My question is can Dovecot be run as a IMAP PRoxy without the hassle > of setting up mysql etc? I use /etc/passwd for auth on the actual > dovecot server running IMAP & POP3. > Using checkpassword or SQLite would probably be the easiest solution. Or > generation of a passwd-file, but you'd have to keep it updated if users > change. If I used a passwd file on the webmail machine, running dovecot as a Proxy how would I configure dovecot on the webmail server? It has to accept connections from the squirrelmail and talk to the mailserver. I can do a rsync of the passwd file from the mail server to the webmail server using a small script whenever the file changes on the mail server. Of course the fundamental question from your response, I guess is, without a proxy why is the webmail slow? If proxy is not my problem, can you pls throw some light on why I am facing performance issues on my webmail alone? Thanks and Regards V S Rao
Re: [Dovecot] How to run Dovecot as IMAP Proxy?
On Thu, 2009-05-21 at 05:21 -0700, V S Rao wrote: > Hi All, > > I have migrated from uw-imap to Dovecot for POP3 & IMAP service. I run > webmail using squirrelmail. When running uw-imap I used to run > up.imapproxyd on the webmail server for faster responses. After > migrating to Dovecot, I find that up.imapproxyd does not work well > with dovecot. Why not? > I don't understand Databases and don't want to install mysql etc for a > simple thing like IMAP PRoxy. Do you just want it to proxy everyone to a single destination or to different servers based on username? If you want everyone to the same destination, why do you even want to use a proxy? > My question is can Dovecot be run as a IMAP PRoxy without the hassle > of setting up mysql etc? I use /etc/passwd for auth on the actual > dovecot server running IMAP & POP3. Using checkpassword or SQLite would probably be the easiest solution. Or generation of a passwd-file, but you'd have to keep it updated if users change. signature.asc Description: This is a digitally signed message part