Re: [Dovecot] Smart IMAP proxying with imapc storage
On 2011-02-28 11:20 AM, Timo Sirainen wrote: > On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: > >> Some potential (future) uses for imapc include: >> >> * Locally caching IMAP proxy >> * Maybe even fully transparent proxy to client, so at office it would >> be using Dovecot proxy while at home it would connect directly to remote >> server. (A bit tricky with SSL connections, but possible I'd think.) >> * Filtering proxy that modifies messages on the fly >> * Fixing Exchange's (and everyone else's) IMAP support by being in >> front of it > > Anyone have good ideas what this thing could be called? :) Calling it > "proxy" would confuse it with the dummy proxying feature of Dovecot. And > calling it "smart proxy" isn't very clear either what it does, > especially since I don't think it even is a proxy really. > > So far the best I've come up with is a buzzwordy "IMAP accelerator". IMAP collector? proxyfetch? proxyconnect? -- Best regards, Charles
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 28/02/2011 16:20, Timo Sirainen wrote: On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: Some potential (future) uses for imapc include: * Locally caching IMAP proxy * Maybe even fully transparent proxy to client, so at office it would be using Dovecot proxy while at home it would connect directly to remote server. (A bit tricky with SSL connections, but possible I'd think.) * Filtering proxy that modifies messages on the fly * Fixing Exchange's (and everyone else's) IMAP support by being in front of it Anyone have good ideas what this thing could be called? :) Calling it "proxy" would confuse it with the dummy proxying feature of Dovecot. And calling it "smart proxy" isn't very clear either what it does, especially since I don't think it even is a proxy really. So far the best I've come up with is a buzzwordy "IMAP accelerator". "Cache" is a good word for this kind of application? Perhaps something like "IMAP Cache"? Ed W
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 2/28/2011 9:20 AM, Timo Sirainen wrote: On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: Some potential (future) uses for imapc include: * Locally caching IMAP proxy * Maybe even fully transparent proxy to client, so at office it would be using Dovecot proxy while at home it would connect directly to remote server. (A bit tricky with SSL connections, but possible I'd think.) * Filtering proxy that modifies messages on the fly * Fixing Exchange's (and everyone else's) IMAP support by being in front of it Anyone have good ideas what this thing could be called? :) Calling it "proxy" would confuse it with the dummy proxying feature of Dovecot. And calling it "smart proxy" isn't very clear either what it does, especially since I don't think it even is a proxy really. So far the best I've come up with is a buzzwordy "IMAP accelerator". Accelerator would work since a good use could be at the edge of a company branch network... especially if it had a "prefetch = yes" option and was given sufficient disk space.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: > Some potential (future) uses for imapc include: > > * Locally caching IMAP proxy > * Maybe even fully transparent proxy to client, so at office it would > be using Dovecot proxy while at home it would connect directly to remote > server. (A bit tricky with SSL connections, but possible I'd think.) > * Filtering proxy that modifies messages on the fly > * Fixing Exchange's (and everyone else's) IMAP support by being in > front of it Anyone have good ideas what this thing could be called? :) Calling it "proxy" would confuse it with the dummy proxying feature of Dovecot. And calling it "smart proxy" isn't very clear either what it does, especially since I don't think it even is a proxy really. So far the best I've come up with is a buzzwordy "IMAP accelerator".
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 19.2.2011, at 1.03, Chris Davies wrote: > Timo Sirainen wrote: >> * Fixing Exchange's (and everyone else's) IMAP support by being in >> front of it > > Davmail makes a nice job of this. I see no reason why you couldn't > inspect its code, should you want to do so. Interesting project. However, I find it doubtful that it can implement a fully compliant IMAP server in just 1700 lines of code. A quick look shows that its LIST/LSUB handling code has been written to just handle a couple of IMAP clients, not a full correct support. I don't see it handling FETCH BODY[HEADER.FIELDS (x y z)] at all. Probably other things too.
Re: [Dovecot] Smart IMAP proxying with imapc storage
Timo Sirainen wrote: > * Fixing Exchange's (and everyone else's) IMAP support by being in > front of it Davmail makes a nice job of this. I see no reason why you couldn't inspect its code, should you want to do so. Chris
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 6.2.2011, at 1.23, Mike Korizek wrote: > Feb 6 00:17:44 hostname dovecot: master: Error: service(imap): child > 20283 killed with signal 11 (core dumps disabled) .. > # 2.1.UNSTABLE (4e4c7f982fd5): /usr/local/etc/dovecot/dovecot.conf 4e4c7f982fd5 was a buggy version that always crashed at startup. I fixed it a couple of days ago.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 01/-10/-28163 08:59 PM, Timo Sirainen wrote:
> imapc settings have moved away from plugin {} section and mail_location.
> Now instead use:
>
> mail_location = imapc:
>
> imapc_host = imap.example.com
> #imapc_port = 143 # default
>
> #imapc_user = %u # default
> imapc_password = secret
>
> imapc_ssl = no # or imaps or starttls
> imapc_ssl_ca_dir = /etc/ssl/certs
>
> Note the imapc_password change also. If passdb/userdb returned
> userdb_pass/pass previously, return now instead
> userdb_imapc_password/imapc_password.
I get the following error in the log:
Feb 6 00:17:44 hostname dovecot: auth:
static(user.n...@domain.ch,127.0.0.1): No password
Feb 6 00:17:44 hostname dovecot: imap-login: Login:
user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
mpid=20283, secured
Feb 6 00:17:44 hostname dovecot: master: Error: service(imap): child
20283 killed with signal 11 (core dumps disabled)
dovecot -n
# 2.1.UNSTABLE (4e4c7f982fd5): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-27-generic i686 Ubuntu 10.04.1 LTS
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
debug_log_path = /home/local_user/dovecot.log
default_login_user = nobody
imapc_host = mail.domain.ch
mail_debug = yes
mail_gid = local_user
mail_home = /var/run/dovecot/empty
mail_location = imapc:
mail_plugins = mail_filter
mail_uid = local_user
passdb {
args = nopassword=y userdb_imapc_password=%w
driver = static
}
plugin {
imapc_password = secret
mail_filter = mail-filter %u
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename
mail_log_fields = uid box msgid size from subject
}
protocols = imap
service mail-filter {
executable = script
/home/local_user/mydata/workspace_perl/sec_dovecot_filter/sec_dovecot_filter.pl
unix_listener mail-filter {
mode = 0666
user = root
}
user = local_user
}
ssl = no
userdb {
driver = prefetch
}
Andy idea how to resolve.
Thanks,
Mike
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Tue, 2011-02-01 at 12:25 -0700, Willie Gillespie wrote:
> On 02/01/2011 10:20 AM, Timo Sirainen wrote:
> > imapc settings have moved away from plugin {} section and mail_location.
> >
> > #imapc_user = %u # default
> > imapc_password = secret
>
> Is there a way to pass through a password like is done with the user name?
Depends on what passdb and userdb is used. With sql, ldap, passwd-file,
checkpassword it's possible by returning userdb_imapc_password=%w and
using userdb prefetch to pass it through. This works of course only with
plaintext authentication.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 02/01/2011 10:20 AM, Timo Sirainen wrote:
imapc settings have moved away from plugin {} section and mail_location.
#imapc_user = %u # default
imapc_password = secret
Is there a way to pass through a password like is done with the user name?
Re: [Dovecot] Smart IMAP proxying with imapc storage
imapc settings have moved away from plugin {} section and mail_location.
Now instead use:
mail_location = imapc:
imapc_host = imap.example.com
#imapc_port = 143 # default
#imapc_user = %u # default
imapc_password = secret
imapc_ssl = no # or imaps or starttls
imapc_ssl_ca_dir = /etc/ssl/certs
Note the imapc_password change also. If passdb/userdb returned
userdb_pass/pass previously, return now instead
userdb_imapc_password/imapc_password.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Mon, 2011-01-31 at 04:05 +0200, Timo Sirainen wrote:
> On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote:
> > > I just committed a very early initial implementation of "imapc" storage
> > > backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1
> >
> > * SSL/TLS support to remote server
Implemented. I tested with gmail:
mail_location = imapc:imap.gmail.com
plugin {
pass = secret
imapc_user = u...@gmail.com
imapc_port = 993
imapc_ssl_ca_dir = /etc/ssl/certs
# If you want to use STARTTLS (with port 143), set:
#imapc_ssl_starttls = yes
}
So in above config /etc/ssl/certs/ contains all the valid CA certs. The
code requires a valid certificate to exist, but doesn't currently verify
that the name actually matches imap.gmail.com. I really need to put that
code there soon so I won't forget it..
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: > > I just committed a very early initial implementation of "imapc" storage > > backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1 > > This is "fully functional" now, at least in theory. TODO: > > * Don't read entire message bodies into a string in memory > * Fetch more than one message at a time when it's known that more > messages are going to be fetched The above features are done now. So the performance shouldn't be too horrible anymore. > * SSL/TLS support to remote server > * Support locally stored indexes > * Error handling could be better > * Eventually make lib-storage API more asynchronous -> make imapc more > asynchronous These are not. Another TODO item is: * Add support for multiple connections to IMAP server. This is required for e.g. THREAD command to work correctly.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 01/28/2011 11:17 AM, Timo Sirainen wrote: On Sun, 2011-01-16 at 18:18 +0200, Timo Sirainen wrote: I just committed a very early initial implementation of "imapc" storage backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1 This is "fully functional" now, at least in theory. TODO: * SSL/TLS support to remote server * Don't read entire message bodies into a string in memory * Fetch more than one message at a time when it's known that more messages are going to be fetched * Support locally stored indexes * Error handling could be better * Eventually make lib-storage API more asynchronous -> make imapc more asynchronous Some potential (future) uses for imapc include: * Locally caching IMAP proxy * Maybe even fully transparent proxy to client, so at office it would be using Dovecot proxy while at home it would connect directly to remote server. (A bit tricky with SSL connections, but possible I'd think.) * Filtering proxy that modifies messages on the fly * Fixing Exchange's (and everyone else's) IMAP support by being in front of it * Could be used for sharing mailboxes across multiple Dovecot servers without shared storage * Easy "perfect migration" from any IMAP server to Dovecot with dsync +100! This is great!
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Sun, 2011-01-16 at 18:18 +0200, Timo Sirainen wrote: > I just committed a very early initial implementation of "imapc" storage > backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1 This is "fully functional" now, at least in theory. TODO: * SSL/TLS support to remote server * Don't read entire message bodies into a string in memory * Fetch more than one message at a time when it's known that more messages are going to be fetched * Support locally stored indexes * Error handling could be better * Eventually make lib-storage API more asynchronous -> make imapc more asynchronous Some potential (future) uses for imapc include: * Locally caching IMAP proxy * Maybe even fully transparent proxy to client, so at office it would be using Dovecot proxy while at home it would connect directly to remote server. (A bit tricky with SSL connections, but possible I'd think.) * Filtering proxy that modifies messages on the fly * Fixing Exchange's (and everyone else's) IMAP support by being in front of it * Could be used for sharing mailboxes across multiple Dovecot servers without shared storage * Easy "perfect migration" from any IMAP server to Dovecot with dsync
Re: [Dovecot] Smart IMAP proxying with imapc storage
On 16/01/2011 16:18, Timo Sirainen wrote: I just committed a very early initial implementation of "imapc" storage backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1 I know "me too" posts aren't that useful, but this does sound like a terrific direction. Thanks for development Ed W
[Dovecot] Smart IMAP proxying with imapc storage
I just committed a very early initial implementation of "imapc" storage
backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1
You can't really do anything except open INBOX and read mails from it,
so it's currently only intended for initial testing. It sucks in many
ways right now, but I'll be improving it.
The idea is that you could set for example:
mail_location = imapc:imap.gmail.com
And then Dovecot could act as a proxy to gmail. It won't actually work
currently with gmail, because there's no SSL support for outgoing
connections.
Currently index files are forcibly disabled, but it would be easy to
enable them, allowing Dovecot to do caching locally to improve
performance. In future perhaps it will also support caching message
headers/bodies to avoid unnecessary traffic.
Besides the mail_location setting, you'll need to also forward the
user's password to imap process in "pass" userdb extra field. How to do
that depends on what passdb/userdb you're using. While testing you could
simply set a static password:
plugin {
pass = yourpassword
}
signature.asc
Description: This is a digitally signed message part
