[Dovecot] dovecot ldap search result ordering?

[chasye]

I use openLDAP as dovecot pass/user db.
here is main part of dovecot-ldap.conf:

pass_attrs = mail=user, uid=userdb_home=/home/xadmin/%d/%$,userPassword=password
pass_filter = (&(objectClass=posixAccount)(uid=%u))
default_pass_scheme = SSHA

user_attrs = mail=user, uid=home=/home/xadmin/%d/%$
user_filter = (&(objectClass=posixAccount)(|(mail=%u)(mailAlias=%u)))

It works fine, but some account failed to get their domain in pass_attrs.
I check the log.
when it success, the result order is "mail, uid, userPassword"
when it failed, the order is "userPassword, uid, mail"

we just use a username to login. I think dovecot cant get the domain before
username changed to "usern...@domain". So can dovecot sort the result 
order and How?



-- 
?s?o
???X
???T?e??
???I

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Angelo Chen]

hi, i just tried 3.x, it looks like i can't drag and drop sub folders, and one 
problem is, imap folders are on top, and local folders(imported from outlook) 
are below, and it has more than 300, drag and drop toward imap folders on top 
is very difficult.

On Jun 26, 2010, at 11:24 AM, Stan Hoeppner wrote:

> Charles Marcus put forth on 6/25/2010 7:25 AM:
>> On 2010-06-25 3:19 AM, Stan Hoeppner wrote:
>>> It's interesting that you weren't able/willing to track down the 
>>> source of the problem. It's also interesting how you mention "single
>>> drag and drop". In my experience, you can't drag/drop _folders_ in
>>> Thunderbird at all. All the migrations I've done this way required
>>> creating new folders on the IMAP server (Dovecot), group selecting
>>> all of the mail in the respective source folder, and selecting "copy"
>>> or "move" to the destination folder. I couldn't find a way to
>>> drag/drop a folder of group of folders. Care to share your secret?
>> 
>> No secret - TB used to not like dragging entire folders at all, but
>> slowly got better - and with 3.0, it got much better.
>> 
>> I just tested some (moving about 4GB Sent folder to a gmail account)
>> with 3.1rc2, and it moved everything perfectly, although it took a few
>> hours...
>> 
>> There were no sub folders though... I'll test that next...
> 
> Ahh, ok.  I think the last time I did such a migration was in the 2.x days,
> which actually wasn't all that long ago. Last fall maybe?  Oct/Nov 2009?
> Somewhere around there IIRC.
> 
> -- 
> Stan
> 
> 

Re: [Dovecot] Thunderbird's (in)ability to mass-upload mails to IMAP (was: Re: importing outlook express messages to dovecot imap server)

[Stan Hoeppner]

Patrick Nagel put forth on 6/25/2010 9:27 AM:
> Hi Stan,
> 
> On 2010-06-25 07:19 UTC Stan Hoeppner wrote:
>> Patrick Nagel put forth on 6/25/2010 1:02 AM:
>>> On 2010-06-25 03:51 UTC Stan Hoeppner wrote:
> [...]
 The first method is a single step process and is reliable.
>>>
>>> I tried something similar (with Thunderbird) once, and it caused a lot of
>>> trouble. We only had around 1.3 GB, IIRC, but thousands of folders. Here
>>> is a blog article that I wrote after I got it done with a perl script:
>>>
>>> https://patrick-nagel.net/blog/archives/77
>>
>> It's interesting that you weren't able/willing to track down the source of
>> the problem.
> 
> I spent some time on it, as I wrote in the blog article, but then there were 
> just more important things to get done. The most annoying thing was, that 
> there was no error message whatsoever. Error handling doesn't seem to be one 
> of Thunderbird's strengths (also things like 
> https://bugzilla.mozilla.org/show_bug.cgi?id=257735 come to mind).
> 
>> It's also interesting how you mention "single drag and
>> drop".  In my experience, you can't drag/drop _folders_ in Thunderbird at
>> all.  All the migrations I've done this way required creating new folders
>> on the IMAP server (Dovecot), group selecting all of the mail in the
>> respective source folder, and selecting "copy" or "move" to the
>> destination folder.  I couldn't find a way to drag/drop a folder of group
>> of folders.  Care to share your secret?
> 
> I don't know when you tried it, but with the version I used at the time 
> (2.0.0.17? .18?), it was possible to copy one folder at a time from one 
> account to another, by drag & drop. 99% of the folders were inside one top 
> level folder, which I attempted to drag & drop, and which Thunderbird than 
> began to upload - only to silently fail after a few sub-folders.

Hmm.  At that time I was trying to copy folders into the main account tree,
which is how they were setup on the other account.  I just tried it in 3.0.5
and it works with an empty test folder created in Local Folders and dragged
into the main tree of my IMAP account.  IIRC this didn't work back in 2.x.  I
know I wasn't able to get it to work, that is.  However, I wasn't working with
Local Folders, but another IMAP account as the source.  Maybe it was and
IMAP->IMAP drag drop issue in 2.x?

-- 
Stan

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Stan Hoeppner]

Charles Marcus put forth on 6/25/2010 7:25 AM:
> On 2010-06-25 3:19 AM, Stan Hoeppner wrote:
>> It's interesting that you weren't able/willing to track down the 
>> source of the problem. It's also interesting how you mention "single
>> drag and drop". In my experience, you can't drag/drop _folders_ in
>> Thunderbird at all. All the migrations I've done this way required
>> creating new folders on the IMAP server (Dovecot), group selecting
>> all of the mail in the respective source folder, and selecting "copy"
>> or "move" to the destination folder. I couldn't find a way to
>> drag/drop a folder of group of folders. Care to share your secret?
> 
> No secret - TB used to not like dragging entire folders at all, but
> slowly got better - and with 3.0, it got much better.
> 
> I just tested some (moving about 4GB Sent folder to a gmail account)
> with 3.1rc2, and it moved everything perfectly, although it took a few
> hours...
> 
> There were no sub folders though... I'll test that next...

Ahh, ok.  I think the last time I did such a migration was in the 2.x days,
which actually wasn't all that long ago. Last fall maybe?  Oct/Nov 2009?
Somewhere around there IIRC.

-- 
Stan

Re: [Dovecot] Thunderbird problem

[Stan Hoeppner]

Steffen Kaiser put forth on 6/25/2010 7:01 AM:

> 2) thunderbird opened too many simultaneous connections to the server. I
> do not remember where they would blocked or terminated, but in some
> cases thunderbird did not seem to detect this failure

This is a pet peeve of mine.  Recent TB revs default to opening 5 IMAP
connections.  Some time ago I did more than cursory testing and found that
TBird only uses 1 of the 5.  The other 4 just sit there idle, adding clutter
to the process list and eating a small amount of RAM.  If it weren't for
Linux' share memory management these processes would be eating a whole ton of
RAM on busy Dovecot servers.

Anyway, since testing I force all my user TBird configs to a single IMAP
connection.  No problems.

-- 
Stan

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Timo Sirainen]

On 26.6.2010, at 3.08, Brandon Davidson wrote:

> We run a proxy instance on each of the webmail hosts, with the communication
> between the web application add the proxy being done in cleartext, but with
> the proxy -> Dovecot communication secured over SSL. Besides preventing a
> lot of extra SSL handshakes and login/logout actions,

Yes, SSL handshakes are extra. Although SSL supports some kind of quick 
renegotiation too, but Dovecot doesn't support that yet. No one's ever 
requested it..

> it also helps tie a
> user session to a single backend node in our pool of IMAP servers. It seems
> like there might also be other benefits to having Dovecot not tear down all
> of the user session state between page loads.

Some, but I suspect mostly CPU performance related (which usually doesn't 
matter).

> A lot of this stuff might be nice to see in the Director some day. If there
> was an option to not immediately close the Director proxy's backend
> connections when the user logs out, (ie leave the connection active and
> logged in for X seconds, and reuse it if the user logs in to the Director
> again),

It hasn't really ever been in my plans to do anything like this. A separate 
imapproxy program for this is probably better. At least I don't really see any 
benefits compared to it if it was done by Dovecot.

I mainly just wish someone would give some kind of real numbers when running 
with and without imapproxy with Dovecot (and not some other server they used to 
run years ago) when they're talking about imapproxy helping the performance a 
lot. So far the best reports I've heard are "imapproxy improves a little bit, 
but adds too much complexity/fragility". So I'm a bit sceptical about using it.

I've also recently heard from a large installation who was complaining about 
Dovecot's memory usage (compared to Courier). imapproxy would definitely make 
their memory usage a lot higher, since the connections would stay around for 
longer. Although I'm still wondering why Dovecot would take so much more memory 
with large maildirs .. should look into that some day.

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Brandon Davidson]

Timo,

On 6/24/10 4:23 AM, "Timo Sirainen"  wrote:
>> 
>> I'd recommend also installing and configuring imapproxy - it can be
>> beneficial with squirrelmail.
> 
> Do you have any about a real world numbers about installation with and without
> imapproxy?

We run imapproxy behind our Roundcube instance, and our old in-house Perl
mail system has a custom equivalent written in C that also does some caching
of folder metadata and message headers.

We run a proxy instance on each of the webmail hosts, with the communication
between the web application add the proxy being done in cleartext, but with
the proxy -> Dovecot communication secured over SSL. Besides preventing a
lot of extra SSL handshakes and login/logout actions, it also helps tie a
user session to a single backend node in our pool of IMAP servers. It seems
like there might also be other benefits to having Dovecot not tear down all
of the user session state between page loads.

A lot of this stuff might be nice to see in the Director some day. If there
was an option to not immediately close the Director proxy's backend
connections when the user logs out, (ie leave the connection active and
logged in for X seconds, and reuse it if the user logs in to the Director
again), and if the auth caching works as well as you say, then I could
definitely see a day where we replace imapproxy with a director instance on
the webmail host.

-Brad

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Noel Butler]

On Sat, 2010-06-26 at 01:02 +0100, Timo Sirainen wrote:

> On 26.6.2010, at 0.57, Noel Butler wrote:
> 
> > As I mentioned earlier It doesnt with squirrelmail, I tried it on
> > production and the I/O increased a bit, watching logging alone was
> > giving me a 
> > headache (no, not in debug mode either)..
> 
> How much is "a bit" that the I/O increased? Less than 10%?
> 


perhaps around the 10-15% (see below)


> > So I thought I'd try it on a dev box, in case the cache wasnt large
> > enough, but nope, incidentally imap with evolution works as you indicate
> > it should, but webmail, not a hope in . it is login -action logout
> > constantly, imapproxy avoids all this crud, so it obviously does
> > something differently, maybe you find that and incorporate it in
> > dovecot :) that way all this login logout stuff is only seen in say,
> > debug mod?
> 
> It still sounds like the only problem you have is the excessive log 
> messages?.. Sure, Dovecot doesn't even try to get rid of those.
> 


this likely is the sole reason for the I/O increase, but yes. Oh and the
local (replicated) DB was seeing maybe 40 requests a second rather than
only a few, no big deal there though.

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Noel Butler]

apologies, I sent this direct to Eric not the list ore OP, my bad :)
 
On Fri, 2010-06-25 at 13:46 -0700, Eric Shubert wrote:





> 
> > now i would also need to edit the dovecot.conf file
> > could you guide me on what changes do i need to dovecot.conf file ?
> 
> Others on this list would be more help than me regarding this. From the 
> looks of it though (after browsing the dovecot.conf file, which is well 
> documented btw) I don't think you'd need to change a thing. We'll 
> certainly find out if that's the case or not. I'd just give it a go. 
> Perhaps on a test account initially. ;)
> 

With Qmail (in particular if using vpopmail) from memory you need to
include  first_valid_uid  and  first_valid_gid  options
i
<>

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Timo Sirainen]

On 26.6.2010, at 0.57, Noel Butler wrote:

> As I mentioned earlier It doesnt with squirrelmail, I tried it on
> production and the I/O increased a bit, watching logging alone was
> giving me a 
> headache (no, not in debug mode either)..

How much is "a bit" that the I/O increased? Less than 10%?

> So I thought I'd try it on a dev box, in case the cache wasnt large
> enough, but nope, incidentally imap with evolution works as you indicate
> it should, but webmail, not a hope in . it is login -action logout
> constantly, imapproxy avoids all this crud, so it obviously does
> something differently, maybe you find that and incorporate it in
> dovecot :) that way all this login logout stuff is only seen in say,
> debug mod?

It still sounds like the only problem you have is the excessive log messages?.. 
Sure, Dovecot doesn't even try to get rid of those.

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Noel Butler]

On Fri, 2010-06-25 at 13:49 +0100, Timo Sirainen wrote:

> On Fri, 2010-06-25 at 14:44 +0200, Amon Ott wrote:
> 
> > For whatever it is worth, we use imapproxy, because it allows us to use 
> > one-time passwords for webmail users. One login, not many.
> 
> With large enough auth cache, I think that should also work with Dovecot
> directly (although if the session takes longer than auth_cache_ttl, the
> user needs to re-login).
> 
> 


As I mentioned earlier It doesnt with squirrelmail, I tried it on
production and the I/O increased a bit, watching logging alone was
giving me a 
headache (no, not in debug mode either)..

So I thought I'd try it on a dev box, in case the cache wasnt large
enough, but nope, incidentally imap with evolution works as you indicate
it should, but webmail, not a hope in . it is login -action logout
constantly, imapproxy avoids all this crud, so it obviously does
something differently, maybe you find that and incorporate it in
dovecot :) that way all this login logout stuff is only seen in say,
debug mod?
Though the overhead for running imapproxy is non existent it is still an
needed devil in a busy environment.







<>

Re: [Dovecot] Pigeonhole Managesieve issue with Dovecot 2.0

[Tim Traver]

On 6/25/2010 3:26 PM, Stephan Bosch wrote:
> Basically, one of the top items on our TODO list. Changes to the
> Dovecot service configuration in Dovecot v2.0 broke the ability to
> dynamically obtain the ManageSieve capabilities before the user has
> logged in. We are close to fixing this, but until then you can use the
> managesieve_sieve_capability setting to set this response manually
> (refer to INSTALL file). 

Stephan,

Thanks for the workaround. That did the trick, although I would like it
to dynamically determine that list, because I don't know if my list is a
complete one...

Tim.

Re: [Dovecot] Pigeonhole Managesieve issue with Dovecot 2.0

[Stephan Bosch]

Tim Traver wrote:

Hi all,

ok, I am running version v2.0.beta6 and the latest pigeonhole build to
run managesieve and the sieve delivery filters...

I am getting the following response from the managesieve port when
connecting :





Am I missing something?



A little.


what might cause it to not broadcast the sieve implementations that it has?



Basically, one of the top items on our TODO list. Changes to the Dovecot 
service configuration in Dovecot v2.0 broke the ability to dynamically 
obtain the ManageSieve capabilities before the user has logged in. We 
are close to fixing this, but until then you can use the 
managesieve_sieve_capability setting to set this response manually 
(refer to INSTALL file).


Regards,

Stephan.

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Eric Shubert]

Rajesh M wrote:

Rajesh M wrote:

Rajesh M wrote:

eric

i studied LDA a bit

 >

if i use lda that means all my 5000+ users' email index files will be
continuously updated when every email arrives -- means a lot of writes
to
disk ... is that correct ?

Yes, but I think you make it sound worse than it is. Updating the index
as each email arrives doesn't involve rebuilding the whole index like
what happens at login. It only updates what's there instead of
rebuilding the whole thing. I can't speak from experience, but I expect
that it's very efficient, and you'll never notice any increased load on
the machine. Perhaps the equivalent of adding journaling to a
filesystem. You can ask the dovecot list to verify this.

Also, it just occurred to me that if you're using ext3, you should be
sure to optimize the filesystem. Off hand, I'd use the noatime and
nodiratime mount options (man mount) and dir_index filesystem option
(man tune2fs).


if yes then i dont want that since only around 200 or so people out of
the
above 5000 use webmail.

Just to be clear, it affects all imap usage, not only webmail (although
your uses may not use imap other than webmail).


i will study a some more and revert

I hope you can figure a way to try this out. I expect that QMT's move
to
dovecot in the future will include the deliver LDA, and your work here
will be a big help getting us there.

Let me know if I can be of any further help.

--
-Eric 'shubes'



hi eric

few questions

i wil enable lda exclusively for this one 30 gb email box using .qmail
file see the performance improvement .. is that ok ?

I believe that would be fine.


can i safely enable atime for this user maildir  ?

Sure. That's simply a performance consideration. I don't believe that
atime is used for much of anything, so either atime or noatime is safe.
noatime is just a little faster because the inode (where that piece of
information is stored) doesn't need to be updated each time a file is
accessed.


thanks for your help

Sure. I'm eager to see your results.

--
-Eric 'shubes'




hi eric

this is what i plan to do

as per
http://wiki.dovecot.org/LDA/Qmail

i need to enter the following line in a .qmail file under the Maildir
directory of the user
|/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -d $...@$user


the location is /usr/libexec/dovecot/deliver on my system.


i would add an email forward using qmailadmin web interface (which creates
the .qmail file) and then manually edit the .qmail file and put the above
line in it (so that ownership and permissions are taken care off)


That sounds ok. The permissions/ownership I'm seeing on .qmail files is:
-rw---   1 vpopmail vchkpw   101 Mar 26  2008 .qmail


now i would also need to edit the dovecot.conf file
could you guide me on what changes do i need to dovecot.conf file ?


Others on this list would be more help than me regarding this. From the 
looks of it though (after browsing the dovecot.conf file, which is well 
documented btw) I don't think you'd need to change a thing. We'll 
certainly find out if that's the case or not. I'd just give it a go. 
Perhaps on a test account initially. ;)


--
-Eric 'shubes'

[Dovecot] Pigeonhole Managesieve issue with Dovecot 2.0

[Tim Traver]

Hi all,

ok, I am running version v2.0.beta6 and the latest pigeonhole build to
run managesieve and the sieve delivery filters...

I am getting the following response from the managesieve port when
connecting :

Escape character is '^]'.
"IMPLEMENTATION" "Dovecot"
"SIEVE" ""
"SASL" "PLAIN"
"STARTTLS"
"VERSION" "1.0"
OK "Dovecot ready."


But, I think that the SIEVE line is suppose to have all of the supported
sieve functions like so :

"SIEVE" "comparator-i;ascii-numeric fileinto reject vacation imapflags
notify include envelope body relational regex subaddress copy"

I modified the dovecot sieve config file to try and add (although they
should all be added by default), and it still comes up blank...

Am I missing something?

what might cause it to not broadcast the sieve implementations that it has?

Thanks,

Tim.

Re: [Dovecot] Plugin Handle input messages

[Timo Sirainen]

On Mon, 2010-06-21 at 16:43 -0300, Alex Baule wrote:
> if (imail->data.stream != NULL ||
>(_mail->uid == 0 && zuser->save_handler == NULL)) {
>return zmail->super.get_stream(_mail, hdr_size, body_size,
>   stream_r);
>}
> 
> I think this is the line that return the stream that is the stream sent by
> client.
> 
> I try to change this line and put inside this IF , a call to my function to
> handle with the input email from client. But i got some errors about the
> input part.

I'm not really sure what you mean by this. What input are you sending
where? Show some example non-working code?

> In other email, Timo said, to get the S and W flags in the email name, i
> need to intercept the email input from client to dovecot, in this part is
> done the W and S calculation.
> 
> Timo, where is the part that i can get the input from client ?

IIRC you wanted to add some extra headers to the mail (and you wanted it
to be visible to clients, right? if not, you could do it similarly than
how mbox hides some headers). All mails are saved via:

int mailbox_save_begin(struct mail_save_context **ctx, struct istream
*input);

And the input comes from there. You need to replace that input with your
own input stream that contains the wanted headers and other changes
(probably something built with istream-concat and
istream-header-filter). You can replace the input by overriding
mailbox.save_begin() method, for example see acl plugin how it uses
acl_save_begin(). Maybe something like:

static int
your_save_begin(struct mail_save_context *ctx, struct istream *input)
{
struct mailbox *box = ctx->transaction->box;
struct your_mailbox *abox = YOUR_CONTEXT(box);
struct istream *new_input;
int ret;

new_input = build_changed_input(input);
ret = abox->module_ctx.super.save_begin(ctx, new_input);
i_stream_unref(&new_input);
return ret;
}

Re: [Dovecot] Expunge email Plugin

[Timo Sirainen]

On Mon, 2010-06-21 at 17:27 -0300, Alex Baule wrote:
> Hello everyone.
> 
> 
> I Need to know when a email is expunge, but i don't need change this action.
> I Need to call another action + the default action.
> 
> There is a plugin to do this ?

In v2.0 there is a notify plugin which you can pretty easily use to
catch expunge events. mail_log plugin also uses it.

Re: [Dovecot] Dovecot-2.0 conf misc questions

[Timo Sirainen]

On Wed, 2010-06-23 at 18:57 +0200, Thomas Hummel wrote:
> On Wed, Jun 23, 2010 at 04:40:35PM +0100, Timo Sirainen wrote:
> 
> Hello Timo,
> 
> > >   doveconf: Fatal: Error in configuration file 
> > > /usr/local/dovecot-1.2.12/etc/dovecot.conf line 176: Unknown setting: 
> > > process_limit
> > 
> > This was caused by the old settings translator. What was in line 176?
> 
>   login_max_processes_count = 1024

That alone doesn't break it with my tests. Could you send me the entire
config file so I could get this fixed?

> > imap process_limit + pop3 process_limit + whatever other protocols you
> > have enabled and + their process_limit.
> 
> Ok. Can you explain what are :
> 
>   #default_process_limit = 100 -> which processes ?
>   #default_client_limit = 1000 -> 1 client == 1 remote ip ?
>   #default_vsz_limit = 256M -> ?

These are all defaults for service {} blocks. Some services override
them, most keep the defaults. So:

default_process_limit: Nearly all processes of same type.

default_client_limit: This is mainly for non-imap/pop3 processes, since
they have overridden it. 1 client = 1 connection, usually from another
Dovecot process.

default_vsz_limit: This is the default for almost all processes. If the
processes' VSZ memory area grows beyond that, kernel kills it.

You're not normally supposed to change or really even understand
those :) I've added a few checks where Dovecot complains if the limits
aren't right. If they're too low, you'll get an error message explaining
it. In that case you could also report it to me and I'll see if I can
add an early warning for that.

> > >   . why is this default not in 10-auth.conf file ?
> > 
> > You mean why isn't there an example remote {} block there? 
> 
> No. I mean, why isn't the 'disable_plaintext_auth' directive not in the
> auth.conf file, since it's an auth related directive ?

Oh. Yeah, I guess it is. Moved:
http://hg.dovecot.org/dovecot-2.0/rev/5326d6b2f36e

> > >   . would I have been allowed to do, for instance, in that file at the 
> > > same line
> > > 
> > > protocol imap {
> > >  remote  {
> > >  disable_plaintext_auth = no
> > > }
> > > 
> > > ?
> > 
> > Yes.
> 
> But didn't you just say that "Currently auth settings don't support 
> local/remote blocks" ?

disable_plaintext_auth isn't really "auth setting" :) It's handled by
login processes, not auth processes. That's also why I didn't think of
putting it into auth.conf first.

> > > Finally, would it make sense to declare other auth listeners than the two
> > > listed by default in the 10-master.conf file ?
> > 
> > The defaults also have one example auth(-client) socket commented out
> > for Postfix. You can create more of them if you want, but unless
> > something actually uses them they're a bit pointless.
> 
> Ok, maybe I meant "the auth-userdb" unix listener is mandatory" : is it
> ?

It's not mandatory if you don't use anything that needs it (dovecot-lda,
some doveadm commands, etc).

Re: [Dovecot] auth_cache_size

[Timo Sirainen]

On Fri, 2010-06-25 at 08:09 -0700, Daniel L. Miller wrote:
> Let's start small.  Assume ten users, all connected - and everybody also 
> likes looking at everybody else's mailbox.  So there are ten clients, 
> each with at least ten connections (I say at least because each folder 
> viewed within an account is its own connection, right?).
> 
> How is the auth_cache used?  If I have ten user/pass combinations - is 
> that all that is stored?

It caches passdb and userdb lookups+replies. Both are independent of
each others.

> Or are there connection-specific entries as well?

No.

> In other words, for the above example, do I have 10 x 250 bytes 
> yields auth_cache_size=3, or do I need 10 x 10 x 250 yields 
> auth_cache_size=30 - or am I totally off?

10 x 250. But negative hits ("user not found") are also cached (unless
you disable them) and they can also increase the cache usage.

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Michael M. Slusarz]

Quoting Eric Rostetter :


Quoting "Daniel L. Miller" :

I know when I was playing web clients - particularly squirrelmail -  
there was a definite perceived improvement - but I never measured it.


Webmail clients are basically stateless.  Over time, they have "cached"
some info for performance, but basically they are still stateless.  So
almost any time you click on a link dealing with an email (delete, purge,
next message, forward, etc) it has to open a _new_ connection to the imap
(or pop) server, which means a new login.

Simple proxies (like say perdition) don't help, as each connection will
still be a new login.

Some proxies (like imapproxy) however can keep a login session open, with
the proxy caching the authentication issue.  For each connection after
the first, it can do the authentication itself and use the existing login
connection to the pop/imap server, avoiding a login for each operation after
the first (until a timeout is reached).  By avoiding the constant  
login/logout

cycle, it will generally perform at least slightly better with the proxy
than without (no new connection overhead, no login and logout overhead, same
over head most everywhere else unless the proxy's authentication is slow).


However, imapproxy (at least as of v1.2.6) is still less than  
desirable.  Even though you are saving on the authentication overhead,  
every time your stateless client (i.e. webmail) connects it still  
needs to potentially issue a CAPABILITY command and/or other enabling  
commands (e.g. 'ENABLE QRESYNC').  So that's still a bunch of overhead  
that could potentially be saved.


Solution: make the imapproxy layer visible to the client.  Patches  
were added to imapproxy v1.2.7 to do just that.  When IMAPPROXY is  
being used, XIMAPPROXY is announced via the IMAP capability string.   
More important, if the cached proxy connection is successfully reused,  
a XPROXYREUSE status response is returned after the authentication  
commands are completed.  If this status return is observed by the  
client, it knows that it can reuse its cached information about the  
connection (if it exists locally on the client) without needing to  
check on CAPABILITIES or issuing ENABLING commands.  Additionally, it  
can prevent unnecessary initialization code on the client side that  
only needs to be done when a user logs into the server.


IMP 5 (more specifically, the Horde 4: Horde_Imap_Client socket  
driver) does just this to prevent unnecessary traffic to the IMAP  
server.


michael

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Eric Rostetter]

Quoting Timo Sirainen :

But with auth cache enabled, there is no extra database load. The  
index files are also most likely in OS's cache (assuming local  
disk), so no extra disk I/O to read them either. I'm sure it's a bit  
more extra CPU usage, but I'm not all that certain that it's really  
that a big of a difference with Dovecot.


Just the amount of logs generated could be significant for disk load...
Not to mention network overhead and all the rest...

Again, your millage may vary...

--
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Eric Rostetter]

Quoting "Daniel L. Miller" :


On 6/24/2010 4:23 AM, Timo Sirainen wrote:
I'd recommend also installing and configuring imapproxy - it can  
be beneficial with squirrelmail.


Do you have any about a real world numbers about installation with  
and without imapproxy?


What, you want me to actually back up that statement with data?!   
Who do you think I am?!  Never mind - don't answer that.


I don't have any numbers either, but...

I know when I was playing web clients - particularly squirrelmail -  
there was a definite perceived improvement - but I never measured it.


Webmail clients are basically stateless.  Over time, they have "cached"
some info for performance, but basically they are still stateless.  So
almost any time you click on a link dealing with an email (delete, purge,
next message, forward, etc) it has to open a _new_ connection to the imap
(or pop) server, which means a new login.

Simple proxies (like say perdition) don't help, as each connection will
still be a new login.

Some proxies (like imapproxy) however can keep a login session open, with
the proxy caching the authentication issue.  For each connection after
the first, it can do the authentication itself and use the existing login
connection to the pop/imap server, avoiding a login for each operation after
the first (until a timeout is reached).  By avoiding the constant login/logout
cycle, it will generally perform at least slightly better with the proxy
than without (no new connection overhead, no login and logout overhead, same
over head most everywhere else unless the proxy's authentication is slow).

That's why a connection/authentication caching proxy is generally
recommended for webmail setups.  It also keeps the log files from filling
up on the pop/imap server with the constant login/logout log lines.

This is also why you can get a speedup even if the proxy is on the same host
as the pop/imap server (assume sufficient memory and other resources).


Daniel



--
Eric Rostetter
The Department of Physics
The University of Texas at Austin

Go Longhorns!

Re: [Dovecot] Thunderbird's (in)ability to mass-upload mails to IMAP (was: Re: importing outlook express messages to dovecot imap server)

[Charles Marcus]

On 2010-06-25 10:27 AM, Patrick Nagel wrote:
> I don't know when you tried it, but with the version I used at the time 
> (2.0.0.17? .18?), it was possible to copy one folder at a time from one 
> account to another, by drag & drop. 99% of the folders were inside one top 
> level folder, which I attempted to drag & drop, and which Thunderbird than 
> began to upload - only to silently fail after a few sub-folders.

I had similar experiences with those older version... but that was with
a Courier-imap server...

-- 

Best regards,

Charles

[Dovecot] auth_cache_size

[Daniel L. Miller]

I was going to ask how much space I should allocate for auth_cache_size 
- but decided to read the wiki first.  Now I'll be more specific -


Wiki says about 50 bytes per passdb, and up to 200 bytes for userdb - so 
my arithmetic says each combined entry needs 250 bytes budgeted.  
Assuming that's correct, I need some clarification.


Let's start small.  Assume ten users, all connected - and everybody also 
likes looking at everybody else's mailbox.  So there are ten clients, 
each with at least ten connections (I say at least because each folder 
viewed within an account is its own connection, right?).


How is the auth_cache used?  If I have ten user/pass combinations - is 
that all that is stored?  Or are there connection-specific entries as 
well?  In other words, for the above example, do I have 10 x 250 bytes 
yields auth_cache_size=3, or do I need 10 x 10 x 250 yields 
auth_cache_size=30 - or am I totally off?

--
Daniel

[Dovecot] Thunderbird's (in)ability to mass-upload mails to IMAP (was: Re: importing outlook express messages to dovecot imap server)

[Patrick Nagel]

Hi Stan,

On 2010-06-25 07:19 UTC Stan Hoeppner wrote:
> Patrick Nagel put forth on 6/25/2010 1:02 AM:
> > On 2010-06-25 03:51 UTC Stan Hoeppner wrote:
[...]
> >> The first method is a single step process and is reliable.
> > 
> > I tried something similar (with Thunderbird) once, and it caused a lot of
> > trouble. We only had around 1.3 GB, IIRC, but thousands of folders. Here
> > is a blog article that I wrote after I got it done with a perl script:
> > 
> > https://patrick-nagel.net/blog/archives/77
> 
> It's interesting that you weren't able/willing to track down the source of
> the problem.

I spent some time on it, as I wrote in the blog article, but then there were 
just more important things to get done. The most annoying thing was, that 
there was no error message whatsoever. Error handling doesn't seem to be one 
of Thunderbird's strengths (also things like 
https://bugzilla.mozilla.org/show_bug.cgi?id=257735 come to mind).

> It's also interesting how you mention "single drag and
> drop".  In my experience, you can't drag/drop _folders_ in Thunderbird at
> all.  All the migrations I've done this way required creating new folders
> on the IMAP server (Dovecot), group selecting all of the mail in the
> respective source folder, and selecting "copy" or "move" to the
> destination folder.  I couldn't find a way to drag/drop a folder of group
> of folders.  Care to share your secret?

I don't know when you tried it, but with the version I used at the time 
(2.0.0.17? .18?), it was possible to copy one folder at a time from one 
account to another, by drag & drop. 99% of the folders were inside one top 
level folder, which I attempted to drag & drop, and which Thunderbird than 
began to upload - only to silently fail after a few sub-folders.

Patrick.

-- 
Key ID: 0x86E346D4http://patrick-nagel.net/key.asc
Fingerprint: 7745 E1BE FA8B FBAD 76AB 2BFC C981 E686 86E3 46D4


signature.asc
Description: This is a digitally signed message part.

Re: [Dovecot] imap without smtp

[John S]

On 6/25/2010 10:05 AM, Charles Marcus wrote:

On 2010-06-25 9:58 AM, Angelo Chen wrote:
   

possible to setup a imap server without smtp? just want to store some old 
emails.
 

Of course, one has nothing to do with the other.

   
A lot of trouble, more than just importing them into your e-mail client! 
You would require no configuration, or very little, other than putting 
the e-mails in a home/usernameofpersononlinux/.mail folder and just 
logging into imap with your usernameofpersononlinux's password.

Re: [Dovecot] imap without smtp

[Charles Marcus]

On 2010-06-25 9:58 AM, Angelo Chen wrote:
> possible to setup a imap server without smtp? just want to store some old 
> emails.

Of course, one has nothing to do with the other.

-- 

Best regards,

Charles

[Dovecot] imap without smtp

[Angelo Chen]

hi,

possible to setup a imap server without smtp? just want to store some old 
emails.

Angelo

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Charles Marcus]

On 2010-06-25 8:44 AM, Amon Ott wrote:
> For whatever it is worth, we use imapproxy, because it allows us to use 
> one-time passwords for webmail users. One login, not many.

That's what I meant by cleaner logs... ;)

On 2010-06-25 8:49 AM, Timo Sirainen wrote:
> With large enough auth cache, I think that should also work with 
> Dovecot directly (although if the session takes longer than
> auth_cache_ttl, the user needs to re-login).

Great! Now to test this... thanks Timo... :)

-- 

Best regards,

Charles

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Xavi Montero]

It's interesting that you weren't able/willing to track down the 
source of the problem. It's also interesting how you mention "single

drag and drop". In my experience, you can't drag/drop _folders_ in
Thunderbird at all. All the migrations I've done this way required


I migrated to my new server by drag/drop from Thunderbird. Using IMAP. 
Using Thunderbird 2.0.0.24. No command line needed to convert from 
server to server, all done via the GUI of the client.


Original server was mbox, destination is dovecot with maildirs.

The ONLY problem I had was with folders which contained a "dot" in the 
name. An old folder called "2010-june.providers" was understood as a 
folder called "providers" inside a folder called "2010-june" due to the 
name convention used in the dovecot server. The copy failed there and 
the destination dir was not behaving well.


Just deleted the erroneous destination folder, I renamed to something 
like "2010-june_providers" and retried from there on.


I have migrated a 3 GB account (mine) in this way.

For security I did not do "all at a time". I just moved 10 folders, upon 
finish 10 more folders and so on.


TB handles well if the destination folder exists. Example:

Supose I have:

old/2001 to old/2009

Spoose in the DST you already have old/1996 to old/2000

By dragging "old" onto "old" it will copy from 2001 to 2009 inside "old" 
at the same level than 1996 to 2000.


NEVERTHELESS IT IS TRUE that sometimes TB refuses to do the drag-n-drop. 
At this time, just close TB and reopen it and Drag-n-drop worked again.


Hope to help.
Xavi.

--
Xavier Montero - 93 589 71 91 - 630 59 01 62 - xmont...@dsitelecom.com

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Timo Sirainen]

On Fri, 2010-06-25 at 14:44 +0200, Amon Ott wrote:

> For whatever it is worth, we use imapproxy, because it allows us to use 
> one-time passwords for webmail users. One login, not many.

With large enough auth cache, I think that should also work with Dovecot
directly (although if the session takes longer than auth_cache_ttl, the
user needs to re-login).

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Amon Ott]

On Friday 25 June 2010 wrote Charles Marcus:
> On 2010-06-24 9:18 PM, Timo Sirainen wrote:
> > But with auth cache enabled, there is no extra database load. The
> > index files are also most likely in OS's cache (assuming local disk),
> > so no extra disk I/O to read them either. I'm sure it's a bit more
> > extra CPU usage, but I'm not all that certain that it's really that a
> > big of a difference with Dovecot.
>
> We have very few web users, so I really didn't see anything noticeable
> performance wise, but I'll always use it just for the much cleaner logs.

For whatever it is worth, we use imapproxy, because it allows us to use 
one-time passwords for webmail users. One login, not many.

Amon Ott
-- 
Dr. Amon Ott - m-privacy GmbH
Am Köllnischen Park 1, 10179 Berlin
Tel: +49 30 24342334
Fax: +49 30 24342336
Web: http://www.m-privacy.de
Handelsregister:
 Amtsgericht Charlottenburg HRB 84946
Geschäftsführer:
 Dipl.-Kfm. Holger Maczkowsky,
 Roman Maczkowsky
GnuPG-Key-ID: EA898571

Re: [Dovecot] Thunderbird problem

[Frank Bonnet]

Thanks for your *complete* answer Charles .

On 06/25/2010 02:32 PM, Charles Marcus wrote:

On 2010-06-25 7:48 AM, Frank Bonnet wrote:

Some users had occasionally a problem with Thunderbird , some random
emails with attachement(s) cannot be read anymore
the email appears empty and TB seems to enter in an infinite loop
saying it is downloading the message

The only solution I found was :

1 - stop thunderbird on the client machine
2 - remove the .imap user's directory
3 - restart thunderbird on the client machine

After doing that all emails re-appear normally and attachements
are readable/downloadable again.

ALL our clients use the IMAP(S) protocol.

Anyone had the same problem ?


Yes, this is a known bug that is fixed in 3.1...

An easier fix is to simply drag the problem message to a different
folder - you can usually open the attachment right away, but often only
once before the local copy gets corrupted again. I created a _Temp
folder for our users having this problem. Sometimes you'd have to
rebuild the index on that folder, even more than once. Then you could
move the message back to the Inbox (or wherever it was).

I seemed to notice this was mostly a problem with messages sent from
Outlook clients.

Anyway - update to 3.1 and the problem is gone.

Re: [Dovecot] Thunderbird problem

[Charles Marcus]

On 2010-06-25 7:48 AM, Frank Bonnet wrote:
> Some users had occasionally a problem with Thunderbird , some random
> emails with attachement(s) cannot be read anymore
> the email appears empty and TB seems to enter in an infinite loop
> saying it is downloading the message
> 
> The only solution I found was :
> 
> 1 - stop thunderbird on the client machine
> 2 - remove the .imap user's directory
> 3 - restart thunderbird on the client machine
> 
> After doing that all emails re-appear normally and attachements
> are readable/downloadable again.
> 
> ALL our clients use the IMAP(S) protocol.
> 
> Anyone had the same problem ?

Yes, this is a known bug that is fixed in 3.1...

An easier fix is to simply drag the problem message to a different
folder - you can usually open the attachment right away, but often only
once before the local copy gets corrupted again. I created a _Temp
folder for our users having this problem. Sometimes you'd have to
rebuild the index on that folder, even more than once. Then you could
move the message back to the Inbox (or wherever it was).

I seemed to notice this was mostly a problem with messages sent from
Outlook clients.

Anyway - update to 3.1 and the problem is gone.

-- 

Best regards,

Charles

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Charles Marcus]

On 2010-06-25 2:02 AM, Patrick Nagel wrote:
> I tried something similar (with Thunderbird) once, and it caused a lot of 
> trouble. We only had around 1.3 GB, IIRC, but thousands of folders.

3.1 is *much* better now...

Now to figure out if there is a way to do this for an entire account -
Inbox, Sent special folders, etc - maybe with a special/custom
namespace? It would have to present all of the accounts folders as
*sub-folders* of a top-level folder, so that the top-level folder could
be dragged/dropped...

Timo? Does that sound crazy?

-- 

Best regards,

Charles

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Charles Marcus]

On 2010-06-25 3:19 AM, Stan Hoeppner wrote:
> It's interesting that you weren't able/willing to track down the 
> source of the problem. It's also interesting how you mention "single
> drag and drop". In my experience, you can't drag/drop _folders_ in
> Thunderbird at all. All the migrations I've done this way required
> creating new folders on the IMAP server (Dovecot), group selecting
> all of the mail in the respective source folder, and selecting "copy"
> or "move" to the destination folder. I couldn't find a way to
> drag/drop a folder of group of folders. Care to share your secret?

No secret - TB used to not like dragging entire folders at all, but
slowly got better - and with 3.0, it got much better.

I just tested some (moving about 4GB Sent folder to a gmail account)
with 3.1rc2, and it moved everything perfectly, although it took a few
hours...

There were no sub folders though... I'll test that next...

-- 

Best regards,

Charles

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Charles Marcus]

On 2010-06-24 9:18 PM, Timo Sirainen wrote:
> But with auth cache enabled, there is no extra database load. The
> index files are also most likely in OS's cache (assuming local disk),
> so no extra disk I/O to read them either. I'm sure it's a bit more
> extra CPU usage, but I'm not all that certain that it's really that a
> big of a difference with Dovecot.

We have very few web users, so I really didn't see anything noticeable
performance wise, but I'll always use it just for the much cleaner logs.

-- 

Best regards,

Charles

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[mail...@securitylabs.it]

 On 25/06/2010 13:46, Timo Sirainen wrote:
Most people seem to be saying imapproxy helps a lot, because they 
noticed it helped so much with Courier/UW-IMAP..


Agree.

Re: [Dovecot] Thunderbird problem

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 25 Jun 2010, Frank Bonnet wrote:

Some users had occasionally a problem with Thunderbird , some random emails 
with attachement(s) cannot be read anymore


Yep.


the email appears empty


that one, too


and TB seems to enter in an infinite loop
saying it is downloading the message


that one in two cases:

1)  after I hit the memory limit in Dovecot causing it to crash and 
Thunderbird happily continues forever to wait for a response.


2) thunderbird opened too many simultaneous connections to the server. I 
do not remember where they would blocked or terminated, but in some cases 
thunderbird did not seem to detect this failure



After doing that all emails re-appear normally and attachements
are readable/downloadable again.


In my case the local cache was broken, so any action to rebuilt it was 
helpful:


+ delete the local mbox-cache file
+ hit the "Reindex" button of the folder properties of newer versions of 
Thunderbird


Regards,

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCSarr+Vh58GPL/cAQIe1Qf/TvATE+msFb5XIODHBaOxlBiG5kENnU2+
MUlbJit00CUp2vTLXopm7M/cetd+zGp81G5uP2xGMmpJfYKXXh0dLW0zLRdhvUJi
Slz7Z2hiFV0bIF2Zx4CQqoRn6kRuLL0nj9+jDJR2eYkBt5LM0/z6cTCXIJqk5gOZ
Qr+jWjbk5k1EKR9T+ka8HmD+DLcnqcUluoxwvgOJBL01Yqk4/4b0TPlivpPK9AQ1
3+zYjCexJomImiKlwwsAjaVpddh7CI19fpD8rBzadxFJqmotCfQ/zIinRsW5ScH8
k7+WbZHQi+jFIFzJeFaGv8R1Xu91cWAgOoaQvumDsqz+fpKdvTSh8A==
=BRDt
-END PGP SIGNATURE-

[Dovecot] Thunderbird problem

[Frank Bonnet]

Hello

Some users had occasionally a problem with Thunderbird , some random 
emails with attachement(s) cannot be read anymore

the email appears empty and TB seems to enter in an infinite loop
saying it is downloading the message

The only solution I found was :

1 - stop thunderbird on the client machine
2 - remove the .imap user's directory
3 - restart thunderbird on the client machine

After doing that all emails re-appear normally and attachements
are readable/downloadable again.

ALL our clients use the IMAP(S) protocol.

Anyone had the same problem ?

Thanks
F

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Timo Sirainen]

On 25.6.2010, at 8.04, Denny Schierz wrote:

> Am Donnerstag, den 24.06.2010, 22:30 -0500 schrieb Stan Hoeppner:
>> Does running imapproxy in this scenario yield any benefit, given there
>> is no network traffic involved? 
> 
> it does. IMAPPRoxy caches for example all directories. If the user open
> a folder, then you have a new connection to the IMAP server, that can
> take a while (auth ...). After a few seconds the connection is closed
> (that's quite normal for webapplications like webmail etc.) and than the
> user requests a new folder ... . So the imapproxy can cache all these
> things and make the Webmail faster. The only problem is, if there are
> new folders, so maybe it can take a few minutes, before the folder is
> visible.

Last I looked, imapproxy had some kind of a SELECT cache and possibly something 
else. They were implemented in an unreliable way and unless something has 
changed, I wouldn't recommend enabling them. Also I doubt any of them help with 
performance with Dovecot.

Most people seem to be saying imapproxy helps a lot, because they noticed it 
helped so much with Courier/UW-IMAP..

Re: [Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[Xavi Montero]

- Might this time-jump cause dovecot to die?
- I don't know wehere the dovecot logs are written to. Where do I 


I've confirmed in the logs that the die was really controlled and caused 
by the time-jump:


Here's the log:

"
Jun 25 06:25:00 hidrogeno dovecot: Time just moved backwards by 9
seconds. This might cause a lot of problems, so I'll just kill myself
now. http://wiki.dovecot.org/TimeMovedBackwards
"

So I installed ntp to control time in a smooth way.

Thanks for confirming that timejumps were the most probable cause and 
helping me to find the logs to confirm that.


Thanks to all again.
Xavi.


--
Xavier Montero - 93 589 71 91 - 630 59 01 62 - xmont...@dsitelecom.com

Re: [Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[Jakob Curdes]

Am 25.06.2010 10:35, schrieb William Blunn:

Xavi Montero wrote:
It has been running for several weeks but from one week ago or so, it 
suddently dies about once per day.


The only change done in the server one week ago has been to install 
ntpdate, running once per day in cron-daily.


In the cron reports, ntpdate reports a time-jump of 9 seconds each 
night. We did this because "apparently" we did not a "soft time 
drift" installing the full ntp daemon, but I start thinking the 
time-jump crashes dovecot.
Exactly. On a production server you should alsways set the time on 
startup against a good source, make sure your timezone settings are 
correct, anbd then let ntpd keep the time precise using the same source .




QUESTIONS:

- Might this time-jump cause dovecot to die?
It WILL cause dovecot to die. You will see this in the logs once you 
found them.
- I don't know wehere the dovecot logs are written to. Where do I 
start searching?

cf. http://wiki.dovecot.org/Logging

or wherever you configured it to go 

Re: [Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[Tapio Sokura]

On 25.6.2010 10:53, Xavi Montero wrote:

The only change done in the server one week ago has been to install
ntpdate, running once per day in cron-daily.

In the cron reports, ntpdate reports a time-jump of 9 seconds each


Time jumps, especially backwards, are not a good thing on a server. You 
should really consider changing from ntpdate to for example ntpd that 
takes care of keeping the clock synchronized, without unnecessarily 
stepping the clock. You should probably do this even if the actual cause 
of Dovecot crashing turns out not to be the time jumps.


Dovecot typically logs via syslog, so the logs go where your syslog 
writes them to. Usually something under /var/log.


  Tapio

Re: [Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[William Blunn]

Xavi Montero wrote:
It has been running for several weeks but from one week ago or so, it 
suddently dies about once per day.


The only change done in the server one week ago has been to install 
ntpdate, running once per day in cron-daily.


In the cron reports, ntpdate reports a time-jump of 9 seconds each 
night. We did this because "apparently" we did not a "soft time drift" 
installing the full ntp daemon, but I start thinking the time-jump 
crashes dovecot.


QUESTIONS:

- Might this time-jump cause dovecot to die?
- I don't know wehere the dovecot logs are written to. Where do I 
start searching?


I would suggest running ntpdate once at startup and then have ntp run 
continuously.


Having step changes in time will likely cause other problems.

Better to treat the problem properly at the source rather than having to 
put sticking-plasters on everything.


Bill

Re: [Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[Frank Bonnet]

I had the same problem due to clock backward


On 06/25/2010 09:53 AM, Xavi Montero wrote:

Hi all, my first post.

After years with an old server, a couple of months ago I've installed a
new one.

We use IMAP. After research, I chosed dovecot.

It has been running for several weeks but from one week ago or so, it
suddently dies about once per day.

The only change done in the server one week ago has been to install
ntpdate, running once per day in cron-daily.

In the cron reports, ntpdate reports a time-jump of 9 seconds each
night. We did this because "apparently" we did not a "soft time drift"
installing the full ntp daemon, but I start thinking the time-jump
crashes dovecot.

QUESTIONS:

- Might this time-jump cause dovecot to die?
- I don't know wehere the dovecot logs are written to. Where do I start
searching?

Thanks to all!
Xavi.

[Dovecot] Dovecot dies, maybe ntpdate related. I'm new to dovecot

[Xavi Montero]

Hi all, my first post.

After years with an old server, a couple of months ago I've installed a 
new one.


We use IMAP. After research, I chosed dovecot.

It has been running for several weeks but from one week ago or so, it 
suddently dies about once per day.


The only change done in the server one week ago has been to install 
ntpdate, running once per day in cron-daily.


In the cron reports, ntpdate reports a time-jump of 9 seconds each 
night. We did this because "apparently" we did not a "soft time drift" 
installing the full ntp daemon, but I start thinking the time-jump 
crashes dovecot.


QUESTIONS:

- Might this time-jump cause dovecot to die?
- I don't know wehere the dovecot logs are written to. Where do I start 
searching?


Thanks to all!
Xavi.

--
Xavier Montero - 93 589 71 91 - 630 59 01 62 - xmont...@dsitelecom.com

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 25 Jun 2010, Patrick Nagel wrote:


I tried something similar (with Thunderbird) once, and it caused a lot of
trouble. We only had around 1.3 GB, IIRC, but thousands of folders. Here is a


I cannot back it, I used TB v1.5 on Linux, where you can drag & drop just 
one folder hierarchie, with success. In fact, I copied from one account of 
one server to another account on another server. It was painly slow and 
also caused lots of local disk I/O.


Regards,

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCRf+b+Vh58GPL/cAQIxawf/RK33KUc42z3f5/VHxHxppnIISKYIdnZf
k5Ry8kGgFNdeZp1MV/g+UABs3ehs0rCTUmE8IsZvG8ZUb13S/2WnEyg+0zJRR9IR
VtWjGW92Uq5BF6fciic++jysJ3AnbfwSTYUY8ubcH7XOVooavLu3IXFrOISgp5/z
rGeudlH6UNmAr5yvnk8I0XluFoJ8YMgHOAzS7ZVLNwyuSIGQakUW/XcKBNa/k+qJ
VtMEdGQmTERObELAs2ff95g8UGyp4828ZCvFIfI68+OQkjdhYmwIOzXO/l+MvZec
4gzoVjA/jdayQ5CFYDHXkFo2nqQC3gh7uwW4tUAeloAXdoQBV9S7jg==
=oycF
-END PGP SIGNATURE-

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 25 Jun 2010, Stan Hoeppner wrote:


Angelo Chen put forth on 6/25/2010 1:32 AM:

Hi,

Nice approach, but the it's Thunderbird, what I have is outlook Express.


The process is the same.  Outlook Express can do this also.  Have you ever
used OE with an IMAP connection?  Do you need instructions on setting up an
IMAP connection?


I guess he referred to the Eudora->mbox migration step, so he could try 
thunderbird at all.


@Angelo:

Google hits some PST to XYZ programs.

IMHO Thunderbird can import messages from pst/OE, esp. when you run it on 
the same computer as Outlook Express.


When you have imported them into TB, they are (more or less) mbox files 
located in the Imap(Mail) subdir of your profile, which you can transform 
with the mb2md.pl script.


=

However, I would still at least *try* the plain IMAP method in OE.

Regards,

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCRerL+Vh58GPL/cAQKWIggApx93NQUCkh380KM4uKPNxzryCUOgqftq
wUO5p5KoOVQ+/wnKdtRIqARtTug6lUccdmAQRNUP6DzNE5OKSZ5HzoYO5N2HI8Cd
Tspwt0sC1bEmuNXYyI8uPYtKYXhSQBVZmGHs/uhDQvAkQvoaE4v6Ta9pcXhYgKl/
UUJ2lhJGtrEyt4uMXCYl4Y+5EFOUrvxz4IqgoqAqATrpbgyWYYA7YA5I4G7LtXjT
6nr6UG0k7oR5mqfMZOIfaMAViEc2E8V2ETiIkl1H2IHg7ysebpc+xCwunet02PsM
sP5FdARUwJ6dj52H5Ex6kFXDDE+HVYATKGY/cL90lFAYDeRIqqHUmQ==
=tedR
-END PGP SIGNATURE-

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Steffen Kaiser]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 25 Jun 2010, Denny Schierz wrote:


it does. IMAPPRoxy caches for example all directories. If the user open
a folder, then you have a new connection to the IMAP server, that can
take a while (auth ...). After a few seconds the connection is closed
(that's quite normal for webapplications like webmail etc.) and than the
user requests a new folder ... . So the imapproxy can cache all these
things and make the Webmail faster. The only problem is, if there are


Hmm, IMHO Dovecot is to cache most of the stuff, isn't it?
Esp. "LOGIN user pass" is cheap, because the password is cached.


new folders, so maybe it can take a few minutes, before the folder is
visible.


imapproxy caches the output of LIST?
On the website the FAQ only talks about "connections", for me it reads 
that imapproxy saves to open another TCP connection, the possible creation 
of other Dovecot processes and the LOGIN command.


Regards,

- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iQEVAwUBTCRcy7+Vh58GPL/cAQLkxgf/cohz4dHwX6gFAVQxEmWIg/rNXHDe+Oll
iHxJ0DFXTSvkLRICoIyKyYFeHx6iQgRfQ+Nu8tLw+OEZrL/QZT4hgxYqIXm+8HID
azsf3mIQeLEMd7X7er2F+xWJwmhPEWj9klxDXkAXjdhIVOdXQ9Nc28dfPbqQUgD0
UpyRwDANTjLz5YU+Itgc3dPMtokkkW7Fl6xu9DdnyXkwi3GOGpNjrpL+dSO3vaZR
JTT/mwA+1K3WKUiT8m7M7ytbFMLXJNALPZxAUCoTTXvwdInrED/5gcsLiXmXOQqx
rFM3Q+sfGdNiwAQ3ileVpG3d9OR/uvgUbyfzIuPcVObw0eQ8qU4E8Q==
=gZIq
-END PGP SIGNATURE-

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Stan Hoeppner]

Angelo Chen put forth on 6/25/2010 1:32 AM:
> Hi,
> 
> Nice approach, but the it's Thunderbird, what I have is outlook Express.

The process is the same.  Outlook Express can do this also.  Have you ever
used OE with an IMAP connection?  Do you need instructions on setting up an
IMAP connection?

http://support.microsoft.com/kb/264580
http://cc.jlab.org/docs/services/email/ms_outlook.html
http://www.nacs.uci.edu/email/oe-win-imap.html

-- 
Stan

Re: [Dovecot] importing outlook express messages to dovecot imap server

[Stan Hoeppner]

Patrick Nagel put forth on 6/25/2010 1:02 AM:
> Hi,
> 
> On 2010-06-25 03:51 UTC Stan Hoeppner wrote:
>> Angelo Chen put forth on 6/24/2010 10:25 PM:
>>> I think that's one option, but it's around 10G data for the last few
>>> years, and more than 100 folders under outlook express, do you think
>>> that will be fast enough by using Outlook Express copying option to an
>>> imap account?
>>
>> Look at it this way:  One way or another, in one format or another, you
>> will have to copy the data over the network from the PC to the Dovecot
>> server.  If you use the method myself and others have recommended, once
>> the data is copied to the IMAP folders, you are done.
>>
>> Now, if you do it the other way, which is copying raw files over, you then
>> have to convert them to maildir format.  It's a multi-step process.  And
>> could be error prone.
>>
>> The first method is a single step process and is reliable.
> 
> I tried something similar (with Thunderbird) once, and it caused a lot of 
> trouble. We only had around 1.3 GB, IIRC, but thousands of folders. Here is a 
> blog article that I wrote after I got it done with a perl script:
> 
> https://patrick-nagel.net/blog/archives/77

It's interesting that you weren't able/willing to track down the source of the
problem.  It's also interesting how you mention "single drag and drop".  In my
experience, you can't drag/drop _folders_ in Thunderbird at all.  All the
migrations I've done this way required creating new folders on the IMAP server
(Dovecot), group selecting all of the mail in the respective source folder,
and selecting "copy" or "move" to the destination folder.  I couldn't find a
way to drag/drop a folder of group of folders.  Care to share your secret?

I actually did some "man in the middle" transfers of mailboxen from an old
Exchange 5.5 server running IMAP to a Dovecot server using the method I just
described, with TBird being the man in the middle.  No problems, no data loss
or garbling.  Worked fine.  And that old Exchange box was a dinosaur, both
software and hardware.

-- 
Stan

Re: [Dovecot] dovecot evaluation on a 30 gb mailbox

[Denny Schierz]

Am Donnerstag, den 24.06.2010, 22:30 -0500 schrieb Stan Hoeppner:
> Does running imapproxy in this scenario yield any benefit, given there
> is no network traffic involved? 

it does. IMAPPRoxy caches for example all directories. If the user open
a folder, then you have a new connection to the IMAP server, that can
take a while (auth ...). After a few seconds the connection is closed
(that's quite normal for webapplications like webmail etc.) and than the
user requests a new folder ... . So the imapproxy can cache all these
things and make the Webmail faster. The only problem is, if there are
new folders, so maybe it can take a few minutes, before the folder is
visible.

cu denny


signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil