[Dovecot] dbmail benchmarking

2007-05-11 Thread Timo Sirainen
I thought I'd try benchmarking with dbmail (v2.2.4) to see how much  
slower a SQL backend could actually be. Skip to bottom for the  
conclusions.


Originally I ran the tests with the databases being in XFS  
filesystem. MySQL's performance was horrible. It went 3-7x faster  
with ext3.


MySQL 5.0.30 backend (innodb):

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=100 logout=0
Logi Sele Appe
100% 100% 100%
5%
   1  291  303

So that's 10 messages/sec saved. Now how about with 5 concurrent  
clients?


Logi Sele Appe
100% 100% 100%
5%
   5 1259 1332

Pretty well. Then something more generic:

./imaptest clients=1 seed=1 secs=30 msgs=100 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
  30%  5%
   1   37   36   75   73  110   34   24   73   78

Without Fetc (header/metadata fetching):
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
 30%  5%
   1   94   94  199  283  102   85  198  210

PostgreSQL 8.1.5 backend:

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=100 logout=0
Logi Sele Appe
100% 100% 100%
5%
   1  267  277

./imaptest clients=5 - append=100 seed=1 secs=30 msgs=100 logout=0
Logi Sele Appe
100% 100% 100%
5%
   5 1094 1144

./imaptest clients=1 seed=1 secs=30 msgs=100 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
  30%  5%
   9   29   40   74   72   99   22   12   64   71

./imaptest clients=1 seed=1 secs=30 msgs=100 logout=0 fetch=0
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
 30%  5%
  35  105   95  200  277   54   70  165  175

The last two tests gave "Unexpected tagged reply:" errors that I  
didn't get with MySQL. So apparently there's some PostgreSQL-specific  
bug in dbmail.


The same values with Dovecot 1.0 + maildir:

./imaptest clients=1 - append=100 seed=1 secs=30 msgs=100 logout=0
Logi Sele Appe
100% 100% 100%
5%
   1  346  364

./imaptest clients=5 - append=100 seed=1 secs=30 msgs=100 logout=0
Logi Sele Appe
100% 100% 100%
5%
   5 1408 1470

./imaptest clients=1 seed=1 secs=30 msgs=100 logout=0
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100% 100%  50% 100% 100% 100%
  30%  5%
   1  130  133  259  258  382  123  127  257  271

./imaptest clients=1 seed=1 secs=30 msgs=100 logout=0 fetch=0
Logi List Stat Sele Fet2 Stor Dele Expu Appe
100%  50%  50% 100% 100%  50% 100% 100% 100%
 30%  5%
   1  155  163  339  478  169  175  338  354

So, what are the conclusions?

 - In raw append speed dbmail is almost as fast as maildir.
 - In raw read/write speed maildir is about 1,6 times faster
 - When adding metadata fetches Dovecot is 4 times faster than  
dbmail. This is most likely because dbmail doesn't have a cache  
equivalent to dovecot.index.cache so it has to do the fetches the  
slow way.


I would have liked to also run the generic tests with more than 1  
client, but then I start hitting dbmail bugs and the test stops  
(reported already to their bugtracker).


PGP.sig
Description: This is a digitally signed message part


Re: [Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Johannes Berg
On Fri, 2007-05-11 at 16:16 -0600, Trever L. Adams wrote:

> I like this last version better. I will use it instead of my version of
> your other version. It is much cleaner.

Yeah, I thought using the imaplib would be good but apparently not :)

> However, your code still lacks conn.expunge(). My testing shows this is
> necessary (at least if you are using Maildir, I imagine it is even more
> necessary with mbox).

I think I had a reason not to do it. I forgot about that reason and just
do it manually once a while. Heh.

johannes


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Trever L. Adams
On Fri, 2007-05-11 at 17:08 +0200, Johannes Berg wrote:
> Interesting. I just noticed another bug too, it pads with spaces:
> " 1-May-07 ..."
> 

Yes, I caught this too.

> Or maybe just replace it all with:
> 
> sentbefore = time.strftime("%d-%b-%Y", time.localtime(time.time() - 
> float(sys.argv[1]) * 24 * 60 * 60))
> 
> johannes

I like this last version better. I will use it instead of my version of
your other version. It is much cleaner.

However, your code still lacks conn.expunge(). My testing shows this is
necessary (at least if you are using Maildir, I imagine it is even more
necessary with mbox).

I suggest you add it before you close the connection.

Thanks for your help Timo and Johannes.

Trever
--
If it's there and you can see it, it's REAL If it's there and you can't
see it, it's TRANSPARENT If it's not there and you can see it, it's
VIRTUAL If it's not there and you can't see it, it's GONE! -- Unknown


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] read only maildir subscriptions

2007-05-11 Thread Nagyon Almos
Ok, thanks for the reply.
It is clear now, now I can subscribe them.

Regards,
A.

Timo Sirainen On Mon, 2007-04-30 at 19:56  0200, Nagyon Almos wrote:
> Under thunderbird, I can successfully log in and read my new mails, 
> but  I cannot subscribe (?) within thunderbird to oldmails 
> (it is grey not black as other folders).
..
> namespace private {
>   separator = /
>   prefix = oldmails/

"oldmails" is the namespace prefix. You can't subscribe to it because
it's not a mailbox. Why would it even be needed? You should be able to
subscribe to mailboxes under it.




___
"Diszlexiás a gyermekem?" - 25 kérdéses gyorsteszt (Hálás lesz utána, bármi is 
az eredmény!)
http://www.varazsbetu.hu/diszlexiateszt/
http://ad.adverticum.net/b/cl,1,6022,133103,204265/click.prm




Re: [Dovecot] header-lines of subfolders have to be rescanned after each client startup

2007-05-11 Thread Rainer Sigl



Timo Sirainen schrieb:


On Thu, 2007-05-10 at 15:53 +0200, Rainer Sigl wrote:
 


Hi all,
after each startup of my different mail clients (thunderbird, outlook, 
...) the contents of subfolders if I click in has to be renewed. This 
takes some time for large folders. I just migrated from courier. And 
there all subfolders were available immediately. Is there something to 
configure?
Another effect regarding subfolders is that the client forgets, which 
subfolders were opened after shutdown and shows the folders closed. This 
means, that necessary subfolders have to be reopened after each startup. 
This was managed  as well by my previous courier server.
   



 


Many thanks,
no, I don't use NFS but it may be a permission problem.
I have virtual users so all the directories belong a artificial user 
'exim' and also group 'exim'. Additionally I saw now that the file 
dovecot-uidlist has the protection 600. The creation date of 
dovecot-uidlist is the date when I synced first time the folders with 
dovecot after installation. But strangely a change of the permissions to 
666 has no effect, dove-cot isn't changed.

A additional info may be that I migrated von courier imap.
So the mainfolder INBOX. and single subfolders have a correct changing 
dovecot-uidlist with 600 and exim exim.
All subfolders below the upper subfolders (.Sent.2005.2-Mar) don't 
change dovecot-uidlist.


Rainer


Sounds like something's really wrong in here. Are you using NFS? Do you
see dovecot-uidlist file in each subfolder directory? Does it get
changed?

 



--
-
Rainer Sigl,  | email: [EMAIL PROTECTED]
MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557
Postfach 1312,| fax:   +49 (89) 3-3569
D-85741 Garching, Germany | web:   http://www.mpe.mpg.de/~sigl/ 



Re: [Dovecot] fs quota plugin and NFS

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 18:01 +0300, Timo Sirainen wrote:
> NFS uses RPC quota commands to get the quota. Dovecot doesn't currently
> support this, but there is a company who implemented a patch for this. I
> can't seem to find it now though. I'll add it to
> http://dovecot.org/patches/ once I get it again.

Found it. http://dovecot.org/patches/quota-rquotad.c (copyright by
xs4all)

You'll also need to add it to quota_backends[] in quota.c and update
Makefile.am so it gets included in the quota plugin.

I'll update the FS quota backend to automatically use RPC for NFS
mounts.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Thunderbird or Dovecot bug? (multiple keywords)

2007-05-11 Thread Troy Engel

Timo Sirainen wrote:


A lot easier way is to look at the IMAP traffic and see what's different
in there. Then you can see if Dovecot is giving wrong replies somewhere
or if it's Thunderbird that gets confused from something.


I just used the TB debugging to trap all the IMAP traffic capturing a 
good session (my account) and the troubled session (his account) using 
the same WindowsXP 2.0.0 client.


Session:
a) add $Label1
b) add $Label2
c) select INBOX.Drafts
d) select INBOX
e) remove $Label2
f) remove $Label1

Alas, I can't spot where the problem is in the conversation; can I zip 
up both logs and send to you? (they're pretty large and contain some 
private info)


-te

--
Troy Engel | Systems Engineer
Fluid, Inc | http://www.fluid.com


Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Charles Marcus

On 5/11/2007 Timo Sirainen ([EMAIL PROTECTED]) wrote:

 - Get the rewritten Squat full text search index working. It probably
needs a name change, since it's not all that close to Cyrus Squat
anymore. Suggestions? Could of course be just "fts_dovecot".


How about 'dove_squat'?

;)

--

Best regards,

Charles


Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 19:41 +0300, Timo Sirainen wrote:
>  - Index file code cleanups, simplifications and optimizations. Make it
> work with NFS with attribute cache enabled.

Better explanation of this for those who are interested:

Currently mmap_disable=yes is faster (but takes more memory) than
mmap_disable=no, although it really should be the other way. I think
this isn't really because of mmaping but because of other differences
between how they work. So one of the main goals would be to get them to
use most of the same code paths so mmap_disable=no would be faster
again.

Another thing I intended to do was make it use less locks. Currently
it's locking files much more often than is necessary. Transaction log
could also use O_APPEND instead of locks. These are explained in
http://dovecot.org/list/dovecot/2007-April/021640.html Less locking
means that index handling becomes slightly faster and throughput higher
with shared mailboxes.

Simplifying the code also makes it easier to add NFS attribute cache
workarounds, make error handling more reliable and index files in
general more resistant against "corrupted index file" rebuilds
(important for cydir and dbox mail storages).

And one final thing that I want to do is that Dovecot needs to be able
to compare two index views. This is needed when a mailbox view is being
synchronized. So the comparing happens between the "current view in
memory" and "latest view in dovecot.index file". Currently the
synchronization works only if Dovecot can read the changes from
dovecot.index.log file. If the client has been idling for a really long
time and a lot of changes have happened in the mailbox, the log file
might have been deleted already, which causes "Out of sync" error and
Dovecot disconnects the client. Even if the log was found, it still
could be a lot faster to compare two small views than read huge log
files.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Virtual User Home Directory, and APOP Clear Text Passwords

2007-05-11 Thread imacat
On Fri, 11 May 2007 17:45:23 +0300
Timo Sirainen <[EMAIL PROTECTED]> wrote:
> On Fri, 2007-05-11 at 12:22 +0800, imacat wrote:
> >  1. I have a few virtual users, and I'm not using IMAP.  Do I still
> > have to give them a writable home directory?  I mean, they exists only
> See the bottom of http://wiki.dovecot.org/MailLocation/Mbox

Oh.  That is very nice.  I obviously missed it.  It is working.  Thank
you.

> >  2. I would like to use APOP in addition to SSL/TLS.  Currently
> > Dovecot saves APOP passwords as clear text.  I understand this.  But is
> > it possible to have some sort of encoding, for example, Base64?  Just to
> Currently there isn't anything like that. But I guess for v1.1 I could
> finally make it possible to specify base64/hex encoding in the scheme,
> so you could use eg. {plain.b64} or {plain.hex}.

That is really very nice.  I'm awaiting.

> If you really need this now you could also modify the sources yourself.
> It should be pretty easy to add a new plain.b64 scheme to
> src/auth/password-scheme.c (could also be implemented as a plugin)

Well, I tried.  It's a little harder than "pretty easy". ^^;  There
are more source files other than src/auth/password-scheme.c that need to
be updated accordingly, too, like src/auth/passdb.c.  But I still
believe it is easy, since base64_encode() is already there.  I'll wait
for the next summer anyway. ^_*'

--
Best regards,
imacat ^_*' <[EMAIL PROTECTED]>
PGP Key: http://www.imacat.idv.tw/me/pgpkey.txt

<> News: http://www.wov.idv.tw/
Tavern IMACAT's: http://www.imacat.idv.tw/
TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug


pgpgqBdZMqSqR.pgp
Description: PGP signature


Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 12:49 -0400, Stewart Dean wrote:
> can we paypal you some money?  It would be personal, not commercial and 
> consequently not that much, but everything helps...

Sure :) http://dovecot.org/donate.html



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Justin McAleer

Stewart Dean wrote:
can we paypal you some money?  It would be personal, not commercial 
and consequently not that much, but everything helps...




http://www.dovecot.org/donate.html 


There, now I've done my part ;)


Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Stewart Dean
Maybe you chould run things like the Chinese doctors of old...you paid 
them as long as you were healthy, but stopped when you got ill.  
Incentive for the doctor to keep you healthy!


Timo Sirainen wrote:

You can probably guess from my flood of mails that I'm no longer busy. I
still have a couple of exams but since I haven't spent any time at all
studying for them or attending the lectures, I can just as easily leave
them for next year.

I've read and answered most of the "easy" mails in this list and in my
INBOX. I'll try to answer the rest of them this weekend, so no need to
forward them to me again, at least not yet. :)

Now that I'm finally non-busy, it also appears that I haven't worked for
a long time and I'm almost broke. :) So now would be a great time for
your corporation to buy/sponsor some Dovecot development. Otherwise I'm
probably soon busy with non-Dovecot related work.

My own high priority TODO contains:

 - Get the rewritten Squat full text search index working. It probably
needs a name change, since it's not all that close to Cyrus Squat
anymore. Suggestions? Could of course be just "fts_dovecot".
 - Shared mailboxes / ACLs
 - dbox cleanups and possibly partial redesign. I'll write a separate
mail about this later.
 - Index file code cleanups, simplifications and optimizations. Make it
work with NFS with attribute cache enabled.

  


--

Stewart Dean, Unix System Admin, Henderson Computer Resources 
Center of Bard College, Annandale-on-Hudson, New York  12504  
[EMAIL PROTECTED]  voice: 845-758-7475, fax: 845-758-7035




Re: [Dovecot] Mails, work and so on

2007-05-11 Thread Stewart Dean
can we paypal you some money?  It would be personal, not commercial and 
consequently not that much, but everything helps...


Timo Sirainen wrote:

You can probably guess from my flood of mails that I'm no longer busy. I
still have a couple of exams but since I haven't spent any time at all
studying for them or attending the lectures, I can just as easily leave
them for next year.

I've read and answered most of the "easy" mails in this list and in my
INBOX. I'll try to answer the rest of them this weekend, so no need to
forward them to me again, at least not yet. :)

Now that I'm finally non-busy, it also appears that I haven't worked for
a long time and I'm almost broke. :) So now would be a great time for
your corporation to buy/sponsor some Dovecot development. Otherwise I'm
probably soon busy with non-Dovecot related work.

My own high priority TODO contains:

 - Get the rewritten Squat full text search index working. It probably
needs a name change, since it's not all that close to Cyrus Squat
anymore. Suggestions? Could of course be just "fts_dovecot".
 - Shared mailboxes / ACLs
 - dbox cleanups and possibly partial redesign. I'll write a separate
mail about this later.
 - Index file code cleanups, simplifications and optimizations. Make it
work with NFS with attribute cache enabled.

  


--

Stewart Dean, Unix System Admin, Henderson Computer Resources 
Center of Bard College, Annandale-on-Hudson, New York  12504  
[EMAIL PROTECTED]  voice: 845-758-7475, fax: 845-758-7035




[Dovecot] Mails, work and so on

2007-05-11 Thread Timo Sirainen
You can probably guess from my flood of mails that I'm no longer busy. I
still have a couple of exams but since I haven't spent any time at all
studying for them or attending the lectures, I can just as easily leave
them for next year.

I've read and answered most of the "easy" mails in this list and in my
INBOX. I'll try to answer the rest of them this weekend, so no need to
forward them to me again, at least not yet. :)

Now that I'm finally non-busy, it also appears that I haven't worked for
a long time and I'm almost broke. :) So now would be a great time for
your corporation to buy/sponsor some Dovecot development. Otherwise I'm
probably soon busy with non-Dovecot related work.

My own high priority TODO contains:

 - Get the rewritten Squat full text search index working. It probably
needs a name change, since it's not all that close to Cyrus Squat
anymore. Suggestions? Could of course be just "fts_dovecot".
 - Shared mailboxes / ACLs
 - dbox cleanups and possibly partial redesign. I'll write a separate
mail about this later.
 - Index file code cleanups, simplifications and optimizations. Make it
work with NFS with attribute cache enabled.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] SASL using different auth database?

2007-05-11 Thread Justin McAleer

Timo Sirainen wrote:

On Fri, 2007-05-11 at 11:39 -0400, Justin McAleer wrote:
  
If you can assign two IPs to the server, one for SMTP, and one for 
POP/IMAP, and you can use a SQL backend, you could craft your db schema 
and queries to pull the appropriate password based on the local IP of 
the connection. I know this isn't exactly what you were looking for, but 
I figured I would throw it out there anyway.



Probably an easier way to do that is to use %s which expands to smtp,
imap or pop3.
  


Doh! I went the route I did because we will be using the local IP to 
support userid-only logins (many domains), so I was familiar with that 
variable. Indeed this is much better :)


Re: [Dovecot] Deliver sending bounces from 'MAILER-DAEMON@'

2007-05-11 Thread Erland Nylend
Hello Timo,

On 2007-05-11, 16:18, Timo Sirainen wrote:
> > It seems to me that dovecot is sending bounce messages from
> > 'MAILER-DAEMON@' ..
> 
> It says to= so I'd guess the Return-Path: header in the
> message was that?

Thanks for the reply. I've managed to reproduce it. 

If I send a bounce to an account over quota ("mail from" set to <>),
then dovecot sends a bounce on the bounce, to "MAILER-DAEMON@":

| # telnet localhost 25
| Trying 127.0.0.1...
| Connected to localhost.localdomain (127.0.0.1).
| Escape character is '^]'.
| 220 my.mail.server ESMTP Postfix
| helo foo
| 250 my.mail.server
| mail from: <>   
| 250 2.1.0 Ok
| rcpt to: [EMAIL PROTECTED]
| 250 2.1.5 Ok
| data
| 354 End data with .
| testing
| .
| 250 2.0.0 Ok: queued as A366C27675

(It seems that postfix sets the return-path header based on the "mail
from", so the mail contains "Return-Path: <>" )

This shows the mail generated by deliver:

| May 11 17:55:27 my postfix/pickup[27580]: D6DB527681: uid=3000 from=<>
| May 11 17:55:27 my postfix/cleanup[14455]: D6DB527681: message-id=<[EMAIL 
PROTECTED]>
| May 11 17:55:27 my postfix/qmgr[3360]: D6DB527681: from=<>, size=1538, 
nrcpt=1 (queue active)
| May 11 17:55:27 my postfix/qmgr[3360]: D6DB527681: to=, 
relay=none, delay=0.01, delays=0.01/0/0/0, dsn=5.1.3, status=bounced (bad 
address syntax)
| May 11 17:55:27 my postfix/qmgr[3360]: D6DB527681: removed

This must be an error, then .. Dovecots deliver should not send
bounces in response to bounce mail. Could you have a look at it?

-- 
Erland Nylend



Re: [Dovecot] SASL using different auth database?

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 11:39 -0400, Justin McAleer wrote:
> If you can assign two IPs to the server, one for SMTP, and one for 
> POP/IMAP, and you can use a SQL backend, you could craft your db schema 
> and queries to pull the appropriate password based on the local IP of 
> the connection. I know this isn't exactly what you were looking for, but 
> I figured I would throw it out there anyway.

Probably an easier way to do that is to use %s which expands to smtp,
imap or pop3.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 11:35 -0400, Charles Marcus wrote:
> Will it be easy to convert from maildir to dbox when it is stable?

Convert plugin was created exactly for that. Although it doesn't yet
support preserving UIDs. I'm planning on migrating my own mail from mbox
to dbox some day, and I want to preserve the UIDs so there's a good
chance of that being implemented :)

Mail.app handles UIDVALIDITY changes horribly. I had to delete all the
IMAP accounts' caches and these Envelope* files until it finally
reloaded the messages from server instead of just going to infinite
loop..


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] SASL using different auth database?

2007-05-11 Thread Justin McAleer

Timo Sirainen wrote:

On Thu, 2007-05-10 at 20:57 -0400, Dave McGuire wrote:
  
   The goal here is to allow users to have different passwords for  
SMTP authentication and POP/IMAP connections.  I am converting an  
existing installation to Dovecot...this installation currently has  
quite a few users who are configured with SMTP authentication (Cyrus  
SASL/saslauthdb) using different passwords than are configured for  
their POP connections (via cucipop, /etc/passwd).  If someone can  
think of any better way to accomplish this goal, I'm all ears.



Currently you'd have to run two Dovecots to be able to do this. If you
do that, set a different base_dir for them and that should be enough.

I'll try to make this work with one dovecot-auth for Dovecot v2.0.

  


If you can assign two IPs to the server, one for SMTP, and one for 
POP/IMAP, and you can use a SQL backend, you could craft your db schema 
and queries to pull the appropriate password based on the local IP of 
the connection. I know this isn't exactly what you were looking for, but 
I figured I would throw it out there anyway.


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Charles Marcus

Timo - can you make a guess as to whether or not single-instance
storage is even a possibility with 2.0?



That'd be a dbox-only feature. I've been thinking about dbox a bit
recently. I'll write a separate mail about that sometimes later. Anyway,
it's possible that it gets implemented even for v1.1.


There will be dancing in the streets when this happens (at least on my 
street)... ;)


Will it be easy to convert from maildir to dbox when it is stable?

--

Best regards,

Charles


Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 10:57 -0400, Bruce Bodger wrote:

> Applied the patch, recompiled v1.1.0 using same line to config as  
> before..
> 
> ./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl
> 
> Sorry... no change: does not fix it.

In that case there's nothing I can do. You'll just have to restart
Dovecot. Or make it log directly into files instead of using syslog
(log_path setting).



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 22:00 +0200, Stefan Klatt wrote:
> What do you think about LDAP (or SQL,...) queries for each userparameter?
> Make it easyer to implement dovecot at raised and/or complex structures.

With SQL it's possible to create complex functions (at least with
PostgreSQL..). With LDAP I suppose it could be useful, but it would make
the code a lot more complex and since you're the first person who has
wanted it, I'm not sure if it's worth the trouble. You could always
create a custom checkpassword script instead.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-18 at 09:51 +0100, David Lee wrote:
> Firstly, congratutulations on the official 1.0;  we are now running this
> in production.
> 
> On Tue, 17 Apr 2007, Timo Sirainen wrote:
> 
> > I'm hoping to release the first alphas/betas in 2-3 months, with v1.1.0
> > maybe even as early as next summer. [...]
> 
> Quick check: Is "next summer" envisaged as 2007 or 2008?

Although v1.0 took 5 years, I wouldn't call 2008 "early". :) So I meant
2007.

> Could I put in a request for the "logfile consistency" item mentioned on
> April 3rd (and receiving some support)?  See:
>http://www.dovecot.org/list/dovecot/2007-April/021532.html
> and subsequent thread.

Added in TODO. I'll probably write a mail about asking people's opinions
on logging when I get closer to updating the logging code.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-18 at 11:11 +0100, Hugo Monteiro wrote:
> one thing i already discussed with you some time ago that would make
> some difference and open lots of possibilities, in my opinion, would be
> the ability to have a virtual INBOX that could be composed by a list of
> folders. That would make a world of difference to those who maintain a
> mixed service of pop and imap to their users.

Yes, I've been thinking about virtual mailboxes also. There are a lot of
nice things you could do with them. Like a virtual Trash mailbox listing
all \Deleted messages would be nice. Will see when I get around to
implementing them. :)



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-18 at 10:51 +0200, Jacek Osiecki wrote:
> One question... I remember that such issue has been raised here some time
> ago, but I don't know if there were any decisions made... Is there a plan
> for using hard links in dovecot-LDA when the mail is going to be delivered
> to many recepients... In fact lack of such functionality is the only thing
> that holds me from switching in multiple locations to dovecot from
> cyrus-imap, which I really don't like (it requires LDAP, which I don't
> love as well ;)

It's a bit annoying to implement. I could of course always just kludge
it in some way, but perhaps when shared mailbox support is finished this
could be implemented more easily:

 - Run deliver as some kind of a master user having access to all users'
mailboxes
 - Write the mail to some temporary mailbox
 - Copy it to the users' mailboxes. With maildir_copy_with_hardlinks=yes
this'll make them hardlinks

The temporary mailbox is needed because otherwise it's possible that the
message gets expunged before it's copied to all the users' mailboxes.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround

2007-05-11 Thread Adam McDougall
On Fri, May 11, 2007 at 03:41:50PM +0300, Timo Sirainen wrote:

  On Sun, 2007-05-06 at 21:34 -0400, Adam McDougall wrote:
  > In short, upper directories are used to limit world access to just one 
group 
  > 'decsall', and once inside, r/o world access could be used as an anonymous 
  > substitution for 'decsall' and the group permissions would allow writing by 
  > 'decsstaff'.  I don't know how courier stores and checks mail directory 
  > permissions, but it was aware when a user would not be able to delete or 
change 
  > messages and would return a READ-ONLY imap error when an attempt to change 
a 
  > message occurred
  
  Looks like Dovecot never returns maildir mailbox as READ-ONLY. So, your
  issues are:
  
  1. Dovecot doesn't give an error when a flag is changed for mails. This
  is intentional, because otherwise you'd keep seeing "mailbox is
  read-only" errors all the time when reading mails (because clients keep
  setting the \Seen flag). Besides you're using a per-user index directory
  so the flag changes actually are saved.
  
  2. EXPUNGE gives an internal error because it discovers too late that it
  can't unlink() the maildir file. This is a problem.. Even worse, you
  can't recover from it without deleting dovecot.index.log file. This
  should have two fixes:
  
   - If Dovecot ever notices that it can't unlink() a file, it should
  recover itself from it without giving internal errors. That would
  probably mean that the expunged message shows up as a new message. I'll
  see if I can get this fixed for v1.1.
  
   - It should treat the maildir as READ-ONLY so the EXPUNGE command is
  never written into dovecot.index.log in the first place. Fixed:
  http://dovecot.org/list/dovecot-cvs/2007-May/008734.html

Thanks for addressing these issues.  I applied this patch and it is a 
good improvement; I no longer get internal errors, and it has the effect
of making thunderbird _appear_ to be able to completely delete an email
because dovecot records the deleted flag and ignores the expunge, but
mutt can see the messages marked deleted and does print to my screen
that the mailbox is readonly when I try to expunge.  So thunderbird with
dovecot doesn't reflect the readonly warning that courier gives when trying
to mark a message deleted, but that is probably acceptable to me since it
looks like it is more beneficial for me to use ACLs.  I would rather use
ACLs so that "deleted" messages reappear on their own, rather than worry
about users "deleting" shared mail and requiring me to erase their dovecot
metadata incase they changed their mind (and don't want to use something
besides thunderbird to fix it).  

I've also noticed that as soon as I enable the acl plugin, clients no longer
have any access at all to shared mailboxes unless a dovecot-acl file is used
to give them access previously granted by the file permissions.  This might
be worth noting in the documentation (if it is expected) so a person doesn't
turn on the acl plugin without being aware of the consequences.  I do appreciate
that it works that way, so I will not forget to apply an acl.
  
  3. EXPUNGE silently ignores no-permissions error. I'm not sure if this
  is good or not. In any case it shouldn't give an error if there are no
  messages with \Deleted flag, but perhaps it should give an error if
  there are.. Some clients do automatic expunges, so it would be annoying
  to see error messages constantly about it. I won't change this in v1.0
  branch, but perhaps in v1.1.
  
  > As a workaround, I wondered if I could apply dovecot ACLs to the public 
folders 
  > to reinforce my intentions of only allowing readonly access to certain 
users.
  
  ACL plugin makes flag changes and expunges also fail silently.
  
  > ACL questions (which might help others understand better as well):
  > Is a 'user=foo rl' entry supposed to remove any possible write permissions 
in a 
  > maildir covered by such an ACL for user foo?
  
  Yes.
  
  > Is it even possible for an ACL to grant additional permissions above what 
is 
  > granted to a user by standard unix permissions?
  
  No.
  
  > Does an empty ACL equate to the exact permissions granted by unix,
  
  No. It means "no permissions".
  
  >  and should an 
  > admin try to design an ACL to modify as little as possible, or should the 
admin 
  > try to use enough acl entries to form a complete description of the desired 
  > resulting permissions? I get the impression that broad ACLs should be 
listed 
  > first, and more specific ACLS listed below can add or remove ACL 
permissions 
  > applied earlier in the file as a 'last match' case.
  
  No comments about that, except that user=x always overrides group=y in
  the file. I'm not sure if the order matters within group= lines,
  probably yes. Then again groups aren't actually working at all currently
  (the code supports them, but there's no way to list user's groups
  currently).
  
  > Should the acl plugin be able to print a summary of

Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-18 at 07:49 -0400, Charles Marcus wrote:

> The other is single-instance storage... and I don't see a mention of 
> that anywhere...
> 
> Timo - can you make a guess as to whether or not single-instance
> storage 
> is even a possibility with 2.0?

That'd be a dbox-only feature. I've been thinking about dbox a bit
recently. I'll write a separate mail about that sometimes later. Anyway,
it's possible that it gets implemented even for v1.1.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Johannes Berg
On Fri, 2007-05-11 at 08:54 -0600, Trever Adams wrote:

> > So you'll need to use "11-May-2007". It doesn't support time.
> >   
> Tim, thank you. I am afraid I do not understand if the date is only 
> allowed as such why does python's imaplib convert the date as such?!? 

Interesting. I just noticed another bug too, it pads with spaces:
" 1-May-07 ..."

Try this in my script:

# this line is already there
sentbefore = imaplib.Time2Internaldate(time.time() - float(sys.argv[1]) * 24 * 
60 * 60)

# add these lines
sentbefore = sentbefore.split('"')[1].split()[0]
sentbefore = '"%s"' % sentbefore

Or maybe just replace it all with:

sentbefore = time.strftime("%d-%b-%Y", time.localtime(time.time() - 
float(sys.argv[1]) * 24 * 60 * 60))

johannes


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] v1.1 plans

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 00:30 -0700, Timothy Martin wrote:
> Are all v1.0 patches and CVS HEAD now 1.1? Specifically I was  
> wondering if the quota rewrite (http://dovecot.org/list/dovecot/2006- 
> September/016134.html) is or will be in the 1.1 builds.

Not all, but quota rewrite is.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] fs quota plugin and NFS

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 17:54 +0200, Nicolas STRANSKY wrote:
> Hi,
> 
> I'm trying to use the Dovecot v1 fs quota plugin. The server uses NFS
> mounted volumes for INBOX and other maildir folders. The /usr/bin/quota
> command is working seamlessly but I get errors with the quota plugin,
> which gives the following logs:
> Apr 19 17:46:15 rouge dovecot: IMAP(xyxyxyx): quotactl(Q_GETQUOTA,
> nfs.xxx.yyy.org:/home) failed: No such file or directory
> Apr 19 17:46:18 rouge dovecot: IMAP(yxyxyxy): quotactl(Q_GETQUOTA,
> nfs.xxx.yyy.org:/home) failed: No such file or directory
> 
> Is there a way to get it work ?

NFS uses RPC quota commands to get the quota. Dovecot doesn't currently
support this, but there is a company who implemented a patch for this. I
can't seem to find it now though. I'll add it to
http://dovecot.org/patches/ once I get it again.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Log data on dovecot-auth socket

2007-05-11 Thread Koen Vermeer
Op vr, 11-05-2007 te 16:01 +0300, schreef Timo Sirainen:
> A bit late, but auth_debug_passwords=yes makes it log most of the
> traffic I think. If not, then I guess there aren't any easy ways.

Thanks for the reply! I needed a real log of the conversation. Anyway, I
got it working without the log, so never mind.

Koen



Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bruce Bodger
Applied the patch, recompiled v1.1.0 using same line to config as  
before..


./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl

Sorry... no change: does not fix it.

Regards,
Bruce

On May 11, 2007, at 10:24 AM, Timo Sirainen wrote:


On Fri, 2007-05-11 at 09:26 -0400, Bruce Bodger wrote:


That would be accurate, Bill.  I suspect the problem to be within
dovecot since, as I mentioned earlier, neither uw-imap or other
running daemons exhibit this problem.


That's probably because UW-IMAP and most other servers keep  
creating new

processes which open new connections to syslog, while Dovecot has one
long-running dovecot process.

If you apply the patch below and send SIGUSR1 signal to dovecot  
process,

does it fix it?


Re: [Dovecot] Semi-static userdb...?

2007-05-11 Thread Timo Sirainen
On Fri, 2007-04-20 at 14:00 +0100, John Robinson wrote:
> I'm trying to add "virtual" mailboxes to a system. Real users with 
> different uids own domains. Each domain has a passwd-file passdb. I 
> don't want to use this passwd-file for the userdb, because I want to fix 
> the home, mail and uid/gid settings. Can I use the static userdb in a 
> less static manner, e.g.
> 
> userdb static {
>args = uid=%{owner of /vmail/%d} gid=%{gid of /vmail/%d} 
> home=/vmail/%d/users/%u 
> mail=mbox:/vmail/%d/users/%u/mail:INBOX=/vmail/%d/users/%u/INBOX
> }

Hmm. You could always use a checkpassword script which does everything,
but the above does look like it could be useful. I'm just not sure what
would be the best way to implement it so that it would be useful for
other kinds of configurations too..

One possibility would be to set "uid_file=/vmail/%d gid_file=/vmail/%d".
I guess that would be good. Added to TODO, but I'm not sure when I get
around to implementing it.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Return error instead of dying on time back skip?

2007-05-11 Thread John Peacock

Ben Winslow wrote:

Clock drift of about 13 seconds/day (150 PPM) is (unfortunately) not
uncommon, and 4-6 seconds/day (50-75 PPM) is about the norm for PC
hardware in my experience.

Of course, this is exactly the reason why you should run ntpd instead
of ntpdate on a cron job (especially a once-per-day cron job...)


I would again recommend clockspeed:

http://cr.yp.to/clockspeed.html
http://foo42.de/devel/sysutils/clockspeed-conf/

for machines which don't have continuous connection to the Internet 
(where [x]ntpd won't do you any good).  It handily reigns in bad clock 
crystals with only a couple of external connections per month.


John

--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD  20706
301-459-3366 x.5010
fax 301-429-5748


Re: [Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Trever Adams

Timo Sirainen wrote:

From IMAP RFC:

search-key  = "ALL" / "ANSWERED" / "BCC" SP astring /
  "BEFORE" SP date / "BODY" SP astring /
..
date= date-text / DQUOTE date-text DQUOTE
date-text   = date-day "-" date-month "-" date-year

So you'll need to use "11-May-2007". It doesn't support time.
  
Tim, thank you. I am afraid I do not understand if the date is only 
allowed as such why does python's imaplib convert the date as such?!? 
Anyway, I understand that is a question for some place else. I hope 
python has something like strtok or similar where I can tokenize the 
first part out to be used.


Thank you very much.

Trever


Re: [Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 06:22 -0600, Trever L. Adams wrote:
> 
> ok, uids = conn.search(None, 'BEFORE', unreadsentbefore, 'SEEN')
> 
> "11-May-2007 06:18:29 -0600" is the value of unreadsentbefore.

From IMAP RFC:

search-key  = "ALL" / "ANSWERED" / "BCC" SP astring /
  "BEFORE" SP date / "BODY" SP astring /
..
date= date-text / DQUOTE date-text DQUOTE
date-text   = date-day "-" date-month "-" date-year

So you'll need to use "11-May-2007". It doesn't support time.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Virtual User Home Directory, and APOP Clear Text Passwords

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 12:22 +0800, imacat wrote:
> Dear all,
> 
> Hi.  This is imacat from Taiwan.  I'm new to this list.
> 
> I was migrating from Qpopper to Dovecot.  (Qpopper copies the
> mailbox when POP3 logged in, which causes quota problem.)  Generally
> Dovecot is nice.  However, I have a few questions:
> 
>  1. I have a few virtual users, and I'm not using IMAP.  Do I still
> have to give them a writable home directory?  I mean, they exists only
> for me to receive mails from fetchmail.  They really have nothing but
> their inbox.  But if I do not give them a writable home directory, I got
> this error in my syslog all the time:
> 
> mbox: Can't create root mail directory /home/homeless/mail: Permission denied

See the bottom of http://wiki.dovecot.org/MailLocation/Mbox

>  2. I would like to use APOP in addition to SSL/TLS.  Currently
> Dovecot saves APOP passwords as clear text.  I understand this.  But is
> it possible to have some sort of encoding, for example, Base64?  Just to
> prevent casual reads.  I have several users.  I do not want to know
> their passwords just to add them to the APOP database.  I do not want
> other roots to read my passwords, intended or not.
> 
> Encoded passwords also have the advantage that I can use
> passwords that contain the colon sign ":".  It would be nice if
> something a little harder than Base64 can be used.

Currently there isn't anything like that. But I guess for v1.1 I could
finally make it possible to specify base64/hex encoding in the scheme,
so you could use eg. {plain.b64} or {plain.hex}.

If you really need this now you could also modify the sources yourself.
It should be pretty easy to add a new plain.b64 scheme to
src/auth/password-scheme.c (could also be implemented as a plugin)


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] No authentication sockets found

2007-05-11 Thread funkypunky drunky

I have lived the same problem in my system. But not evrytime sometimes it
only happens when i restart dovecot while mta is working. I think postfix
(my mta) cannot connect the new authentication socket. If i need the
restarting dovecot, i restart postfix  too. It happens only for the above
situation.
Best regards


2007/5/11, Timo Sirainen <[EMAIL PROTECTED]>:


On Wed, 2007-05-09 at 10:59 -0400, Stewart Dean wrote:
> DC imap host was rebooted today 3 hours ago.  Any info on what this is
about?
..
> imap-login: No authentication sockets found
..
> Background: DC 1.0, invoked from /etc/rc.local during boot.  After
> stopping and restarting DC, things seem to work again. As per an old
> problem, I checked /var/run/dovecot, but it was empty with plenty of
> space.  FWIW, the owner/group for login dir (/var/run/dovecot) is
> dovecot:dovecot, perm 755, while /var/run/dovecot/login is root:dovecot,

The error message means that /var/run/dovecot/login directory or its
contents was deleted while Dovecot was running. I'd guess that you start
Dovecot too early and after startup another script goes and deletes the
entire /var/run/ directory. Or maybe that's done in background. Or
something..





Re: [Dovecot] Conflict -> Dovecot + Maildrop (maildirsize)

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-09 at 10:23 -0300, Roberto Gonzalez Azevedo wrote:
> 'maildirsize' with dovecot:
> "
> 1099511627776S
> 585847701 102542
> 5504023 959
> "
> 
> In both examples the quota is 1GB (1073741824S).
> And now ? Does anybody known how to fix that ?

Looks like you're trying to tell the quota in bytes to Dovecot. Dovecot
wants it to be in kilobytes in its configuration file / userdb (it still
writes it in bytes to maildirsize, as you can see above it writes 1GB *
1024 value in there).



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] header-lines of subfolders have to be rescanned after each client startup

2007-05-11 Thread Timo Sirainen
On Thu, 2007-05-10 at 15:53 +0200, Rainer Sigl wrote:
> Hi all,
> after each startup of my different mail clients (thunderbird, outlook, 
> ...) the contents of subfolders if I click in has to be renewed. This 
> takes some time for large folders. I just migrated from courier. And 
> there all subfolders were available immediately. Is there something to 
> configure?
> Another effect regarding subfolders is that the client forgets, which 
> subfolders were opened after shutdown and shows the folders closed. This 
> means, that necessary subfolders have to be reopened after each startup. 
> This was managed  as well by my previous courier server.

Sounds like something's really wrong in here. Are you using NFS? Do you
see dovecot-uidlist file in each subfolder directory? Does it get
changed?



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] SASL using different auth database?

2007-05-11 Thread Timo Sirainen
On Thu, 2007-05-10 at 20:57 -0400, Dave McGuire wrote:
> 
>The goal here is to allow users to have different passwords for  
> SMTP authentication and POP/IMAP connections.  I am converting an  
> existing installation to Dovecot...this installation currently has  
> quite a few users who are configured with SMTP authentication (Cyrus  
> SASL/saslauthdb) using different passwords than are configured for  
> their POP connections (via cucipop, /etc/passwd).  If someone can  
> think of any better way to accomplish this goal, I'm all ears.

Currently you'd have to run two Dovecots to be able to do this. If you
do that, set a different base_dir for them and that should be enough.

I'll try to make this work with one dovecot-auth for Dovecot v2.0.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Thunderbird or Dovecot bug? (multiple keywords)

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-09 at 12:01 -0700, Troy Engel wrote:
> All this long-winded explanation leads me to believe that somehow the 
> difference in dovecot-keywords is causing this problem, but what I can't 
> tell is who's problem it is -- Dovecot, or Thunderbird?

A lot easier way is to look at the IMAP traffic and see what's different
in there. Then you can see if Dovecot is giving wrong replies somewhere
or if it's Thunderbird that gets confused from something.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Timo Sirainen
On Fri, 2007-05-11 at 09:26 -0400, Bruce Bodger wrote:

> That would be accurate, Bill.  I suspect the problem to be within  
> dovecot since, as I mentioned earlier, neither uw-imap or other  
> running daemons exhibit this problem. 

That's probably because UW-IMAP and most other servers keep creating new
processes which open new connections to syslog, while Dovecot has one
long-running dovecot process.

If you apply the patch below and send SIGUSR1 signal to dovecot process,
does it fix it?

Index: src/master/main.c
===
RCS file: /var/lib/cvs/dovecot/src/master/main.c,v
retrieving revision 1.80.2.21
diff -u -r1.80.2.21 main.c
--- src/master/main.c   11 May 2007 13:28:52 -  1.80.2.21
+++ src/master/main.c   11 May 2007 14:23:59 -
@@ -168,6 +168,7 @@
 static void sig_reopen_logs(int signo __attr_unused__,
void *context __attr_unused__)
 {
+   closelog();
set_logfile(settings_root->defaults);
 }
 



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Return error instead of dying on time back skip?

2007-05-11 Thread Ben Winslow
On Fri, 11 May 2007 14:50:54 +0300
Timo Sirainen <[EMAIL PROTECTED]> wrote:

> The code already allows the clock to move backwards by 5 seconds
> without dying, so how horrible are the clocks in those computers? :)

Clock drift of about 13 seconds/day (150 PPM) is (unfortunately) not
uncommon, and 4-6 seconds/day (50-75 PPM) is about the norm for PC
hardware in my experience.

Of course, this is exactly the reason why you should run ntpd instead
of ntpdate on a cron job (especially a once-per-day cron job...)

--
Ben Winslow <[EMAIL PROTECTED]>


pgpnpNGpF3cyV.pgp
Description: PGP signature


Re: [Dovecot] No authentication sockets found

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-09 at 10:59 -0400, Stewart Dean wrote:
> DC imap host was rebooted today 3 hours ago.  Any info on what this is about? 
>  
..
> imap-login: No authentication sockets found
..
> Background: DC 1.0, invoked from /etc/rc.local during boot.  After 
> stopping and restarting DC, things seem to work again. As per an old 
> problem, I checked /var/run/dovecot, but it was empty with plenty of 
> space.  FWIW, the owner/group for login dir (/var/run/dovecot) is 
> dovecot:dovecot, perm 755, while /var/run/dovecot/login is root:dovecot, 

The error message means that /var/run/dovecot/login directory or its
contents was deleted while Dovecot was running. I'd guess that you start
Dovecot too early and after startup another script goes and deletes the
entire /var/run/ directory. Or maybe that's done in background. Or
something..



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] locking questions

2007-05-11 Thread Timo Sirainen
On Tue, 2007-04-24 at 13:43 -0400, Stewart Dean wrote:
> I have Dovecot 1.0 in trial use by the IT staff, and have some locking 
> questions
> 
> Background, the mail server runs procmail, sendmail and NFS exports the 
> user homedir and mailbox to a) a login shell host and b) a mailing list 
> services host.  It runs UWIMAP on the usual ports and dovecot on a 
> arbitrary port number.  Because of concern with NFS and file access 
> contention. I have the following dovecot.conf file access parms set:
>a) mbox_dirty_syncs =  no
>b) mbox_lazy_writes = no

I'm not sure if these really help anything. UW-IMAP's behavior is the
same as if these were both "yes".

> 1) I am seeing these errmsgs.  Are they anything I should be concerned 
> about, or are they just the inevitable result of lock contention for 
> mailboxes and folders by procmail, imap/dovecot and NFS?
> > In syslog, I am seeing this: 
> >
> > Apr 23 18:46:34 mercury mail:err|error dovecot: IMAP(sdean): 
> > stat(/var/spool/mail/sdean.lock) failed: No such file or directory
> > Apr 23 18:46:34 mercury mail:err|error dovecot: IMAP(sdean): 
> > file_lock_dotlock() failed with mbox file /var/spool/mail/sdean: No such 
> > file or directory

I don't think this is good. I just updated the error message to be
better:

http://dovecot.org/list/dovecot-cvs/2007-May/008740.html

So is this using AIX? Sounds similar to a Solaris NFS problem. Maybe
this helps: http://dovecot.org/list/dovecot/2006-December/018145.html

> 2) I earlier asked, but don't think I got an answer to this question:
> > Everybody wants to lock differently
> > 1) procmail (delivering for sendmail), which seems to want to use 
> > dotlocking, fcntl and lockf locking; for whatever reasons, the compile 
> > time tests seem to disallow flock.
> > 2) UWIMAP which according to wiki.dovecot.org/Migration/UW uses:
> > mbox_read_locks = flock
> > mbox_write_locks = dotlock flock
> > 3) I want to run dovecot in the same environment as I switch over, for 
> > which the locking strategy is supposed to be (according to 
> > http://wiki.dovecot.org/MboxLocking)
> > mbox_read_locks = fcntl
> > mbox_write_locks = dotlock fcntl
> >
> > Since that same page in the wiki says. "*It's important that all 
> > software that's reading or writing to mboxes use the same locking 
> > settings.",* I had recompiled procmail so it only usedotlocking and 
> > fcntl (thus removing lockf in addition to the disallowed flock)...but 
> > now I bumped into the Migration/UW page and there looks to be a conflict.
> >
> > I want to be able to run dovecot (for testing) and UWIMAP (for all 
> > users/production)  at the same time (but never at the same time with 
> > the same mailbox), but UWIMAP is doc'ed to use flock and Dovecot doesn't
> > It's conceivable that the different locking strategies of UWIMAP and 
> > Dovecot don't matter because they would never be going after the same 
> > mail files (right?) but there is procmail, which has to interact with 
> > both.  It appears to not use flock OK, even though the docs say it 
> > should since UWIMAP expects it, and I've disabled lockf as well.  But 
> > will there be a conflict with procmail and dovecot's use of fcntl, 
> > which UWIMAP doesn't use?
> >
> > Help. 

flock works only locally within one computer, so if you're using
multiple that's bad. Just make sure that procmail and Dovecot use
compatible locks, so I guess dotlock and fcntl. If you try to access the
same mailbox using UW-IMAP and Dovecot simultaneously UW-IMAP kills
itself anyway. Also as long as everyone are using dotlock there won't be
any corruption problems.

> 3) I occasionally see "Disconnected for inactivity" message.  How long 
> before that happens?  Is the interval configurable?

It's 30 minutes. It's not configurable, but IMAP RFC says:

"If a server has an inactivity autologout timer, the duration of that
timer MUST be at least 30 minutes."

This is also what all the other servers are using, so the clients should
be smart enough to do at least something within half an hour..



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Mixing MBOX and Maildir?

2007-05-11 Thread Timo Sirainen
On Sat, 2007-04-21 at 09:38 -0700, Marc Perkel wrote:
> Is it possible to have some folders be mbox and some be maildir format?
> 
> Why you might ask? Sometime for diagnostics I have the Exim MTA archive 
> certain messages in MBOX format and then I want to copy them into an 
> imao folder to look at them with Thunderbird. When I used MBOX all I had 
> to do was copy the file into a folder and it worked. So what I'd like to 
> do is have a folder with a special name that is MBOX and the rest be 
> maildir.

You could do this with namespaces. http://wiki.dovecot.org/Namespaces
has an example.

> Ideally have Dovecot be smart so that if the file exists then it is 
> assumed to be MBOX and if the directory exists then it is assumed to be 
> maildir. Has anyone done anything like this?

I was thinking about implementing this to v1.1, but it's not so easy, so
I'm not sure.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Dovecot refuses connections

2007-05-11 Thread Timo Sirainen
On Sun, 2007-04-29 at 19:37 +0100, John Rowe wrote:
> 1. It won't start from xinetd. I get very little in the logs, just:
> 
> Apr  7 07:07:41 xyz xinetd[2545]: START: imaps pid=17445
> from=144.173.228.126
> Apr  7 07:07:41 xyz xinetd[2545]: EXIT: imaps pid=17445 duration=0(sec)
> 
> where 'xyz' is the host name

How exactly did you try to start it?

> 2. If I instead I run dovecot as a daemon then both my machines have
> suffered the problem where dovecot runs for several days and then
> disallows connections with:
> 
> Apr 29 04:06:07 xyz dovecot: imap-login: opendir(.) failed when trying
> to get list of authentication servers: Permission denied

Something has changed the permissions
of /usr/local64/var/run/dovecot/login directory. When this happens, see
what the directory permissions are.

> Killing and restarting dovecot seems to fix it.

Dovecot resets the permissions at startup.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] dovecot 1.0.0 hang on one server

2007-05-11 Thread Timo Sirainen
On Mon, 2007-04-30 at 11:48 +0200, Roberto Tagliaferri wrote:
> Hi, i've upgraded 5 server from dovecot beta7 or old dovecot-stable to 
> 1.0.0 but on one server dovecot crash after few hours without any log
> Apr 23 18:19:32 Info: POP3(username): Disconnected: Logged out top=0/0, 
> retr=0/0, del=0/0, size=0
> Apr 24 09:54:11 Info: Dovecot v1.0.0 starting up

Are you sure there are no errors logged? The above messages could be in
a different log file than errors. See http://wiki.dovecot.org/Logging

If there really are no errors, it can only mean that the main dovecot
process crashed. Then it should have written a core file to base_dir
(/var/run/dovecot/). If not, run ulimit -c unlimited before starting
dovecot.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Changes in authentication's???

2007-05-11 Thread Timo Sirainen
On Thu, 2007-05-10 at 11:01 +0200, Boris Ivanov wrote:
> May 10 02:56:02 scat dovecot: auth(default): client out: OK 1
> user=test   User=   &:/home/test:/tmp/email/bin/nonexistance=

There's some extra data in here.

> etc_path = /usr/local/www/data/Email/RefCont/etc
> pwd_mkdb -d ${etc_path} ${etc_path}master.passwd
> pw useradd -V ${etc_path} -n $1 -g mailusers -s /tmp/email/bin/nonexistance
> echo "$2" | pw usermod -V ${etc_path} -n $1 -h 0

I don't think the passwd-files generated by this are fully compatible
with Dovecot's passwd-file handling. If there's anything after the shell
field, Dovecot treats them as its internal passdb extra fields.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Tracing an IP of a user who deleted the message

2007-05-11 Thread Timo Sirainen
On Thu, 2007-05-10 at 12:55 +0200, Steffen Kaiser wrote:
> > May 10 13:19:04 mailserver dovecot: IMAP(office): copy -> Trash: uid=1131, 
> > msgid=<[EMAIL PROTECTED]>, box=Sent
> > May 10 13:19:05 mailserver dovecot: IMAP(office): deleted: uid=1131, 
> > msgid=<[EMAIL PROTECTED]>, box=Sent
> > May 10 13:53:08 mailserver dovecot: IMAP(office): copy -> Trash: uid=1719, 
> > msgid=<[EMAIL PROTECTED]>
> > May 10 13:53:08 mailserver dovecot: IMAP(office): deleted: uid=1719, 
> > msgid=<[EMAIL PROTECTED]>
> >
> You can log the PID, too:
> 
> # Log prefix for mail processes. See doc/wiki/Variables.txt for list of
> # possible variables you can use.
> #mail_log_prefix = "%Us(%u): "
> mail_log_prefix = "%Us(%u) [%p]: "
> 
> Then you can trace the PID.

And if you just need the IP, even easier to add %r directly there :)



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Dovecot and Outlook - IMAP Timeout error

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-25 at 08:37 -0500, Phillip T. George wrote:
> Hey guys,
> 
> I'm having a problem with Dovecot and Outlook.  I'm testing with Outlook 
> XP (2002), but the problem occurs with Outlook 2003 as well.  I'm using 
> Dovecot 1.0.rc28.
> 
> The specific error generated (for IMAP) is:
> "A connection to the server could not be established.   is 
> now in offline mode."
> In the details:
> "There may be a server or network problems or your timeout interval may 
> be too short.
> Protocol: IMAP
> Server: 
> Port: 143"

I think this means that it couldn't get a TCP connection to the server
at all.

> As you can see from the logs, Thunderbird didn't have a problem, which 
> eliminates the possibility of a bad net connection (I'd assume...but I 
> could be wrong).  The problem occurred at 3AM.  Some people during the 
> day can get it every hour or so.  I was able to get it to occur twice 
> (the first time is not shown in these logs).

I couldn't figure out from the log what were Thunderbird and what were
Outlook connections. Or were they all TB? In any case the important
thing would be to know what is in Dovecot's logs when Outlook gives the
error. If there's nothing, then Dovecot never even saw the connection.
If Dovecot logged a "Disconnected" error (but no login) then there was
some problem with logging in, auth_debug=yes might be helpful there.

> 1. Is there any way to fix this Outlook problem?  I have all of the 
> outlook workarounds enabled (along with some for Netscape):
>   imap_client_workarounds = delay-newmail outlook-idle netscape-eoh
>   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh

The error comes before user is even logged in, so these don't matter.

> 2. Should Outlook really be connecting and disconnecting that often?

Yea, it's stupid and keeps reconnecting all the time for some reason.

> 3. Could the problem be occurring because of POP3 and IMAP being used by 
> the same account, and the quick downloading of email via POP3 is 
> eliminating the email for IMAP, thereby causing inactivity in IMAP?  Is 
> there a workaround or a known Outlook settings to fix this?

No idea about this.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Troubleshooting - another suggestion...

2007-05-11 Thread Charles Marcus
Many people still do not provide basic things like dovecot version, and 
there are secondary config files like dovecot-ldap, whose settings are 
not included in the dovecot -n output.


How difficult would it be to add these to the output so that the output 
would include the running dovecot version, 



I guess this could be added. Maybe:

# v1.0.0: /usr/local/etc/dovecot.conf


Works for me... :)

as well as any extra config 
settings from any other files that are loaded when dovecot starts?



Yes, this would be a really nice feature, but probably too kludgy
feature before the config handling rewrite in v2.0.


Cool - I'll keep it on my list to watch for, and remind later when 
you're getting closer to 2.0 in case you forget...


Thanks!

--

Best regards,

Charles


Re: [Dovecot] Troubleshooting - another suggestion...

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-26 at 08:11 -0400, Charles Marcus wrote:
> Many people still do not provide basic things like dovecot version, and 
> there are secondary config files like dovecot-ldap, whose settings are 
> not included in the dovecot -n output.
> 
> How difficult would it be to add these to the output so that the output 
> would include the running dovecot version, 

I guess this could be added. Maybe:

# v1.0.0: /usr/local/etc/dovecot.conf

> as well as any extra config 
> settings from any other files that are loaded when dovecot starts?

Yes, this would be a really nice feature, but probably too kludgy
feature before the config handling rewrite in v2.0.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bruce Bodger


On May 11, 2007, at 8:48 AM, Bill Cole wrote:

My system.log is rotated nightly followed by a pretty standard  
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/ 
syslog.pid | head -1);


That's not a restart, it is a 'hangup' signal that will make a  
normal syslogd reread the config file. The problem with that is  
that when you do it, Apple's syslogd completely stops logging. I'm  
not sure why that is, but I've been working around it since the  
10.4 release and Apple has had a bug report for that long.


If you actually kill syslogd with a TERM or KILL signal it will be  
restarted by launchd and it should log properly.


Of course you're right regarding HANGUP vs. restart.  However, that's  
the line that appears in Apple's default daily cron job.


I've just tried modifying it to remove -HUP from the line and force  
it to truly be restarted by launchd.  Same result regarding dovecot,  
though.



Killing and restarting dovecot causes it to write to the log again.


That's a bit surprising, since if you are using syslog, the dovecot  
processes don't know anything about the specific log files that  
syslog  is using. The implication of what you are seeing is that  
the breakage is between dovecot and syslogd, and that restarting  
dovecot fixes it.


That would be accurate, Bill.  I suspect the problem to be within  
dovecot since, as I mentioned earlier, neither uw-imap or other  
running daemons exhibit this problem.


Thanks for the feedback.

Regards,
Bruce


Re: [Dovecot] lucene plug-in

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-26 at 09:38 +0800, ??? wrote:
> Hello,
> 
> Has anyone tried the lucene plug-in? I am interested to know more about this 
> feature , how to setup / configure it?

protocol imap {
  mail_plugins = fts fts_lucene
}

plugin {
  fts = lucene
}

Then you'll also need to use X-TEXT-FAST and X-BODY-FAST SEARCH
parameters instead of the standard TEXT and BODY parameters. And of
course there doesn't exist any clients that support those, so this
feature was mostly meant for custom webmail modifications.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] newbie with dovecot acls needs a little help :-)

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-25 at 18:49 +0200, Rachid Zarouali wrote:
> hy all,
> 
> i'm trying to make an acl so a local unix user 'sie' can access exalead 
> mboxes.
> my exalead mboxes are stored in :
> /opt/exalead/mail/sie/
..
> mail_extra_groups: exalead
> mail_location: mbox:/opt/exalead/mail

A bit late, but if you haven't figured out anything yet:

ACL plugin can only be used to remove permissions that would otherwise
be given by filesystem permissions. So I guess your problem is that
whatever user is trying to access the sie mboxes doesn't have permission
to do so. Is it really enough to have users in exalead group to be able
to access the mboxes? Anything in Dovecot's logs?



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] corruption?

2007-05-11 Thread Timo Sirainen
On Wed, 2007-04-25 at 08:28 -0400, Neal Becker wrote:
> I'm using maildir (and have been for a long time).  Today I have a problem:
> 
> ls -al (maildir):
> [...]
> drwx--  5 nbecker nbecker4096 Jan  1  2006 .;SECTION=
> 
> What's this?  It seems to be causing my mail client (kmail, using dimap), to
> hang.

That's a mailbox. You can create it with CREATE ";SECTION=" IMAP
command, and that's what the client did for some reason. If it hangs
kmail, it's a kmail bug (especially if it created it in the first
place).



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Deliver sending bounces from 'MAILER-DAEMON@'

2007-05-11 Thread Timo Sirainen
On Tue, 2007-04-24 at 12:49 +0200, Erland Nylend wrote:
> | Apr 24 11:42:08 smtp2 postfix/qmgr[6176]: 05BAE3B67E: from=<>, size=3055, 
> nrcpt=1 (queue active)
> | Apr 24 11:42:08 smtp2 postfix/qmgr[6176]: 05BAE3B67E: to=, 
> relay=none, delay=0.01, delays=0.01/0.01/0/0, dsn=5.1.3, status=bounced (bad 
> address syntax)
..
> It seems to me that dovecot is sending bounce messages from
> 'MAILER-DAEMON@' ..

It says to= so I'd guess the Return-Path: header in the
message was that?



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] maildir file name flags out of order?

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 13:27 -0400, Aaron Solochek wrote:
> I was trying to figure out why my false-positive spam messages were
> being tagged as "personal" by thunderbird, and I noticed a possible
> problem.  According to DJB: http://cr.yp.to/proto/maildir.html
> 
> > New flags may be defined later. Flags must be stored in ASCII order: e.g., 
> > "2,FRS".
> 
> Looking at my maildir, I am seeing flags out of order, such as "2,Sdae"
> 
> Shouldn't that be "2,Sade"?

Yes, it should. Fixed:
http://dovecot.org/list/dovecot-cvs/2007-May/008736.html



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Log data on dovecot-auth socket

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 16:39 +0200, Koen Vermeer wrote:
> Hi,
> 
> Is there any way I can monitor or log the data that is being
> received/sent on the dovecot-auth socket? I'm trying to debug some
> authentication stuff, and monitoring the conversation would be very
> useful.

A bit late, but auth_debug_passwords=yes makes it log most of the
traffic I think. If not, then I guess there aren't any easy ways.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Disconnected for inactivity in reading our output top=0/0

2007-05-11 Thread Timo Sirainen
On Thu, 2007-04-19 at 13:36 +1000, Chris Gosley wrote:
> Apr 19 12:51:49 mail dovecot: POP3(vu0095): Disconnected for inactivity in
> reading our output top=0/0, retr=1/0, del=0/2, size=25266
> Apr 19 12:51:49 mail dovecot: POP3(vu0095): Disconnected: Logged out
> top=0/0, retr=2/25393, del=2/2, size=25266
> 
> 
> 99% of the time everything is perfect, it is only the three users in the one
> office that get this error, it is of course the head office for the customer
> and so they are very upset, because they get timeout errors on their email
> clients.
> 
> They are using different versions of outlook, and outlook express, I have
> deactivated all nortons email scanning etc

Did you figure out anything? Enabling rawlog
(http://dovecot.org/bugreport.html) for those users and looking at them
might show if the mails contain something special that causes the client
(or Dovecot) to break.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] read only bug?

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-09 at 15:30 +0200, Nagyon Almos wrote:
> I split my mailboxes with mb2md which was
> a very bad decision: not every mailbox started 
> with the INTERNAL FOLDER DATA email and it lost them.
> It seems to have mixed up the filesnames too, because
> ls 1154939281.5.pc1\,* | wc -l 
> says 10 and they are different emails!

The base filename continues up to the ':' character in the filename, so
if there different characters between the '.' and ':' then they're still
treated as different files. Try something like:

ls|sed 's/:2,.*//'|sort|uniq -D

> Is there any way to make dovecot to not correct these?
> Or it is a very important that maildir files should have different
> basenames?

If you have 10 files with the same basename, Dovecot shows only one of
them and the rest are ignored (and if it had write permissions, they
might get overwritten as well).



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] some clarification re: ACL?

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-09 at 11:15 -0400, Matt Zukowski wrote:
> Perhaps adopting Courier-IMAP's ACL specification might make sense? Tab 
> is the delimiter instead of space, so spaces are allowed in identifiers. 

This can't be changed anymore without breaking backwards compatibility.
There could of course be some new setting or maybe the file format could
be changed in a way that it allows spaces in identifiers but is still
backwards compatible, but I'm not sure what would be the best way to do
it.

> Also, negative permissions are possible (i.e. +r to grant read access, 
> -r to disable it).

This should work with Dovecot too.

> Also, I believe precedence is based on the order of 
> the rules in the file, so you could have the group permissions override 
> the user permissions by placing the group constraint further down. 

I don't think this is a good idea. Especially after I add support for
IMAP ACL extension. It doesn't support ordering rules. group-override=xx
is for that.

> Anyway, there's also probably some benefit in using a common ACL 
> format... interoperability is a good thing (unless you're Microsoft).

I thought the dovecot-acl file was compatible with Courier, but I guess
not if I missed the TAB separator thing. :)

> Regarding the group permissions not working, am I to understand that as 
> of now they are implemented but don't actually do anything? i.e. how can 
> I make use of the group constraints?

The code looks like this (src/plugins/acl/acl-storage.c):

backend = acl_backend_init(acl_env, storage, user_env, NULL,
  owner_username);

The NULL in there is the list of groups the user is in. So if you wanted
to use groups, you'd have to set it somehow. Perhaps this would work:

backend = acl_backend_init(acl_env, storage, user_env,
getenv("GROUPS") == NULL ? NULL : t_strsplit(getenv("GROUPS", 
","),
owner_username);

Then you'll need to return "groups=a,b,c,d" from userdb. I'm not yet
sure how I want this to work in Dovecot v1.1.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bill Cole
At 7:04 AM -0400 5/11/07, Bruce Bodger  imposed structure on a stream 
of electrons, yielding:

It appears as though dovecot doesn't like syslogd being restarted.


That's probably not it.

The whole point of syslog is for individual message sources (e.g. 
dovecot) to not have to worry about where the messages go.


This is actually at least partly a bug in the MacOS X syslog 
implementation. With 10.4, Apple decided that neither traditional BSD 
syslog or syslog-ng was what they wanted, so they reimplemented from 
the ground up and did a lousy job.


My system.log is rotated nightly followed by a pretty standard 
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat 
/var/run/syslog.pid | head -1);


That's not a restart, it is a 'hangup' signal that will make a normal 
syslogd reread the config file. The problem with that is that when 
you do it, Apple's syslogd completely stops logging. I'm not sure why 
that is, but I've been working around it since the 10.4 release and 
Apple has had a bug report for that long.


If you actually kill syslogd with a TERM or KILL signal it will be 
restarted by launchd and it should log properly.




It appears from that point on dovecot no longer writes to mail.log.



Dovecot was never writing to mail.log. Syslogd was, and it was broken.


Killing and restarting dovecot causes it to write to the log again.


That's a bit surprising, since if you are using syslog, the dovecot 
processes don't know anything about the specific log files that 
syslog  is using. The implication of what you are seeing is that the 
breakage is between dovecot and syslogd, and that restarting dovecot 
fixes it.




--
Bill Cole  
[EMAIL PROTECTED]




Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Charles Marcus

Bruce Bodger wrote:
Thank you very much, Charles.  I have resorted to that workaround 
(restarting dovecot) which does, indeed, work.  I just wanted to 
document the situation since dovecot's behavior in this regard is unlike 
other imap servers.


This thread mentioned only sending an HUP - or maybe even just a USR1 - 
signal - so, maybe restarting dovecot (which interrupts all processes) 
is not necessary?


Specifically:


With 1.0-tests you could instead send HUP signal which makes Dovecot
reopen the log files as well as re-reading the configuration files.
Hmm. Looks like Apache uses USR1 signal as well for just log 
reopening, I'll add that too.


--

Best regards,

Charles


Re: [Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround

2007-05-11 Thread Timo Sirainen
On Tue, 2007-05-08 at 14:36 -0400, Matt Zukowski wrote:
> The shared mailbox and all its files and subdirectories are owned by the 
> 'dovecot' user and by the 'domain users' group that all users belong to. 

dovecot user should never own anything at all in the filesystem.
http://wiki.dovecot.org/UserIds http://wiki.dovecot.org/VirtualUsers

> And as I said, the user= constraint seems to work fine, but 
> group= does not. It looks like the group= 
> constraint just never matches anyone.

They don't. Groups aren't supported currently.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Dovecot not handling r/o mailboxes completely, and problem with ACL as a workaround

2007-05-11 Thread Timo Sirainen
On Sun, 2007-05-06 at 21:34 -0400, Adam McDougall wrote:
> In short, upper directories are used to limit world access to just one group 
> 'decsall', and once inside, r/o world access could be used as an anonymous 
> substitution for 'decsall' and the group permissions would allow writing by 
> 'decsstaff'.  I don't know how courier stores and checks mail directory 
> permissions, but it was aware when a user would not be able to delete or 
> change 
> messages and would return a READ-ONLY imap error when an attempt to change a 
> message occurred

Looks like Dovecot never returns maildir mailbox as READ-ONLY. So, your
issues are:

1. Dovecot doesn't give an error when a flag is changed for mails. This
is intentional, because otherwise you'd keep seeing "mailbox is
read-only" errors all the time when reading mails (because clients keep
setting the \Seen flag). Besides you're using a per-user index directory
so the flag changes actually are saved.

2. EXPUNGE gives an internal error because it discovers too late that it
can't unlink() the maildir file. This is a problem.. Even worse, you
can't recover from it without deleting dovecot.index.log file. This
should have two fixes:

 - If Dovecot ever notices that it can't unlink() a file, it should
recover itself from it without giving internal errors. That would
probably mean that the expunged message shows up as a new message. I'll
see if I can get this fixed for v1.1.

 - It should treat the maildir as READ-ONLY so the EXPUNGE command is
never written into dovecot.index.log in the first place. Fixed:
http://dovecot.org/list/dovecot-cvs/2007-May/008734.html

3. EXPUNGE silently ignores no-permissions error. I'm not sure if this
is good or not. In any case it shouldn't give an error if there are no
messages with \Deleted flag, but perhaps it should give an error if
there are.. Some clients do automatic expunges, so it would be annoying
to see error messages constantly about it. I won't change this in v1.0
branch, but perhaps in v1.1.

> As a workaround, I wondered if I could apply dovecot ACLs to the public 
> folders 
> to reinforce my intentions of only allowing readonly access to certain users.

ACL plugin makes flag changes and expunges also fail silently.

> ACL questions (which might help others understand better as well):
> Is a 'user=foo rl' entry supposed to remove any possible write permissions in 
> a 
> maildir covered by such an ACL for user foo?

Yes.

> Is it even possible for an ACL to grant additional permissions above what is 
> granted to a user by standard unix permissions?

No.

> Does an empty ACL equate to the exact permissions granted by unix,

No. It means "no permissions".

>  and should an 
> admin try to design an ACL to modify as little as possible, or should the 
> admin 
> try to use enough acl entries to form a complete description of the desired 
> resulting permissions? I get the impression that broad ACLs should be listed 
> first, and more specific ACLS listed below can add or remove ACL permissions 
> applied earlier in the file as a 'last match' case.

No comments about that, except that user=x always overrides group=y in
the file. I'm not sure if the order matters within group= lines,
probably yes. Then again groups aren't actually working at all currently
(the code supports them, but there's no way to list user's groups
currently).

> Should the acl plugin be able to print a summary of the resulting permissions 
> in 
> the log (with mail_debug=yes) when accessing a folder with an ACL?

Maybe. Not that high priority and I'm not sure if it's that easy to
implement.


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bruce Bodger
Thank you very much, Charles.  I have resorted to that workaround  
(restarting dovecot) which does, indeed, work.  I just wanted to  
document the situation since dovecot's behavior in this regard is  
unlike other imap servers.


Best regards,
Bruce


On May 11, 2007, at 8:27 AM, Charles Marcus wrote:


Bruce Bodger wrote:

It appears as though dovecot doesn't like syslogd being restarted.


Hmmm... could this possibly be a fix, or at least a workaround?

www.dovecot.org/list/dovecot/2004-November/005219.html

--

Best regards,

Charles




Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Charles Marcus

Bruce Bodger wrote:

It appears as though dovecot doesn't like syslogd being restarted.


Hmmm... could this possibly be a fix, or at least a workaround?

www.dovecot.org/list/dovecot/2004-November/005219.html

--

Best regards,

Charles


[Dovecot] Problems with BEFORE searches.

2007-05-11 Thread Trever L. Adams
Ok, I am trying to modify some code given to me by the author of the 
dspam plugin for dovecot (sorry the name eludes me). I am modifying it 
to handle UNSEEN differently than SEEN.


Below is the python code. I am not a python programmer, but I get the 
basic idea.


ok, uids = conn.search(None, 'BEFORE', unreadsentbefore, 'SEEN')

"11-May-2007 06:18:29 -0600" is the value of unreadsentbefore.

However, this doesn't match any emails even though they are all dated as 
5/10 and they have all been read for several hours (the time was the 
current to figure out why it isn't working).


Is there a problem with BEFORE and SEEN together? Is there a BEFORE bug 
in the system? Some of the messages are -500 and others, I am not sure.


Thanks,
Trever Adams


Re: [Dovecot] global vs user script

2007-05-11 Thread Magnus Holmgren
On Friday 11 May 2007 03:53, M1 wrote:
> Dear Mark,
>
> Yes, you're right. I posted the wrong link.
>
> I do not know much about rfc system. What does it mean by expired?

Internet Drafts must be updated, or submitted for official review and 
publication as an RFC, within six months, otherwise they are considered dead, 
which means they are deleted from the IETF's public repository.

http://www.ietf.org/ietf/1id-guidelines.html#expiry
http://en.wikipedia.org/wiki/Internet_Draft

-- 
Magnus Holmgren[EMAIL PROTECTED]
   (No Cc of list mail needed, thanks)

  "Exim is better at being younger, whereas sendmail is better for 
   Scrabble (50 point bonus for clearing your rack)" -- Dave Evans


pgp4rseT5uRX5.pgp
Description: PGP signature


Re: [Dovecot] Problems with Shared Mailbox

2007-05-11 Thread Timo Sirainen
On Mon, 2007-05-07 at 14:11 +0200, Gerhard Schmidt wrote:
>   location = 
> mbox:/var/mail/shared:CONTROL=/var/mail/shared/.control:INDEX=/var/mail/shared/.index

mbox has no CONTROL directory.

> -rw-rw  1 dovecot  vhsal  0 May  7 13:50 dovecot-shared
> -rw---  1 estartu  vhsal376 May  7 13:53 dovecot.index
> -rw-rw  1 dovecot  vhsal  23552 May  7 13:53 dovecot.index.cache
> -rw-rw  1 dovecot  vhsal   7116 May  7 13:53 dovecot.index.log
> 
> How can I gate dovecot to set the permissions of this files to rw-rw 

mbox doesn't support dovecot-shared file either. So unfortunately this
won't work.

Also if you used maildir, the dovecot-shared would have to be in the
maildir directory, not in the index directory.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Error: Master request X not found

2007-05-11 Thread Timo Sirainen
On Tue, 2007-05-08 at 11:15 +0200, Jordi Moles wrote:
> dovecot: May 07 19:57:24 Error: auth(default): Master request 
> 88729.43523 not found

Show your settings with dovecot -n? Is there a high load in the system?
Are there non-plaintext logins?

Maybe this would help:
http://dovecot.org/list/dovecot-cvs/2007-May/008732.html

Although that would mean that a successful authentication lasts for
longer than 3,5 minutes, which is a bit unlikely. But maybe you have
some weird clients. :)


signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] Return error instead of dying on time back skip?

2007-05-11 Thread Timo Sirainen
On Wed, 2007-05-02 at 10:52 +0200, Amon Ott wrote:

> Now I wonder if Dovecot could return errors to the users instead of 
> dying until time is fine again, e.g. "System time has moved 
> backwards, please come back in n seconds". If the time skip is just a 
> few seconds, it can of course delay and then go on as it does now. 
> 
> With this change, no admin would be needed to carefully restart 
> Dovecot at the right time. I have not looked into Dovecot code myself 
> yet, but could try a patch if necessary. 

I think this is just way too much trouble for handling a situation that
really shouldn't be happening in the first place.. The code already
allows the clock to move backwards by 5 seconds without dying, so how
horrible are the clocks in those computers? :)

It's anyway easy to increase that time by modifying the sources.
src/lib/ioloop.c IOLOOP_MAX_TIME_BACKWARDS_SLEEP



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] read only maildir subscriptions

2007-05-11 Thread Timo Sirainen
On Mon, 2007-04-30 at 19:56 +0200, Nagyon Almos wrote:
> Under thunderbird, I can successfully log in and read my new mails, 
> but  I cannot subscribe (?) within thunderbird to oldmails 
> (it is grey not black as other folders).
..
> namespace private {
>   separator = /
>   prefix = oldmails/

"oldmails" is the namespace prefix. You can't subscribe to it because
it's not a mailbox. Why would it even be needed? You should be able to
subscribe to mailboxes under it.



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] logging IMAP size?

2007-05-11 Thread Timo Sirainen
On Tue, 2007-05-01 at 16:57 +0200, Armijn Hemel wrote:
> hi all,
> 
> is there a way to somehow log the size of an IMAP session with Dovecot?
> With POP3 there is this pop3 format string I can set in dovecot.conf,
> but I don't see an equivalent for IMAP.
> 
> If so, are there any plans to add this functionality?

IMAP has multiple mailboxes, so I don't see how this could work in any
useful way. It could of course log just INBOX's size, but since the
client wouldn't necessarily even open the INBOX that would be just extra
work for nothing but a log line.

What do you need it for anyway?



signature.asc
Description: This is a digitally signed message part


Re: [Dovecot] namespace

2007-05-11 Thread Timo Sirainen
On Sun, 2007-05-06 at 20:10 +0200, Stefan Klatt wrote:
> 
>   location:
> mbox:/srv/imapd/shared/archiv/maildir:CONTROL=/srv/imapd/user/%
> u/shared-settings/archiv/
> control:INDEX=/srv/imapd/user/%u/shared-settings/archiv/index
> 
> Dovecot use the index statement with the correct subdirectories and
> indexfiles at /srv/imapd/user/%u/shared-settings/archiv but don't use
> the control statement and use a .subscriptions files at
> /srv/imapd/shared/archiv/maildir instead. :-( 

mbox doesn't have CONTROL setting at all, so this kind of a
configuration unfortunately doesn't work.



signature.asc
Description: This is a digitally signed message part


[Dovecot] error in logfile

2007-05-11 Thread Luigi Rosa

With Dovecot 1.0.0:

May 11 12:16:35 mail dovecot: IMAP(xxx): 
stat(/var/spool/mail2/xxx/.Trash.Squid-it/new) failed: No such file or 
directory



With Thunderbird 2.0 IMAP client:
* user deleted the mailbox
* TB moved the mailbox to Trash
* user asked TB to empty Trash
* the above message appeared

No warning/error was reported bt TB to user.


I don't think it's an error and should not be logged.



Ciao,
luigi

--
/
+--[Luigi Rosa]--
\

You can tune a radio, but you can't tuna fish.



Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bruce Bodger

It appears as though dovecot doesn't like syslogd being restarted.

My system.log is rotated nightly followed by a pretty standard  
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/ 
syslog.pid | head -1);


It appears from that point on dovecot no longer writes to mail.log.   
Killing and restarting dovecot causes it to write to the log again.


If you care to take a stab at a suggestion, please kind in mind that  
I'm having to run dovecot with the -F switch (see below).


Thanks in advance,

B. Bodger
Ronkonkoma, NY



On May 11, 2007, at 6:17 AM, Bruce Bodger wrote:

Please disregard the below reported bug report.  I experienced the  
same problem today and this time "sleep" was not involved.


I rotate mail.log weekly.  This morning I noticed that dovecot was  
no longer logging to mail.log although, again, it was running fine.


Is it possible that dovecot has a problem after midnight and will  
not log with the new day?  Something is stopping dovecot from  
logging and it appears that it might have to do with change of the  
day?


This is a new (two day old) dovecot installation.  Coming from uw- 
imap.  It never exhibited this problem.


Thank you,
B. Bodger
Ronkonkoma, NY


On May 10, 2007, at 7:34 AM, Bruce Bodger wrote:


Version: dovecot --version:  1.0.0
OS: OS X - 10.4.9
Architecture: G5 Dual/1.8Ghz

Had a fairly normal time correction last night:
May  9 20:56:56 G518X2 ntpd[219]: time set -1.212733 s

Which dovecot duly noted:
May  9 20:56:56 G518X2 dovecot: auth(default): Time just moved  
backwards by 1 seconds. I'll sleep now until we're back in present.


This morning all of the dovecot processes are running fine but  
dovecot has stopped logging.  (I have modified nothing in  
dovecot.conf relative to logging: dovecot logs to syslog using the  
"mail" syslog_facility.)


ps -aux | grep dovecot
root53   0.0 -0.027772544  ??  Ss4:23PM
0:04.69 /usr/local/sbin/dovecot -F
root79   0.0 -0.028176556  ??  S 4:23PM
0:03.21 dovecot-auth
dovecot   9548   0.0 -0.027780496  ??  S 7:12AM
0:00.05 imap-login
dovecot   9631   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login
dovecot   9632   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login


I am starting dovecot via OS X's launchdaemon facility thus the "- 
F" command line switch requirement.


Sending a "kill -HUP" to the process did not get it to start logging.
Sending a "Kill -9" to the process forced a restart and logging  
began again.


ps -aux | grep dovecot
root  9738   0.0 -0.027772540  ??  Ss7:26AM
0:00.03 /usr/local/sbin/dovecot -F
root  9739   0.0 -0.028176544  ??  S 7:26AM
0:00.02 dovecot-auth
dovecot   9740   0.0 -0.027780500  ??  S 7:26AM
0:00.02 imap-login
dovecot   9751   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login
dovecot   9762   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login


Before posting this bug I compared v1.0.0's /src/lib/ioloop.c to  
that in the nightly snapshot and saw no adjustments in, what I  
presumed to be, the relevant portion of the code.


Output of "dovecot -n"

protocols: imap
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
first_valid_uid: 490
mail_extra_groups: mail
mail_location: mbox:~/Library/Mail/IMAP:INBOX=/var/mail/%u:INDEX=% 
h/Library/Mail/DOVECOT.indexes

dotlock_use_excl: yes
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd

Thank you.






Re: [Dovecot] BUG REPORT - Stops logging after sleep

2007-05-11 Thread Bruce Bodger
Please disregard the below reported bug report.  I experienced the  
same problem today and this time "sleep" was not involved.


I rotate mail.log weekly.  This morning I noticed that dovecot was no  
longer logging to mail.log although, again, it was running fine.


Is it possible that dovecot has a problem after midnight and will not  
log with the new day?  Something is stopping dovecot from logging and  
it appears that it might have to do with change of the day?


This is a new (two day old) dovecot installation.  Coming from uw- 
imap.  It never exhibited this problem.


Thank you,
B. Bodger
Ronkonkoma, NY


On May 10, 2007, at 7:34 AM, Bruce Bodger wrote:


Version: dovecot --version:  1.0.0
OS: OS X - 10.4.9
Architecture: G5 Dual/1.8Ghz

Had a fairly normal time correction last night:
May  9 20:56:56 G518X2 ntpd[219]: time set -1.212733 s

Which dovecot duly noted:
May  9 20:56:56 G518X2 dovecot: auth(default): Time just moved  
backwards by 1 seconds. I'll sleep now until we're back in present.


This morning all of the dovecot processes are running fine but  
dovecot has stopped logging.  (I have modified nothing in  
dovecot.conf relative to logging: dovecot logs to syslog using the  
"mail" syslog_facility.)


ps -aux | grep dovecot
root53   0.0 -0.027772544  ??  Ss4:23PM
0:04.69 /usr/local/sbin/dovecot -F
root79   0.0 -0.028176556  ??  S 4:23PM
0:03.21 dovecot-auth
dovecot   9548   0.0 -0.027780496  ??  S 7:12AM
0:00.05 imap-login
dovecot   9631   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login
dovecot   9632   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login


I am starting dovecot via OS X's launchdaemon facility thus the "- 
F" command line switch requirement.


Sending a "kill -HUP" to the process did not get it to start logging.
Sending a "Kill -9" to the process forced a restart and logging  
began again.


ps -aux | grep dovecot
root  9738   0.0 -0.027772540  ??  Ss7:26AM
0:00.03 /usr/local/sbin/dovecot -F
root  9739   0.0 -0.028176544  ??  S 7:26AM
0:00.02 dovecot-auth
dovecot   9740   0.0 -0.027780500  ??  S 7:26AM
0:00.02 imap-login
dovecot   9751   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login
dovecot   9762   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login


Before posting this bug I compared v1.0.0's /src/lib/ioloop.c to  
that in the nightly snapshot and saw no adjustments in, what I  
presumed to be, the relevant portion of the code.


Output of "dovecot -n"

protocols: imap
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
first_valid_uid: 490
mail_extra_groups: mail
mail_location: mbox:~/Library/Mail/IMAP:INBOX=/var/mail/%u:INDEX=%h/ 
Library/Mail/DOVECOT.indexes

dotlock_use_excl: yes
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd

Thank you.




Re: [Dovecot] Sieve plugin

2007-05-11 Thread Gunter Ohrner
Am Freitag, 11. Mai 2007 schrieb Luigi Rosa:
> "It's also not a good idea to build the plugin against self-compiled
> Dovecot sources, but then actually use a prebuilt binary package of
> Dovecot."

> Why isn't a good idea?

In this case there there's a significant probability that either the 
plugin was compiled against a different dovecot version, using a 
different compiler than the dovecot build and/or using different shared 
library versions to link against, each of which can lead to obscure and 
hard to debug misbehaviour / "bugs", which could be easily avoided.

So either use both programs as prepackaged binaries which belong together, 
or build both yourself from the same sources, using the same compiler and 
the same library versions.

Greetings,

  Gunter

-- 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Plankton lobbyist: "NUKE THE WHALES!"
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   PGP-verschlüsselte Mails bevorzugt! +
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


pgprgV3l78tY2.pgp
Description: PGP signature