Re: [Dovecot] how to talk with doveadmin unix socket

[Nikita Koshikov]

On Thu, Feb 28, 2013 at 1:53 AM, Piotr Rotter wrote:

> Hello,
>
> I would like to get mailbox quota by doveadm unix socket
> (/var/run/dovecor/doveadm-**server). I get from strace syntax of
> comunication:
>
> write(9, 
> "VERSION\t1\t0\nUSER\t1\tpiotr**.rot...@active24.pl
> \tservice=**doveadm\n", 60) = 60
> read(9, "VERSION\t1\t1\nSPID\t20805\n"**, 8192) = 23
> read(9, "USER\t1\tpiotr.rotter@**active24.pl 
> \temail=piotr.**rot...@active24.pl \tclear=**
> 7T33z456\thome=/vm/08/A/active**24.pl/piotr.rotter/\tuid=502\**
> tgid=502\tquota_rule=*:**storage=2097152000B\tquota_**
> rule2=*:messages=5\n",
> 8169) = 196
>
> and I wrote super simple client in php:
>
>  $sock = stream_socket_client('unix:///**var/run/dovecot/doveadm-**server',
> $errno, $errstr);
> fwrite($sock, 
> "VERSION\t1\t0\nUSER\t1\tpiotr**.rot...@active24.pl
> \tservice=**doveadm\n");
> echo fread($sock, 4096)."\n";
> echo fread($sock, 4096)."\n";
> fclose($sock);
> ?>
>
> But i get only "+" sign and line in errorlog as below:
>
> 2013-02-28T00:39:56.377437+01:**00 kurier9 dovecot: doveadm: Error:
> doveadm client not compatible with this server (mixed old and new binaries?)
>
> I would be very grateful for your help with this problem, what I doing
> wrong.
>
> My current dovecot version is 2.1.13
>
>
Here is my example on perl

socket(TSOCK, PF_UNIX, SOCK_STREAM,0);
connect(TSOCK, sockaddr_un("$conf{quota_socket}"));

if ($? != 0) {
  #error here
}

#After connect on socket, dovecot returns "+" or "-". Just a simple
check for data.
if (defined(my $answer = )) {

print TSOCK "VERSION\tdoveadm-server\t1\t0\n";
print TSOCK "PLAIN\t$conf{'doveadm_password'}\n";
TSOCK->flush;
#'+' here
$answer=;
print TSOCK "\t$local_part\@$domain\tquota get\n";
TSOCK->flush;

my $quota_data = ;


#"+" if everything is OK
$answer = ;

close TSOCK;
}

Don't forget to set doveadm_password= in dovecot config(clearttext) and you
must encode password in base64 for using in a script (after PLAIN command).

PS
dovecot version 2.1.15 here.

Re: [Dovecot] Default fallback behaviour

[Nikita Koshikov]

Yes, thanks a lot - this config is working as expected.

Re: [Dovecot] Default fallback behaviour

[Nikita Koshikov]

On Tue, Nov 27, 2012 at 3:04 AM, Timo Sirainen  wrote:

> On 23.11.2012, at 9.46, Nikita Koshikov wrote:
>
> > Hello list,
> >
> > Here is the problem:
> > I have few:
> > passdb {
> > #1
> > }
> > passdb {
> > #2
> > }
> > And relative userdb sections. If user not found in 1) section it
> fallbacks
> > to next one - it's expected and right, IMHO. But when the user exists in
> > both section and password verification fails on 1) database it
> successfully
> > authenticated on next one. I think this behaviour should be configured.
> The
> > main goal of 1) section for this server is to overwrite users in main
> > (section2) database.
>
> It's not always possible to know why #1 failed. For example PAM doesn't
> always tell if the password was wrong or if the user didn't exist.
>
> > Maybe I missed something and this option is already in dovecot code and I
> > can't find it ? Or if not - will it be added in the future ?
>
>
> I'm not very interested in adding it, especially because it can't be done
> reliably.
>
>
Thank's for the anwer. It's a pity to hear, because it's security feature I
need to provide. The problem  - that main passdb - is ldap and there are
about - 5-7 people who can edit it and simply to login as different users.
Yes, activity is logged - but mailbox can be read\stolen. The main goal for
passwd-file database is to revrite ldap very critical mailboxes to local
file. It can be edited only but 1 person - it is nativly to trust 1, but
not to 7.

Re: [Dovecot] Default fallback behaviour

[Nikita Koshikov]

On Fri, 23 Nov 2012 12:20:23 +0100
Alessio Cecchi wrote:

> Il 23/11/2012 08:46, Nikita Koshikov ha scritto:
> > Hello list,
> >
> > Here is the problem:
> > I have few:
> > passdb {
> > #1
> > }
> > passdb {
> > #2
> > }
> > And relative userdb sections. If user not found in 1) section it fallbacks
> > to next one - it's expected and right, IMHO. But when the user exists in
> > both section and password verification fails on 1) database it successfully
> > authenticated on next one. I think this behaviour should be configured. The
> > main goal of 1) section for this server is to overwrite users in main
> > (section2) database.
> >
> > Maybe I missed something and this option is already in dovecot code and I
> > can't find it ? Or if not - will it be added in the future ?
> >
> > Dovecot version 2.1.10.
> >
> 
> You can enable this features by adding "deny=yes" in the passdb 
> extra_fields of specific users.
> 
> You can find more information here: 
> http://wiki2.dovecot.org/AuthDatabase/PasswdFile ot you can use the 
> "auth-deny.conf.ext" example configuration.
> 
> Ciao

Maybe I was not so clear - but this is not what I'm searching for.
deny=yes will deny user in corresponding database - I want - allow user to 
login, if and only if, users password matches and if it fail(in current passdb) 
- not trying next passdb for checking his password, even if user exists in the 
next database.

[Dovecot] Default fallback behaviour

[Nikita Koshikov]

Hello list,

Here is the problem:
I have few:
passdb {
#1
}
passdb {
#2
}
And relative userdb sections. If user not found in 1) section it fallbacks
to next one - it's expected and right, IMHO. But when the user exists in
both section and password verification fails on 1) database it successfully
authenticated on next one. I think this behaviour should be configured. The
main goal of 1) section for this server is to overwrite users in main
(section2) database.

Maybe I missed something and this option is already in dovecot code and I
can't find it ? Or if not - will it be added in the future ?

Dovecot version 2.1.10.

[Dovecot] Libdovecot-sieve segfaults

[Nikita Koshikov]

Hello Stephan, 

Preface - one of my users went to vacation and changed his sieve script to run 
autoreply vacation message, after that - I have noticed some annoying errors in 
system log, like:

dovecot-lda[2400]: segfault at 3ebf ip b7446c7a sp bfaab050 error 4 in 
libdovecot-sieve.so.0.0.0[b7429000+5b000]

Dovecot-error log is empty, but in exim log(exim runs dovecot-lda) I got:

2012-08-13 09:35:27 1T0oFK-aW-6j ** s...@domain.com (s...@domain.com, 
r...@domain.com)  F=<59942270...@mymail.in.ua> 
R=ldap_accept T=local_delivery: Child process of local_delivery transport 
(running command "/usr/libexec/dovecot/dovecot-lda -e -d $local_part@$domain") 
was terminated by signal 11 (Segmentation fault)

s...@domain.com - is the user, who created such sieve-script, which cause 
segfaults.
I have tried run the same message over this script from sieve-test utility - 
but result is success.

Can you please, take a look to this sieve script and check is it ok, I'll send 
it to you privately since it contains user's data. Also I'll attach message.eml 
which triggers segfault.

Dovecot version - 2.0.21, Sieve version - 0.2.6. 
Relevant to sieve dovecot config section:
plugin {
  sieve = ~/.dovecot.sieve
  sieve_after = /etc/dovecot/sieve/default-after.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +notify +spamtest
  sieve_spamtest_max_value = 10
  sieve_spamtest_status_header = X-Spam-Scan-Score
  sieve_spamtest_status_type = score
}

Hope you'll help.

Re: [Dovecot] LDAP Lookup not returning value in maxStorage

[Nikita Koshikov]

On Wed, 28 Mar 2012 09:39:37 +1300
Bruce, Andrew wrote:

> On 28 March 2012 09:36, Bruce, Andrew  wrote:
> > On 27 March 2012 19:14, Nikita Koshikov  wrote:
> >> On Tue, 27 Mar 2012 13:57:04 +1300
> >> Bruce, Andrew wrote:
> >>
> >> Hi there,
> >>
> >> We're setting up a Dovecot virtual email setup - we've got everything
> >> working perfect with LDAP logins authenticating against AD and so
> >> forth, but we're having issues with retrieving the maxStorage value
> >> from AD (this is a pre-setup field in AD that we'd like to use to set
> >> per user quotas).
> >>
> >> In our LDAP lookup, we have the maxStorage entry listed under
> >> user_attrs for the quota (user_attrs =
> >> maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see
> >> it trying to get the entry, but it fails with:
> >> Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): user
> >> search: base=dc=site,dc=local scope=subtree
> >> filter=(&(objectClass=person)(| (userPrincipalName=username@site)
> >> (|(mail=username@site)(samAccountName=username@site
> >> fields=maxStorage
> >> Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): no
> >> fields returned by the server
> >>
> >> At this point, we then see the default quota applied.
> >>
> > Try to change your quota rule to be like:
> > maxStorage=quota_rule=*:bytes=%$
> >                        ^
> > And put the value in bytes to maxStorage - if I remember correct - this is 
> > integer field and no K\M\G values is valid here.
> >
> > PS We successfully using maxStorage field to obtain non-default quota from 
> > AD, dovecot version 2.0.x
> >>
> >> If we change the name of the field from maxStorage to instanceType we
> >> see the value show up in the logs and passed through to the quota
> >> system and applied successfully:
> >> Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): user
> >> search: base=dc=site,dc=local scope=subtree
> >> filter=(&(objectClass=person)(| (userPrincipalName=username@site)
> >> (|(mail=username@site)(samAccountName=username@site
> >> fields=instanceType
> >> Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): result:
> >> instanceType(quota_rule=*:storage=%$M)=*:storage=4M
> >> Mar 27 11:09:01 auth: Debug: master out: USER   3901227009
> >> username@site    quota_rule=*:storage=4M
> >>
> >>
> >> Which seems a bit weird.
> >>
> >> If we use ldapsearch and pass it the same search string and look for
> >> the field maxStorage, we clearly see the field and the value being
> >> returned.  The result looks the same if we also lookup instanceType.
> >>
> >> We're using Dovecot 2.0.9.
> >>
> >> Does anyone have any idea as to why we can't use this field?
> >>
> >> Thanks,
> >>
> >> Andrew
> 
> Tried your suggestion Nikita, no joy unfortunately.  It still looks
> like the value never gets returned from the LDAP server to Dovecot.
> It definitely has something in the field (equivalent of 10GB, but in
> bytes as suggested) and I changed the user_attrs also, but still get
> the same "no fields returned by the server" error message.
> 
> Modifying the user_attrs to lookup from a different field
> (instanceType) definitely works.
> 
> What exact version are you using - perhaps it's a problem with our
> copy of 2.0.9.
> 
> Thanks,
> 
> Andrew

Show your full dovecot-ldap.conf file, also what port do you using ? maybe you 
met restriction of ldap port 3268?(http://wiki2.dovecot.org/AuthDatabase/LDAP) 

And show exact result of ldapsearch tool, binding under user from 
dovecot-ldap.conf + debug for this user when it trying to login and 'doveadm -D 
 quota get -u $user' for this one.

Also ensure that your search query returns only 1 result.

We are using dovecot 2.0.19 now, but all versions of dovecot 2.0 branch was 
there in the past. I'm updating server since version 2.0.1 - no problem found.

Re: [Dovecot] LDAP Lookup not returning value in maxStorage

[Nikita Koshikov]

On Tue, 27 Mar 2012 13:57:04 +1300
Bruce, Andrew wrote:

> Hi there,
> 
> We're setting up a Dovecot virtual email setup - we've got everything
> working perfect with LDAP logins authenticating against AD and so
> forth, but we're having issues with retrieving the maxStorage value
> from AD (this is a pre-setup field in AD that we'd like to use to set
> per user quotas).
> 
> In our LDAP lookup, we have the maxStorage entry listed under
> user_attrs for the quota (user_attrs =
> maxStorage=quota_rule=*:storage=%$M), and in the debug logs, can see
> it trying to get the entry, but it fails with:
> Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): user
> search: base=dc=site,dc=local scope=subtree
> filter=(&(objectClass=person)(| (userPrincipalName=username@site)
> (|(mail=username@site)(samAccountName=username@site
> fields=maxStorage
> Mar 27 13:19:27 auth: Debug: ldap(username@site,192.168.1.5): no
> fields returned by the server
> 
> At this point, we then see the default quota applied.
> 
Try to change your quota rule to be like:
maxStorage=quota_rule=*:bytes=%$
^
And put the value in bytes to maxStorage - if I remember correct - this is 
integer field and no K\M\G values is valid here.

PS We successfully using maxStorage field to obtain non-default quota from AD, 
dovecot version 2.0.x
> 
> If we change the name of the field from maxStorage to instanceType we
> see the value show up in the logs and passed through to the quota
> system and applied successfully:
> Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): user
> search: base=dc=site,dc=local scope=subtree
> filter=(&(objectClass=person)(| (userPrincipalName=username@site)
> (|(mail=username@site)(samAccountName=username@site
> fields=instanceType
> Mar 27 11:09:01 auth: Debug: ldap(username@site,192.168.1.5): result:
> instanceType(quota_rule=*:storage=%$M)=*:storage=4M
> Mar 27 11:09:01 auth: Debug: master out: USER   3901227009
> username@sitequota_rule=*:storage=4M
> 
> 
> Which seems a bit weird.
> 
> If we use ldapsearch and pass it the same search string and look for
> the field maxStorage, we clearly see the field and the value being
> returned.  The result looks the same if we also lookup instanceType.
> 
> We're using Dovecot 2.0.9.
> 
> Does anyone have any idea as to why we can't use this field?
> 
> Thanks,
> 
> Andrew

Re: [Dovecot] Timeout leak in get quota

[Nikita Koshikov]

On Thu, 29 Sep 2011 15:05:53 +0200
Danilo wrote:

> Hello!
> I wrote perl script for read current quota usage, it works but when I run the
> program I receive a warning in Dovecot log:
> 
> 2011-09-29 14:45:30 doveadm(gu...@testmail.com): Warning: Timeout leak:
> 0x7f14800ad970
> 
> This is the script:
> ===
> #!/usr/bin/perl
> 
> use strict;
> use Socket;
> use IO::Handle;
> 
> socket(TSOCK, PF_UNIX, SOCK_STREAM,0);
> connect(TSOCK, sockaddr_un("/var/run/dovecot/doveadm-server")) or 
> print("ERROR!");
> 
> print "VERSION\tdoveadm-server\t1\t0\n";
> print "\tguest\@testmail.com\tquota get\n";
> print "=" x 20 . "\n";
> 
> if (defined(my $messg = )) {
> 
> print $messg;
> 
>   print TSOCK "VERSION\tdoveadm-server\t1\t0\n";
>   TSOCK->flush;
> 
>   print TSOCK "\tguest\@testmail.com\tquota get\n";
>   TSOCK->flush;
> 
>   $messg = ;
>   print $messg;
> 
>   $messg = ;
>   print $messg;
> 
>   close TSOCK;
> }
> =
> I hope someone could help me to fix the warning.
> Thanks in advance,
>Dany

I'm using doveadm socket too for getting quota values. No warning. The main 
difference from yours script that I'm also using auth:

print TSOCK "VERSION\tdoveadm-server\t1\t0\n";
print TSOCK "PLAIN\tbase64_encoded_pass_here\n";
print TSOCK "\t$local_part\@$domain\tquota get\n";
TSOCK->flush;

Dovecot version 2.0.15

Re: [Dovecot] Dovecot-Antispam 2.0 with SpamAssassin (sa-learn)

[Nikita Koshikov]

On Tue, 16 Aug 2011 22:38:34 +0200
Nick Rosier wrote:

> Hi,
> 
> has anyone figured out how to use the dovecot-antispam plugin with 
> sa-learn? I need to pass the username (u...@domain.tld) to sa-learn and 
> --ham or --spam depending on the move. I was thinking of using the CRM14 
> option and replace the command and arguments but cannot find anything 
> about the ability to pass the username.
> Any pointers would be welcome.
> 
> N.
Write a wrapper on sa-learn. The script should parse env first and that exec 
learning with appropriative args, like:
antispam_mail_sendmail = /etc/dovecot/plugins/spam.sh

Scrip, something like:
#!/bin/bash

(
env > /tmp/antispam.$$
/usr/bin/sa-learn $@
exit 0
)
search for PWD environment in  /tmp/antispam.$$ .

Re: [Dovecot] Dovecot quota dict

[Nikita Koshikov]

On Tue, 09 Aug 2011 11:18:55 -0300
Juan Bernhard wrote:

> Hi Nikita, thanks for the help, but I don't have any problem with 
> bouncing the mail after the deliver (in fact, the mail pass two MTAs 
> servers before reaching the final storage)
> I will try (with my limited english) to explain it better. Now I m 
> working with a single quota rule for everyone. What I need is a 
> mechanism to give a custom quota to each user. I thought that a flat 
> file will be the simple and easy option to implement, and then use 
> "quota=${lookup {$local_part} lsearch .}" in the exim transport. I 
> need a way to configure dovecot to follow the same file (or at least a 
> modified copy). I've been told on this list that I can't rely on the 
> maildirsize file, because it can be deleted under some conditions... so 
> i neet to stablish quotas to dovecot somewhere else.
> My question is: can dovecot get a custon quota for a user from a file 
> (not sql or ldap) and use at the same time pam as userdb?
> 
> 
> Thanks, Juan.

So, you need customizable userdb lookup without ldap or sql. You can try too 
use checkpassword as userdb and then, in the script - parse quota-file, make 
pam lookup and return userdb_quota* values for specific user. More on 
http://wiki2.dovecot.org/AuthDatabase/CheckPassword . I don't have experience 
with this authdatabase method, but seems that it suitable for you. 
But in general - it's better to prevent editing maildirsize file from 2 
instances.

Re: [Dovecot] Dovecot quota dict

[Nikita Koshikov]

On Mon, 08 Aug 2011 16:35:53 -0300
Juan Bernhard wrote:

> Hi, Im using dovecot with maildir++ quota and I have this problem, maybe 
> some of you already solve this and can helpme.
> I want to use some flat file to tell exim and dovecot the quota limit 
> for each user, i tried to work with exim and maildirsize file, but exim 
> just ignores the limits there.
> I need a way (a really simple one, i dont want to create a database or 
> an ldap server just for this) to tell exim and dovecot to lookup some 
> file to get the quota limits. I know how to doit with exim, now i need 
> to know how to tell dovecot to read the quota limits from this file. I 
> know that dovecot can read the quota limit from the userdb, but im using 
> pam and i dont want to change the entire auth schema.
> Can I configure dovecot to get the quota limit from a file using pam as 
> userdb?
> 
> Thanks, Juan.
> 
> pd: here is my doveconf -n
> 
> # 2.0.13: /usr/local/etc/dovecot/dovecot.conf
> # OS: Linux 2.6.18-238.9.1.el5 x86_64 CentOS release 5.6 (Final) ext3
> auth_cache_size = 10 M
> auth_debug = yes
> auth_verbose = yes
> base_dir = /var/run/dovecot/
> default_client_limit = 5000
> default_process_limit = 1000
> disable_plaintext_auth = no
> listen = *
> login_greeting = Server ready.
> mail_debug = yes
> mail_location = maildir:/var/mail/%u:INDEX=~/
> mail_plugins = quota
> maildir_very_dirty_syncs = yes
> passdb {
>args = failure_show_msg=yes %s
>driver = pam
> }
> plugin {
>mail_log_events = delete undelete expunge copy mailbox_delete 
> mailbox_rename
>mail_log_fields = uid box msgid size subject
>quota = maildir:Cuota de correo
>quota_rule = *:storage=1G
>quota_rule2 = Trash:storage=+100M
> }
> protocols = imap pop3
> service auth {
>client_limit = 12000
> }
> service imap {
>process_limit = 5000
> }
> service pop3 {
>process_limit = 5000
> }
> ssl = no
> ssl_cert =  ssl_key =  userdb {
>driver = passwd
> }
> protocol imap {
>imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags
>mail_plugins = quota imap_quota
> }
> protocol pop3 {
>pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>pop3_lock_session = yes
>pop3_uidl_format = %08Xv%08Xu
> }
> 

Try to google "exim smtp-time quota bounce". This is what you need.
Exim can't do rcpt acl quota checks from the box, thus you need:
1)Some external script which will periodically create overquota file\database 
of users and reject them at smtp-time using lookup
2)Or write perl-script and exec it in router stage. Script will open 
maildirsize file, count quota and gives you 'yes\no'. 

Also you need to deliver via dovecot deliver. This scheme will work.

Re: [Dovecot] Pigeon Sieve, redirect action and SPF

[Nikita Koshikov]

On Tue, 02 Aug 2011 16:43:13 +0200
Stephan Bosch wrote:

> On 7/29/2011 8:53 AM, Nikita Koshikov wrote:
> > On Thu, 28 Jul 2011 20:43:45 +0200
> > Stephan Bosch wrote:
> >
> >> Would the header name need to be configurable, or is 
> >> X-Sieve-Redirected-From good for all purposes? Regards, Stephan. 
> > I saw this
> >
> > X-ResentFrom:
> > X-SRS-Rewrite:
> > X-Forwarded-(To/For):
> >
> > So the name is not significant. If it's not so difficult to code - than 
> > configurable header of course is better. But it's only a matter of taste.
> 
> Adding more settings is best to be avoided. I've added the header:
> 
> http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/81b37c0055c3
> http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/efad75f779de (d'oh)
> 
> > One thing to note for implementing:
> > It will be much more simple for setting up srs, if sieve will export to 
> > environment some variables indicating need for rewrite. And than we will be 
> > able to write wrapper for sendmail_path that will selectively choose what 
> > should we do:
> > For example, I don't want srs to happen if user is forwarding for local 
> > domain mailbox, or if action that trigger mail sending is - "out of office 
> > reply".
> > So, wrapper will analyze need srs or no, and after all invoke srs binary 
> > for signing, or just call pure mta.
> 
> I am not sure Sieve has access to enough information to make such an 
> assessment. And I don't think adding SPF/SRS-specific code to Sieve is a 
> good idea. It would also not be very straight-forward since the 
> low-level mail sending code is part of Dovecot itself.
> 
> Regards,
> 
> Stephan.

I meant export to environment some variables, like dovecot does:
USER=
IP=
MAIL=

Adding more like:
SIEVE_ACTION=
SIEVE_RECIPIENT=
etc
And then execute sendmail_path. Thus program\wrapper can be aware of mail 
originated from sieve. This is not specific spf\srs code. Seemed, that this 
section of code need to be in dovecot-lda region, so maybe Timo will be able to 
catch it. 

Thanks for your work.

Re: [Dovecot] Pigeon Sieve, redirect action and SPF

[Nikita Koshikov]

On Thu, 28 Jul 2011 20:43:45 +0200
Stephan Bosch wrote:

> On 07/26/2011 08:37 AM, Nikita Koshikov wrote:
> > I have raised this problem earlier here 
> > (http://www.mail-archive.com/dovecot@dovecot.org/msg20945.html), but seems 
> > it stuck at that point.
> >
> > The simple way, is to use redirecting user mail_from, so messages will 
> > appear like originated from redirecting smtp server.
> 
> Yes. I read a bit about this though, and obviously this breaks bounce 
> processing.
> 
> 
> > The right way for redirecting messages with sieve is to use SRS mechanism. 
> > And this should be configured on SMTP server.
> 
> Right.
> 
> > The problem is when sieve exec sendmail_path smtp server has no clue about 
> > "redirect" recipient( user who wrote the sieve script) and srs can't be 
> > configured. So adding one more header when redirecting sieve message with 
> > original sender will be half of the work - another part it to set up smtp 
> > for understanding this header and construct new 'mail from:' address in 
> > appropriative way(something like 
> > srs0_forward_user-fuzzy_staff-orig.address@forwarded_domain. Exact format 
> > should be determine.). Maybe this will be hard to setup, but we at least 
> > have something(original sender address and forwarded one) and this is 
> > another story.
> 
> Would the header name need to be configurable, or is 
> X-Sieve-Redirected-From good for all purposes?
> 
> Regards,
> 
> Stephan.
> 
> 
I saw this

X-ResentFrom:
X-SRS-Rewrite:
X-Forwarded-(To/For):

So the name is not significant. If it's not so difficult to code - than 
configurable header of course is better. But it's only a matter of taste. 

One thing to note for implementing:
It will be much more simple for setting up srs, if sieve will export to 
environment some variables indicating need for rewrite. And than we will be 
able to write wrapper for sendmail_path that will selectively choose what 
should we do:
For example, I don't want srs to happen if user is forwarding for local domain 
mailbox, or if action that trigger mail sending is - "out of office reply".
So, wrapper will analyze need srs or no, and after all invoke srs binary for 
signing, or just call pure mta.

Re: [Dovecot] Pigeon Sieve, redirect action and SPF

[Nikita Koshikov]

On Tue, 26 Jul 2011 01:13:36 +0200
Stephan Bosch wrote:

> Hi Pedro,
> 
> On 7/25/2011 6:53 PM, Pedro Ribeiro wrote:
> > Since I've upgraded to dovecot 2.0.13 + Pigeonhole 0.2.3 (Gentoo 
> > package) I've received a few complains of users about rejected messages.
> >
> > Investigating the problem, I've seen that when the external sender 
> > server publishes SPF information, when some message is sent from there 
> > to one of my users that has a Sieve redirect action active to another 
> > external system (that does SPF validation), the message is rejected 
> > because our system only changes the envelope "rcpt to" in the 
> > redirection and the "mail from" remains the original.
> > When the end server does the SPF validation it fails because my server 
> > isn't in the IP allow list for the domain of the original sender.
> 
> Right.
> 
> > I never seen this problem in the old Dovecot+CMUSieve.
> 
> What version? The sources I checked (v1.1 and v1.2) use the return path 
> of the original message, e.g.:
> 
> http://hg.dovecot.org/dovecot-1.1/file/3698dfe0f21c/src/deliver/mail-send.c#l186
> 
> >
> > Shouldn't the sender be changed in the envelope "mail from" from the 
> > original one to the recipient that made the Sieve redirection?
> 
> The now obsolete RFC3028 only mentions modifying the envelope recipient, 
> which is probably what the CMUSieve plugin is based on. The behavior of 
> Pigeonhole Sieve in such ambiguous cases is mostly derived from CMUSieve 
> for backwards compatibility, which is why - indeed - only the envelope 
> recipient is modified.
> 
> However, the new RFC5228 is not very strict on this point (Section 4.2):
> 
> The envelope sender address on the outgoing message is chosen by the
> sieve implementation.  It MAY be copied from the message being
> processed.  However, if the message being processed has an empty
> envelope sender address the outgoing message MUST also have an empty
> envelope sender address.  This last requirement is imposed to prevent
> loops in the case where a message is redirected to an invalid address
> when then returns a delivery status notification that also ends up
> being redirected to the same invalid address.
> 
> This suggests making this configurable, which I think would be no problem.
> 
> Its a bit late, I'll give this a look later.
> 
> Also, do others have a view on this? Is there perhaps an easier way to 
> fix this issue, other than adding a config option?
> 
> Regards,
> 
> Stephan.
> 
I have raised this problem earlier here 
(http://www.mail-archive.com/dovecot@dovecot.org/msg20945.html), but seems it 
stuck at that point. 

The simple way, is to use redirecting user mail_from, so messages will appear 
like originated from redirecting smtp server.

The right way for redirecting messages with sieve is to use SRS mechanism. And 
this should be configured on SMTP server. The problem is when sieve exec 
sendmail_path smtp server has no clue about "redirect" recipient( user who 
wrote the sieve script) and srs can't be configured. So adding one more header 
when redirecting sieve message with original sender will be half of the work - 
another part it to set up smtp for understanding this header and construct new 
'mail from:' address in appropriative way(something like 
srs0_forward_user-fuzzy_staff-orig.address@forwarded_domain. Exact format 
should be determine.). Maybe this will be hard to setup, but we at least have 
something(original sender address and forwarded one) and this is another 
story...

[Dovecot] sqlite quota dict

[Nikita Koshikov]

Hello Timo, and list followers,

Sqlite supports good feature like mysql 'ON DUPLICATE KEY UPDATE' - it calls 
REPLACE/INSERT OR REPLACE

This clause can be added to INSERT query, like:
INSERT OR REPLACE INTO table VALUES()
REPLACE INTO table VALUES() // alias to previous one 

Also this can be done during CREATE TABLE syntax, like so:

CREATE TABLE quota (
username text not null,
bytes integer not null default 0,
messages integer not null default 0,
primary key (username) ON CONFLICT REPLACE
);

So the examples will work:
INSERT INTO quota (bytes,username) VALUES ('112497180','f...@spam.dom');
INSERT INTO quota (bytes,username) VALUES ('112497181','f...@spam.dom');
SELECT * FROM quota;
f...@spam.dom|112497181|0

But 1 thing to note - if we updating not all fields and column has NOT NULL 
DEFAULT value, then DEFAULT value will alter current one:

INSERT INTO  quota (bytes,username) VALUES ('112497180','f...@spam.dom');
INSERT INTO quota (messages,username) VALUES ('1743','f...@spam.dom');
SELECT * FROM quota;
f...@spam.dom|0|1743

This can be easily bypassed if INSERT will be:
INSERT INTO  quota (bytes,messages,username) VALUES 
('112497180','1743','f...@spam.dom');

So if dovecot will INSERT values to sql dict in this manner, sqlite can be used 
for storing quota data.

Re: [Dovecot] lmtp logging to separate file

[Nikita Koshikov]

On Sat, Feb 12, 2011 at 2:23 PM, Timo Sirainen  wrote:

> On 12.2.2011, at 9.46, Nikita Koshikov wrote:
>
> >> service lmtp {
> >>  executable = lmtp -L
> >> }
> >>
> >>
> > This works, thank you.
> >
> > Quick src code view gave more lmtp params, can you tell what is relevant
> command line keys for this options and maybe comment some of them? wiki2
> still lacks all this info.
>
> These flags have just about nothing to do with command line parameters.
> They're all about how the binary should work correctly.
>
> > MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT |
>
> User root isn't allowed for mail actions (pretty much everything has this,
> root is evil).
>
> > MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP |
>
> Users are looked up via userdb lookup, rather than from environment
> variables and current process euid.
>
> > MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP |
>
> Privileges are dropped only temporarily (this is the biggest thing I hate
> about LMTP processes - they effectively run as root).
>
> > MAIL_STORAGE_SERVICE_FLAG_NO_LOG_INIT |
>
> The "mail storage service" code doesn't change anything related to logging.
> I don't remember exactly why this matters, but I think lmtp does this
> internally.
>
> > MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT;
>
> UNIX socket connections to auth-userdb aren't disconnected. Mostly just an
> optimization.


Thanks for explanation.

Re: [Dovecot] lmtp logging to separate file

[Nikita Koshikov]

On Fri, 11 Feb 2011 23:27:45 +0200
Timo Sirainen wrote:

> On Thu, 2011-02-10 at 15:27 +0200, Nikita Koshikov wrote:
> > Hello list,
> > 
> > Testing lmtp deliver I can't constrain it to log all activity to separate 
> > file, like dovecot-lda does. Here is config:
> 
> LMTP processes are started by dovecot master process, so it logs through
> it. But you can avoid this with:
> 
> service lmtp {
>   executable = lmtp -L
> }
> 
> 
This works, thank you.

Quick src code view gave more lmtp params, can you tell what is relevant 
command line keys for this options and maybe comment some of them? wiki2 still 
lacks all this info.

MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT |
MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP |
MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP |
MAIL_STORAGE_SERVICE_FLAG_NO_LOG_INIT |
MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT;

[Dovecot] lmtp logging to separate file

[Nikita Koshikov]

Hello list,

Testing lmtp deliver I can't constrain it to log all activity to separate file, 
like dovecot-lda does. Here is config:

protocol lmtp {
  info_log_path = /var/log/dovecot/dovecot-deliver.log
  log_path = /var/log/dovecot/dovecot-deliver.log
  mail_plugins = acl quota trash virtual zlib expire sieve
}
protocol lda {
  info_log_path = /var/log/dovecot/dovecot-deliver.log
  log_path = /var/log/dovecot/dovecot-deliver.log
  mail_plugins = acl quota trash virtual zlib expire sieve
}

When message is proceed via protocol lda - record appear in 
/var/log/dovecot/dovecot-deliver.log, but when message arrives by protocol lmtp 
- logs got to /var/log/dovecot.log file.

Is this settings suppose to be overwritten ?

Re: [Dovecot] LDAPS fault tolerance not working with dovecot

[Nikita Koshikov]

On Thu, 10 Feb 2011 01:00:16 +0200
Timo Sirainen wrote:

> On Wed, 2011-02-09 at 11:41 +0200, Nikita Koshikov wrote:
> 
> > uris = ldaps://host1 ldaps://host2 ldaps://host2
> > 
> > Today host1 hangs and new connections can't be established with mail
> > server. Connected users worked fine (auth_cache_size = 5 k
> > auth_cache_ttl = 15 mins). At this time - host2 and host3 are working
> > fine, but switching to them was not happen. Why ?
> 
> Well, I can only say that it's libldap's (= OpenLDAP's) responsibility
> to do the switch automatically.
> 
> 
Ok, I'll try to get answer there. 

Thanks for response.

[Dovecot] LDAPS fault tolerance not working with dovecot

[Nikita Koshikov]

Hello list, 

In my dovecot-ldap.ext I have:

uris = ldaps://host1 ldaps://host2 ldaps://host2

Today host1 hangs and new connections can't be established with mail server. 
Connected users worked fine (auth_cache_size = 5 k auth_cache_ttl = 15 mins). 
At this time - host2 and host3 are working fine, but switching to them was not 
happen. Why ?

In logs I got:
Feb 09 10:20:36 imap-login: Error: Timeout waiting for handshake from auth 
server. my pid=29932, input bytes=0
Feb 09 10:20:36 imap-login: Error: Timeout waiting for handshake from auth 
server. my pid=29867, input bytes=0
Feb 09 10:20:37 imap-login: Error: auth: connect(login) failed: Resource 
temporarily unavailable
Feb 09 10:20:37 imap-login: Error: auth: connect(login) failed: Resource 
temporarily unavailable
Feb 09 10:20:39 imap-login: Error: Timeout waiting for handshake from auth 
server. my pid=29943, input bytes=0
Feb 09 10:20:40 imap-login: Error: auth: connect(login) failed: Resource 
temporarily unavailable

I also tried to set:
hosts = host1:636 host2:636 host3:636

But with above config error log fulls with messages:
Feb 09 10:44:57 auth: Error: LDAP: Connection lost to LDAP server, reconnecting

And none of servers accepted ldaps connection. 

Is someone has a success make ldaps fault tolerance ?

Re: [Dovecot] Sieve 0.2.2 how to use sievec

[Nikita Koshikov]

On Tue, 18 Jan 2011 06:23:42 -0500
Charles Marcus wrote:

> On 2011-01-18 5:04 AM, Nikita Koshikov wrote:
> > 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.14
> 
> Why are you using a kernel that is no longer even in portage?
> 
It just working and well tested under current server's hardware. Updating is 
planned once a year.

Re: [Dovecot] Sieve 0.2.2 how to use sievec

[Nikita Koshikov]

On Tue, 18 Jan 2011 11:56:29 +0200
Timo Sirainen wrote:

> On 17.1.2011, at 10.02, Nikita Koshikov wrote:
> 
> > sievec(root): Error: user root: Initialization failed: Namespace '': 
> > mkdir(/root/data) failed: Permission denied (euid=8(mail) egid=12(mail) 
> > missing +w perm: /root, euid is not dir owner)
> 
> Show your doveconf -n output.
> 
> 
root@mail dovecot 0:0 # doveconf -n
# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.14 
auth_cache_negative_ttl = 0
auth_cache_size = 5 k
auth_cache_ttl = 15 mins
auth_default_realm = domain.com
auth_failure_delay = 1 secs
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_format = %Lu
auth_worker_max_count = 50
debug_log_path = /var/log/dovecot/debug.log
dict {
  acl = sqlite:/etc/dovecot/plugins/acl.conf
  expire = sqlite:/etc/dovecot/plugins/expire.conf
}
first_valid_gid = 12
first_valid_uid = 8
info_log_path = /var/log/dovecot/dovecot.log
last_valid_gid = 12
last_valid_uid = 8
listen = *
log_path = /var/log/dovecot/dovecot-error.log
login_greeting = Server ready.
mail_gid = 12
mail_location = maildir:~/.data
mail_log_prefix = "%Us(%u %r): "
mail_plugins = acl quota trash virtual zlib
mail_uid = 8
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date spamtest notify
namespace {
  inbox = yes
  list = yes
  location = maildir:~/data
  prefix = 
  separator = /
  subscriptions = yes
  type = private
}
namespace {
  hidden = yes
  list = yes
  location = virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
  prefix = Company/
  separator = /
  subscriptions = no
  type = private
}
namespace {
  list = children
  location = maildir:%%h/data:INDEX=~/shared/%%u
  prefix = Shared/%%u/
  separator = /
  subscriptions = no
  type = shared
}
passdb {
  args = /etc/dovecot/passdb/master.pwd
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/passdb/users.pwd
  driver = passwd-file
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  acl = vfile:/etc/dovecot/acl:cache_secs=300
  acl_shared_dict = proxy::acl
  antispam_allow_append_to_spam = no
  antispam_backend = mailtrain
  antispam_mail_notspam = --ham
  antispam_mail_sendmail = /etc/dovecot/plugins/spam.sh
  antispam_mail_sendmail_args = -u mail
  antispam_mail_spam = --spam
  antispam_mail_tmpdir = /tmp
  antispam_spam = Spam
  antispam_trash = Trash
  autocreate = Drafts
  autocreate2 = Sent
  autocreate3 = Spam
  autocreate4 = Trash
  autosubscribe = Drafts
  autosubscribe2 = Sent
  autosubscribe3 = Spam
  autosubscribe4 = Trash
  expire = Trash
  expire2 = Trash/*
  expire3 = Spam
  expire_dict = proxy::expire
  quota = maildir:Mailbox quota
  quota_rule = *:storage=750M
  quota_rule2 = Trash:storage=+10%%
  quota_warning = storage=90%% quota-warning 95 %u
  sieve = ~/.dovecot.sieve
  sieve_after = /etc/dovecot/sieve/default-after.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +notify +spamtest
  sieve_spamtest_max_value = 10
  sieve_spamtest_status_header = X-Spam-Scan-Score
  sieve_spamtest_status_type = score
  trash = /etc/dovecot/plugins/dovecot-trash.conf
}
protocols = imap pop3 sieve
service anvil {
  client_limit = 3096
}
service auth-worker {
  unix_listener auth-worker {
group = dovecot_auth
mode = 0660
  }
  user = dovecot_auth
}
service auth {
  client_limit = 12288
  unix_listener auth-client {
group = dovecot_auth
mode = 0660
user = mail
  }
  unix_listener auth-userdb {
mode = 0600
user = mail
  }
  user = dovecot_auth
}
service dict {
  unix_listener dict {
mode = 0600
user = mail
  }
  user = mail
}
service imap-login {
  inet_listener imap {
port = 0
  }
  process_limit = 2048
  process_min_avail = 50
}
service imap {
  drop_priv_before_exec = yes
  process_limit = 2048
}
service managesieve-login {
  process_limit = 50
  process_min_avail = 3
}
service managesieve {
  drop_priv_before_exec = yes
  process_limit = 50
}
service pop3-login {
  inet_listener pop3 {
port = 0
  }
  process_limit = 20
  process_min_avail = 5
}
service pop3 {
  drop_priv_before_exec = yes
  process_limit = 20
}
service quota-warning {
  executable = script /etc/dovecot/plugins/quota_warning.sh
  unix_listener quota-warning {
mode = 0600
user = mail
  }
  user = mail
}
ssl_cert = 

Re: [Dovecot] Sieve 0.2.2 how to use sievec

[Nikita Koshikov]

On Tue, 18 Jan 2011 01:02:14 +0100
fakessh @ wrote:

> 
> I managed to run in the root folder lda and sievec . I add the group
> mail to root more +w flag and everything should work properly
> 

My global scripts located in /etc/dovecot/sieve. IMHO it's not a good idea to 
allow write access to this folder or add group mail to root. I think, the 
problem can be easy enclosed if sievec won't drop privileges and have 
comandline option to redefine user's home dir. 

[Dovecot] BUG:v2.0.9 + expire plugin segfault

[Nikita Koshikov]

Hello list,

New version of dovecot segfaulting with expire plugin(sqlite db). Steps to 
reproduce:
1)DB file has permitions:
-rw-r--r-- 1 mail mail 87040 Янв 17 10:04 /var/mail/expire.db

2)Dict dovecot config section:
service dict {
  # If dict proxy is used, mail processes should have access to its socket.
  # For example: mode=0660, group=vmail and global mail_access_groups=vmail
  unix_listener dict {
mode = 0600
user = mail
  }
  #user = mail
}

3)Trying to add new record to database I get:
Jan 17 10:03:12 dict: Error: sqlite: exec(INSERT INTO expires 
(expire_stamp,username,mailbox) VALUES 
('1294852805','u...@domain.com','Trash')) fail
ed: attempt to write a readonly database (8)
Jan 17 10:03:12 IMAP(u...@domain.com 192.168.2.252): Error: 
read(/var/run/dovecot/dict) failed: Remote disconnected
Jan 17 10:03:12 master: Error: service(dict): child 13957 killed with signal 11 
(core not dumped)

This crash has gone after service dict got enough permission to access db 
file(user mail in my case).

Don't know how to get backtrace with dict service, but can reproduce crash any 
time.

[Dovecot] Sieve 0.2.2 how to use sievec

[Nikita Koshikov]

Hello list, 

How can I compile sieve script under new 0.2.2 version from comandline? 

Under root:
#sievec default-before.sieve
sievec(root): Error: user root: Initialization failed: Namespace '': 
mkdir(/root/data) failed: Permission denied (euid=8(mail) egid=12(mail) missing 
+w perm: /root, euid is not dir owner)
sievec(root): Fatal: Invalid user settings. Refer to server log for more 
information.

Re: [Dovecot] Feature request

[Nikita Koshikov]

On Mon, 13 Dec 2010 06:16:56 -0500
Charles Marcus wrote:

> On 2010-12-13 2:45 AM, Nikita Koshikov wrote:
> > On Sat, 11 Dec 2010 10:23:43 -0500, Charles Marcus wrote:
> >> Did you already try deleting the local copy of the Inbox?
> 
> > The problem has solved. User account was recreated - that helped. 
> > Also, we have this problem with various dovecot (starting from 1.2x
> > branch) and various thunderbird releases. I don't think this is
> > dovecot issue. Dovecot works very well.
> 
> When you say you recreated the user account - you mean in Thunderbird?
> 
Yes, account was recreated on client side(in thunderbird).

> And you never said whether or not you actually did try just deleting the
> local cached copy of the Inbox?
>
This also helped with other acoounts, I mean deleting stuck syncing mailfolder. 
And after restarting thunderbird - problem disappeared.
 
> Also - what version of Thunderbird is this?
> 

This time it was thunderbird 3.0.2 . Few times this situation happened with 
thunderbird 3.1.x - latest version on that moment.

Re: [Dovecot] Feature request

[Nikita Koshikov]

On Sat, 11 Dec 2010 10:23:43 -0500
Charles Marcus wrote:

> On 2010-12-11 3:00 AM, Nikita Koshikov wrote:
> > Hello Timo, and all.
> > 
> > Preface:
> > From time to time we have problems with thunderbird. The problem is
> > that mail client begin to sync users inbox (or other folder) and this
> > process didn't stop. Last time we have this situation - our
> > monitoring system shows unusual amount of imap traffic send to one of
> > local hosts. This host was - terminal server and 20 users worked with
> > their mailboxes that time. But normal 20 imap session can't fork that
> > amount of traffic (5.2Gb). After measuring local thunderbird's cached
> > mailboxes we find out that one user have local INBOX about 19Gb and
> > size is rising. On server side - this mailbox was 89Mb. And after
> > killing thunderbird in logs appear:
> 
> Did you already try deleting the local copy of the Inbox?
> 
> Also...
> 
> Dovecot version/doveconf -n output?
> 
The problem has solved. User account was recreated - that helped. 
Also, we have this problem with various  dovecot(starting from 1.2x branch) and 
various thunderbird releases. I don't think this is dovecot issue. Dovecot 
works very well.

dovecot -n:

# 1.2.16: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.14 
log_path: /var/log/dovecot/dovecot-error.log
info_log_path: /var/log/dovecot/dovecot.log
protocols: imaps pop3s managesieve
ssl_cert_file: /etc/ssl/dovecot/imaps.crt
ssl_key_file: /etc/ssl/dovecot/imaps.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting: Server ready.
login_processes_count(default): 50
login_processes_count(imap): 50
login_processes_count(pop3): 5
login_processes_count(managesieve): 5
login_max_processes_count: 2048
max_mail_processes: 2048
mail_max_userip_connections(default): 25
mail_max_userip_connections(imap): 25
mail_max_userip_connections(pop3): 10
mail_max_userip_connections(managesieve): 10
first_valid_uid: 8
last_valid_uid: 8
first_valid_gid: 12
last_valid_gid: 12
mail_uid: 8
mail_gid: 12
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): quota imap_quota trash expire zlib autocreate virtual 
antispam
mail_plugins(imap): quota imap_quota trash expire zlib autocreate virtual 
antispam
mail_plugins(pop3): quota virtual
mail_plugins(managesieve): 
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/pop3
mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve
mail_log_prefix: %Us(%u %r): 
imap_client_workarounds(default): delay-newmail
imap_client_workarounds(imap): delay-newmail
imap_client_workarounds(pop3): 
imap_client_workarounds(managesieve): 
pop3_client_workarounds(default): 
pop3_client_workarounds(imap): 
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
pop3_client_workarounds(managesieve): 
namespace:
  type: private
  separator: /
  location: maildir:~/data
  inbox: yes
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: Company/
  location: virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
  hidden: yes
  list: yes
lda:
  postmaster_address: postmas...@agromat.ua
  hostname: mail.agromat.ua
  mail_plugins: quota trash expire sieve virtual
  quota_full_tempfail: yes
  sendmail_path: /usr/sbin/sendmail
  auth_socket_path: /var/run/dovecot/auth-master
  log_path: /var/log/dovecot/dovecot-deliver.log
  info_log_path: /var/log/dovecot/dovecot-deliver.log
auth default:
  mechanisms: plain login
  default_realm: agromat.ua
  cache_size: 5120
  cache_ttl: 900
  cache_negative_ttl: 0
  user: dovecot_auth
  username_format: %Lu
  master_user_separator: *
  worker_max_count: 50
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/master.pwd
master: yes
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  passdb:
driver: ldap
args: /etc/dovecot/dovecot-ldap.conf
  userdb:
driver: prefetch
  userdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  userdb:
driver: ldap
args: /etc/dovecot/dovecot-userdb-ldap.conf
  socket:
type: listen
client:
  path: /var/run/dovecot/auth-client
  mode: 432
  user: mail
  group: dovecot_auth
master:
  path: /var/run/dovecot/auth-master
  mode: 384
  user: mail
  group: mail
plugin:
  quota_warning: storage=90%% /etc/dovecot/plugins/quota

[Dovecot] Feature request

[Nikita Koshikov]

Hello Timo, and all.

Preface:
From time to time we have problems with thunderbird. The problem is that mail 
client begin to sync users inbox (or other folder) and this process didn't 
stop. Last time we have this situation - our monitoring system shows unusual 
amount of imap traffic send to one of local hosts. This host was - terminal 
server and 20 users worked with their mailboxes that time. But normal 20 imap 
session can't fork that amount of traffic (5.2Gb). After measuring local 
thunderbird's cached mailboxes we find out that one user have local INBOX about 
19Gb and size is rising. On server side - this mailbox was 89Mb. And after 
killing thunderbird in logs appear:
Dec 10 17:35:53 IMAP(u...@domain 192.168.2.92): Info: Disconnected: Logged out 
bytes=1133593/1175103
Dec 10 17:35:53 IMAP(u...@domain 192.168.2.92): Info: Disconnected: Logged out 
bytes=15964/44364
Dec 10 17:35:53 IMAP(u...@domain 192.168.2.92): Info: Disconnected: Logged out 
bytes=54758/944989
Dec 10 17:35:53 IMAP(u...@domain 192.168.2.92): Info: Disconnected: Logged out 
bytes=6532/43930
Dec 10 17:35:53 IMAP(dom...@domain 192.168.2.92): Info: Connection closed 
bytes=235408/4797272694

As you can see - 4.8Gb of data was send to this client - and who knows - what 
was that. So we have 5.2Gb send to host with 20 active sessions and 4.8Gb send 
to one user. Of course lifetime of this connection can be much more than last 
day, but problem disappeared after killing it.

This is not single accident. But this one was hard to find out because of 20-25 
users working at the same time from 1 server.

So it will be great if dovecot has feature to limit the amount of data send per 
connection. For example:

service imap {
bytes_per_connection = 100Mb
}

And after limit reached - close connection and log warning. This will much 
simplify  fights with sometimes become crazy imap clients.

That do you think?

Re: [Dovecot] imap_logout_format don't understand %r variable

[Nikita Koshikov]

On Fri, 10 Dec 2010 07:26:39 +
Timo Sirainen wrote:

> On 10.12.2010, at 7.17, Nikita Koshikov wrote:
> 
> > imap_logout_format = ip=%r bytes=%i/%o
> > 
> > But when user disconnecting, to log file dovecot write:
> > IMAP(u...@domain): Info: Disconnected: Logged out ip= bytes=954/8644
> > 
> > Is this suppose to work ?
> 
> No. And I'm not sure if it should. Maybe. Anyway an alternative that already 
> works is:
> 
> mail_log_prefix = "%s(%u %r): "
> 
That works. Thank you.

[Dovecot] imap_logout_format don't understand %r variable

[Nikita Koshikov]

Hello list, 

Dovecot version 1.2.16. In protocol imap section I have

imap_logout_format = ip=%r bytes=%i/%o

But when user disconnecting, to log file dovecot write:
IMAP(u...@domain): Info: Disconnected: Logged out ip= bytes=954/8644

Is this suppose to work ?

Re: [Dovecot] Shared mailboxes errors

[Nikita Koshikov]

On Tue, 03 Aug 2010 23:16:38 +0200
Leander S. wrote:

>   Hi  Nikita Koshikov,
> 
> when I googled for my SERVERBUG which I'm having right now I found your 
> configuration.
> I'm trying to make use of the antispam plugin as you do. Unfortunately 
> I'm always getting a [SERVERBUG] error mesage with my MTC when I try 
> moving mails. The antispam debug.log doesn't really tell a lot of more:
> 
> 
>### Dovecot AntiSpam ###
> # mail signature (used with any backend requiring a signature)
> #antispam_signature = X-Spam-Status
> #antispam_signature_missing = move
> 
> antispam_mail_sendmail = /usr/local/bin/sa-learn
> antispam_mail_sendmail_args = --username=%u;--debug;all
> antispam_mail_spam = --spam
> antispam_mail_notspam = --ham
> antispam_mail_tmpdir = /tmp
> antispam_spam = Spam
> antispam_unsure = Virus
> antispam_trash = Trash
> 

First of all - what OS is this ? BSD ? And how did you install your 
spamassassin (ports\compiling from source)?
Show your local.cf file (/etc/mail/spamassassin or 
/usr/local/etc/mail/spamassasin). 

After reading perldoc Mail::SpamAssassin::Conf add to your local.cf for 
debugging:
bayes_file_mode 0777
bayes_path /tmp/.spamassassin/bayes

Create folder /tmp/.spamassassin and set permitions 777. 
Restart spamd with dovecot  and try it.


> I also read through the sa-learn script but wasn't able to figure out 
> where this number 9 is coming from ;/
> ... SIGKILL 9 Term Kill signal - but where from ?!
> 
sa-learn has lots of die() function, this should be enough for raising SIGKILL.
> 
> 
> Do you maybe have any idea? I tried different Dovecot versions already - 
> alwys the same ... Might there maybe somethign wrong with my syntax?
> 
> 
> 
> Any way - thanks a lot in advance & regards
> 
If you stuck after all, remember that sa-learn has -D key, it generate tons of 
output. Create wrapper and run sa-learn with -D keyword from dovecot, save 
result and have fun.

Re: [Dovecot] Shared mailboxes errors

[Nikita Koshikov]

On Thu, 17 Jun 2010 14:57:08 +0100
Timo Sirainen wrote:

> On Thu, 2010-06-17 at 16:25 +0300, Nikita Koshikov wrote:
> 
> > Jun 17 15:50:33 dict: Error: sql dict iterate: Invalid/unmapped path: 
> > shared/shared-boxes/anyone/
> 
> I think the wiki sql configuration was written/tested by someone who got
> the same errors, but ignored them.. You need to provide also a mapping
> for this. Maybe something like:
> 
> map {
>   pattern = shared/shared-boxes/anyone/$from
>   table = anyone_shares
>   value_field = dummy
> 
>   fields {
>   from_user = $from
>   }
> }
> 
> or maybe to your existing table (just don't have a user called
> "anyone"):
> 
> map {
>   pattern = shared/shared-boxes/user/anyone/$from
>   table = user_shares
>   value_field = dummy
> 
>   fields {
>   from_user = $from
>   }
> }
> 

Thanks for your reply, Timo.

Is any documentation avail about mapping or can you describe it briefly here ?

[Dovecot] Shared mailboxes errors

[Nikita Koshikov]

Hello list,

I'm implementing shared mailboxes on live system and after enabling acl plugin 
I got errors in my log: 

Jun 17 15:50:33 dict: Error: sql dict iterate: Invalid/unmapped path: 
shared/shared-boxes/anyone/

I have no clue why this happening, users don't know about new functionality 
they just use imap as before. Also dict database begin to fill up by records 
like:

select * from user_shares;
u...@domain.com|ad...@domain.com|1

But user don't use setacl command.

Searching for the list gave 
http://www.dovecot.org/list/dovecot/2009-April/038664.html , but question 
seemed still open. So, can someone give point how to fix\avoid this ?


One more question, on live system it's hard to debug dovecot with 
mail_debug=yes for all users, can this option be turn on for individual user? 
or maybe mail_debug stream can be redirected also for individual user ?

dovecot -n 
# 1.2.11: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.13 
log_path: /var/log/dovecot/dovecot-error.log
info_log_path: /var/log/dovecot/dovecot.log
protocols: imaps pop3s managesieve
ssl_cert_file: /etc/ssl/dovecot/imaps.crt
ssl_key_file: /etc/ssl/dovecot/imaps.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(pop3): /usr/libexec/dovecot/pop3-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting: Server ready.
login_processes_count(default): 50
login_processes_count(imap): 50
login_processes_count(pop3): 5
login_processes_count(managesieve): 5
login_max_processes_count: 2048
max_mail_processes: 2048
mail_max_userip_connections(default): 25
mail_max_userip_connections(imap): 25
mail_max_userip_connections(pop3): 10
mail_max_userip_connections(managesieve): 10
first_valid_uid: 8
last_valid_uid: 8
first_valid_gid: 12
last_valid_gid: 12
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(pop3): /usr/libexec/dovecot/pop3
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): quota imap_quota trash expire zlib autocreate virtual 
antispam acl imap_acl
mail_plugins(imap): quota imap_quota trash expire zlib autocreate virtual 
antispam acl imap_acl
mail_plugins(pop3): quota virtual
mail_plugins(managesieve): 
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/pop3
mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve
imap_client_workarounds(default): delay-newmail
imap_client_workarounds(imap): delay-newmail
imap_client_workarounds(pop3): 
imap_client_workarounds(managesieve): 
pop3_client_workarounds(default): 
pop3_client_workarounds(imap): 
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
pop3_client_workarounds(managesieve): 
namespace:
  type: private
  separator: /
  location: maildir:~/data
  inbox: yes
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: Company/
  location: virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
  hidden: yes
  list: yes
namespace:
  type: shared
  separator: /
  prefix: shared/%%u/
  location: maildir:%%h/data:INDEX=%h/shared/%%u
  list: children
lda:
  postmaster_address: postmas...@domain.com
  hostname: mail.domain.com
  mail_plugins: quota trash expire sieve virtual acl
  quota_full_tempfail: yes
  sendmail_path: /usr/sbin/sendmail
  auth_socket_path: /var/run/dovecot/auth-master
  log_path: /var/log/dovecot/dovecot-deliver.log
  info_log_path: /var/log/dovecot/dovecot-deliver.log
auth default:
  mechanisms: plain login
  default_realm: domain.com
  cache_size: 10240
  cache_negative_ttl: 0
  user: dovecot_auth
  username_format: %Lu
  master_user_separator: *
  worker_max_count: 50
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/master.pwd
master: yes
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  passdb:
driver: ldap
args: /etc/dovecot/dovecot-ldap.conf
  userdb:
driver: prefetch
  userdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  userdb:
driver: ldap
args: /etc/dovecot/dovecot-userdb-ldap.conf
  socket:
type: listen
client:
  path: /var/run/dovecot/auth-client
  mode: 432
  user: mail
  group: dovecot_auth
master:
  path: /var/run/dovecot/auth-master
  mode: 384
  user: mail
  group: mail
plugin:
  quota_warning: storage=90%% /etc/dovecot/plugins/quota_warning.sh 90
  quota: maildir:Mailbox quota
  quota_rule: *:storage=500M
  quota_rule2: Trash:storage=10%%
  acl: vfile:/etc/dovecot/acl:cache_secs=3600
  acl_shared_dict: proxy::acl
  trash: /etc/dovecot/plugins/dovecot-trash.conf
  expire: Trash 30 Spam 30
  expire_dict: proxy::expire
  autocreate: Drafts
  autocreate2: Sent
  autocreate3: Spam
  autocreate4:

Re: [Dovecot] sieve spamtest extension

[Nikita Koshikov]

On Sat, 15 May 2010 18:13:38 +0200
Stephan Bosch wrote:

> Nikita Koshikov wrote:
> 
> > If X-Spam-Scan-Score have 6-9 digit after dot - the first one ignored, thus
> > X-Spam-Scan-Score: 3.6 -> "ge" skips. #right
> > X-Spam-Scan-Score: 3.8 -> "ge" works. #wrong
> > X-Spam-Scan-Score: 5.9 -> "ge" works. #wrong
> > X-Spam-Scan-Score: 6.0 -> "ge" works. #right
> > X-Spam-Scan-Score: 6.9 -> "ge" works. #right
> > X-Spam-Scan-Score: 7.3 -> "ge" works. #right
> > 
> 
> Note that the value '0' has a special meaning for the spamtest test. The 
> 'definitely not spam' value is '1'. Therefore, values ranging from 0.0 
> to 10.0 are calculated into a score as follows:
> 
> score = (header_value / 10.0) * 9 + 1
> 
> So, the 5.9 match is correct as it yields 6.3, matching against "6".
> 
> The 3.8 match, however, is not correct. Please run the attached 
> testsuite script I composed from your e-mail as follows:
> 

Today I rerun all test with values from 0.0 to 10.9, everything is working fine 
(starting from value 5.6 the rule begun to work). I can't reproduce yesterday's 
triggering, either with "6" or "6.0". Seemed I'm done something wrong.

Sorry for the noise.

> src/testsuite/testsuite nikita-spamtest.svtest
> 
> At my end, this test case succeeds fully, including the 3.8 match. The 
> test suite script should be fairly easy to understand and modify. If you 
> manage to trigger a bug somehow, please let me now.
>

Test case: /root/ive/sieve-spamteset.svtest:

 1: Test 'Value: 3.6' SUCCEEDED
 2: Test 'Value: 3.8' SUCCEEDED
 3: Test 'Value: 5.9' SUCCEEDED
 4: Test 'Value: 6.0' SUCCEEDED
 5: Test 'Value: 6.9' SUCCEEDED
 6: Test 'Value: 7.3' SUCCEEDED

PASS: 6 tests succeeded.

> > If the rule modified to (spamtest :value "ge" :comparator "i;ascii-numeric" 
> > "6.0") everything works as expected.
> > 
> 
> Matching against "6.0" will end at the first non-digit character, which 
> should normally be equal to matching against "6". Very strange that this 
> somehow helps. What version are you using?
> 
> > One more thing - sieve-test. I couldn't test my scripts with it:
> > 
> >  # sieve-test -x +spamtest -t sieve spam.txt 
> > 0007: SPAMTEST test
> > 0007: spamtest: extension not configured
> > 0013: JMPFALSE (false)
> > 
> 
> You need to set the configuration by setting the appropriate config 
> values in the environment (e.g. SIEVE_SPAMTEST_MAX_VALUE) while calling 
> sieve-test. (For Dovecot v2.0 the tools also still don't use the Dovecot 
> configuration, which needs to be fixed before the first release).

Thanks for clarifications. 

[Dovecot] sieve spamtest extension

[Nikita Koshikov]

Hello Stephan,

First of all, thanks for time you spent implementing this useful extension.

I have found some ugly behavior with such config:

  sieve_extensions = +notify +spamtest
  sieve_spamtest_status_type = score
  sieve_spamtest_status_header = X-Spam-Scan-Score: 
(-?[[:digit:]]+\.[[:digit:]])
  sieve_spamtest_max_value = 10

Sieve:
require 
["fileinto","envelope","imap4flags","spamtest","relational","comparator-i;ascii-numeric"];
# rule:[spam]
if anyof (spamtest :value "ge" :comparator "i;ascii-numeric" "6")
{
fileinto "Spam";
}

If X-Spam-Scan-Score have 6-9 digit after dot - the first one ignored, thus
X-Spam-Scan-Score: 3.6 -> "ge" skips. #right
X-Spam-Scan-Score: 3.8 -> "ge" works. #wrong
X-Spam-Scan-Score: 5.9 -> "ge" works. #wrong
X-Spam-Scan-Score: 6.0 -> "ge" works. #right
X-Spam-Scan-Score: 6.9 -> "ge" works. #right
X-Spam-Scan-Score: 7.3 -> "ge" works. #right

If the rule modified to (spamtest :value "ge" :comparator "i;ascii-numeric" 
"6.0") everything works as expected.

One more thing - sieve-test. I couldn't test my scripts with it:

 # sieve-test -x +spamtest -t sieve spam.txt 
0007: SPAMTEST test
0007: spamtest: extension not configured
0013: JMPFALSE (false)

Performed actions:

  (none)

Implicit keep:

 * store message in folder: INBOX

Info: final result: success

Re: [Dovecot] doveadm fetch

[Nikita Koshikov]

On Tue, 20 Apr 2010 18:49:10 +0300
Timo Sirainen wrote:

> 
> The above syntax would probably have to use IMAP parser.
> 
> > doveadm fetch subject "(foo)"
> 
> This works nowadays though. It's possible to use:
> 
> doveadm fetch INBOX \( subject "(foo)" seen \) or unseen
> 
> This works because there are now "IMAP parser" and "command line
> parser". The command line parser knows that after subject there must
> come a string, so it's not confused by the () characters. Then it also
> knows that when "(" or ")" comes in a separate parameter, it means a
> list. Actually it would have been possible to support \(subject without
> space after \(, but this won't work with the ending \):
> 
> \(subject "(foo)"\)
> 
> vs.
> 
> \(subject "(foo))"
> 
> would look identical to doveadm. So I thought it's better to always
> require the space.
> 
> > but that makes it more difficult to add variables, because if you do:
> > 
> > doveadm fetch 'subject "$foo"'
> 
> $variables also work nicely now without having to escape them.
> 
> Now, the next problem is how to select what to fetch and what the output
> format should look like. I'm thinking about:
> 
> doveadm fetch INBOX "flags uid hdr.received hdr.from body" all
> 
> would look like:
> 
> ===sep
> flags: \seen \draft $Label1
> uid: 1234
> hdr:
> Received: stuff
> Received: more stuff
> From: t...@iki.fi
> 
> body:
> message body
> ..
> ===sep
> flags: ..next message..
> ===sep
> 
> The ===sep is a randomly generated separator string that begins always
> with "===", optionally it could be given as parameter. I was wondering
> about how to return hdr.* fields, if they should be returned separately
> or all in one "hdr". Otherwise separate fields would be nice, but if the
> header exists multiple times, it's not so clear anymore how it should be
> written. So if there's a single hdr then it's at least easy to
> understand that it ends with an empty line.
> 
> Besides the example parameters above there could be "hdr" = full header
> and "text" = alias for "hdr body".
> 
> It would be nice also to support something like:
> 
> doveadm search INBOX from t...@iki.fi
> 
> doveadm next|less
> doveadm next|less
> ..etc..
> 
> So the "next" would return the next matching message based on the
> previous search. I'm not really sure where the state could be kept
> though. Would be nice if it was terminal-specific, and would be nice if
> it didn't write any temporary files.

Maybe you should take exim's queue style for such operations. Here few examples:

#exim -bp //return queue in format:

 4d   99K 1O36Eu-0001Py-Ts 
  is_...@domain.ltd

#doveadm search INBOX from t...@iki.fi
<...id-1>
<...id-2>

<...id-n>
Will return just unique ids for current mailbox. Maybe also few headers/flags 
etc.

#doveadm fetch <...id-2> all
Print full message for id, gotten from doveadm searc.

#doveadm fetch <...id-2> hrd.from hdr.subject
Print corresponding header values.

Exim also have -Mvl option - which print log lines for current id. It will be 
nice if:

#doveadm log  
Found all service records for message and print them to terminal.

With above scheme you don't need to construct separator -  will be 
enough and doveadm next wont be necessary. 

Re: [Dovecot] dovecot-2 new notify plugin

[Nikita Koshikov]

> Its only purpose is to help other plugins to get notifications. Unless you're 
> developing a new plugin, you don't really care about it.
> 

Thanks for info.

[Dovecot] dovecot-2 new notify plugin

[Nikita Koshikov]

Hello list, 

Can someone point me to documentation, if such available, about new dovecot 
notify plugin or describe here what it for and how to use it.

[Dovecot] dovecot 2.0_b3 autocreate plugin bug.

[Nikita Koshikov]

New dovecot-2_beta3 installation.

dovecot plugins section:
plugin {
  autocreate = Drafts
  autocreate2 = Sent
  autocreate3 = Spam
  autocreate4 = Trash
  autosubscribe = Drafts
  autosubscribe2 = Sent
  autosubscribe3 = Spam
  autosubscribe4 = Trash
  expire = Trash 30 Spam 30
  expire_dict = proxy::expire
  quota = maildir:Mailbox quota
  quota_rule = *:storage=500M
  quota_rule2 = Trash:storage=10%%
  quota_warning = storage=90%% /etc/dovecot/plugins/quota_warning.sh 90
  trash = /etc/dovecot/plugins/dovecot-trash.conf
}

Trying to login, I got core dump and no mailboxes is created. If autocreate 
plugin is commented out - than login is success and default maildirs 
mailboxes(cur/new/tmp) become avaliable.

Logs:
==> /var/log/dovecot/dovecot-debug.log <==
Feb 23 16:38:36 imap: Debug: Loading modules from directory: /usr/lib/dovecot
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib10_quota_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib11_imap_quota_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib11_trash_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib20_autocreate_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib20_expire_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib20_virtual_plugin.so
Feb 23 16:38:36 imap: Debug: Module loaded: 
/usr/lib/dovecot/lib20_zlib_plugin.so
Feb 23 16:38:36 imap: Debug: Loading modules from directory: /usr/lib/dovecot
Feb 23 16:38:36 imap(t...@domain.com): Debug: Effective uid=8, gid=12, 
home=/data/mail/domain.com/test
Feb 23 16:38:36 imap(t...@domain.com): Debug: Quota root: name=Mailbox quota 
backend=maildir args=
Feb 23 16:38:36 imap(t...@domain.com): Debug: Quota rule: root=Mailbox quota 
mailbox=* bytes=524288000 messages=0
Feb 23 16:38:36 imap(t...@domain.com): Debug: Quota rule: root=Mailbox quota 
mailbox=Trash bytes=52428800 (10%) messages=0
Feb 23 16:38:36 imap(t...@domain.com): Debug: Quota warning: bytes=471859200 
(90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace: type=private, prefix=, 
sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes
Feb 23 16:38:36 imap(t...@domain.com): Debug: maildir++: 
root=/data/mail/domain.com/test/data, index=, control=, 
inbox=/data/mail/domain.com/test/data
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace: type=private, 
prefix=Company/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no
Feb 23 16:38:36 imap(t...@domain.com): Debug: maildir++: 
root=/var/mail/virtual, index=, control=, inbox=
Feb 23 16:38:36 imap(t...@domain.com): Debug: trash plugin: Added 'Trash' with 
priority 1
Feb 23 16:38:36 imap(t...@domain.com): Debug: trash plugin: Added 'Spam' with 
priority 2
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace : Permission lookup 
failed from /data/mail/domain.com/test/data/.Drafts
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace : Using permissions 
from /data/mail/domain.com/test/data: mode=0700 gid=-1
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace : Permission lookup 
failed from /data/mail/domain.com/test/data/.Drafts
Feb 23 16:38:36 imap(t...@domain.com): Debug: Namespace : Using permissions 
from /data/mail/domain.com/test/data: mode=0700 gid=-1

==> /var/log/dovecot/dovecot-error.log <==
Feb 23 16:38:36 master: Error: service(imap): child 21505 killed with signal 11 
(core dumped)

==> /var/log/dovecot/dovecot.log <==
Feb 23 16:38:36 imap-login: Info: Login: user=, method=PLAIN, 
rip=127.0.0.1, lip=127.0.0.1, TLS



warning: Can not parse XML syscalls information; XML support was disabled at 
compile time.
GNU gdb (Gentoo 7.0 p1) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/libexec/dovecot/imap...done.

warning: Can't read pathname for load map: Ошибка ввода/вывода.
Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done.
Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done.
Loaded symbols for /usr/lib/dovecot/libdovecot.so.0
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/lib/libldap-2.4.so.2...(no debugging symbols 
found)...done.
Loaded symbols for /usr/lib/libldap-2.4.so.2
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/li

Re: [Dovecot] Client behaviour with sieve

[Nikita Koshikov]

On Fri, 19 Feb 2010 15:17:17 +0100
Koenraad Lelong  wrote:

> Hi,
> 
> I have a working dovecot imap-server, with sieve.
> I find it odd that my mail-clients (Thunderbird 2 and 3) don't report 
> anything that's new in the folders.
> What I mean is this :
> Postfix get's a mail and hands it over to dovecot's LDA and sieve moves 
> it to a folder.
> When I log in with Thunderbird, I see new messages in my Inbox. But that 
> message that was moved to a folder is "invisible" until I click on the 
> folder. Then Thunderbird sees there are new messages in that folder and 
> reports the number of new messages and makes the foldername bold.
> 
> Is this the expected behaviour ? Or did I configure something wrong ?
> Thanks for any clarification.
> 
> Regards,
> 
> Koenraad Lelong.

Take a look http://www.mozilla.org/support/thunderbird/tips#beh_downloadstartup

Re: [Dovecot] pop3 quick question

[Nikita Koshikov]

On Sat, 06 Feb 2010 17:15:09 +0200
Timo Sirainen  wrote:

> There's no such option in Dovecot. If you mean from
> http://wiki.dovecot.org/Authentication/RestrictAccess :
> 
> password_query = SELECT password FROM users WHERE userid = '%u' and
> (imap_allowed = true or '%Ls' = 'pop3') 
> 
> You can do the same for both IMAP and POP3 with:
> 
> password_query = SELECT password FROM users \
>   WHERE userid = '%u' and \
>   (imap_allowed = true or '%Ls' != 'imap') and \
>   (pop3_allowed = true or '%Ls' != 'pop3') 
> 

Thanks, I'll look at postlogin script assistance for my issues. 

[Dovecot] pop3 quick question

[Nikita Koshikov]

Hello list, 

I have found imap_allowed option in dovecot which do exactly what I need, for 
imap. Is there something similar for pop3 protocol?
The main goal for it - is to enable pop3 access to specific users. 

Or maybe dovecot have some 'pop3=yes' or 'imap=no' args to userdb/passdb 
sections ?

I know future dovecot 2.* versions will support rewriting for protocols, but 
this is required for 1.2 branch.

Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?

[Nikita Koshikov]

On Thu, 28 Jan 2010 13:07:32 +0500
CoolAtt NNA  wrote:

> 
> Hi All..
> 
> Am working on continous backup of mailboxes using rsync(for e.g by running 
> rsync every 2 min)
> 
> Things gets more complicated when users create Subfolders in INBOX , SENT , 
> etc..
> 
> If anyone among you did that plz guide & advise.
> 
> Thanks
> CoolAtt
> 

Take a look on ccollect project http://unix.schottelius.org/ccollect/. It made 
incremental backups using rsync. That will be working for maildirs perfectly.
> 
> _
> Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
> https://signup.live.com/signup.aspx?id=60969

Re: [Dovecot] [exim] Exim + Dovecot deliver overquota problems

[Nikita Koshikov]

On Mon, 21 Dec 2009 13:44:42 +0100
Joan  wrote:

> >
> > You need to check user's quota on routing stage, not transport.
> >
> It'd be much better, because I would allow me to mix overquota and
> no-overquota recipients.
> But the problem in my case is that I am using some hashing to
> distribute the users' mailboxes, dovecot can handle this easily
> because it supports some interesting stuff
> (http://wiki.dovecot.org/Variables)

Exim have much more variables and functions 
http://exim.org/exim-html-4.69/doc/html/spec_html/ch11.html#SECTexpvar, so if 
you don't use any special dovecot variable data, than it shouldn't be a problem.

> With a perl script I could also do that, I would like only if it's the
> last resort (too much overhead I would think)
> >
Parsing file and counting quota value - is trivial stuff. The main disadvantage 
of this way is executing external aplication on every RCPT TO command. If 
you'll decide to use perl consider using option perl_at_start to minimize 
aplication starting time.

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Mon, 23 Nov 2009 12:01:53 -0500
Timo Sirainen  wrote:

> On Mon, 2009-11-23 at 18:01 +0200, Nikita Koshikov wrote:
> > On Fri, 06 Nov 2009 18:42:49 -0500
> > Timo Sirainen  wrote:
> > 
> > > On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
> > > > I don't understand why gdb says "no debugging symbols found" for some
> > > > dovecot's executables, all binaries under /usr/libexec/dovecot have
> > > > "not stripped"
> > > ..
> > > > (gdb) bt full
> > > > #0  0xb7f88749 in ?? () from /lib/libc.so.6
> > > > No symbol table info available.
> > > > Cannot access memory at address 0xbf690ff8
> > > 
> > > Yeah, something in your system has broken gdb.. Makes the debugging
> > > annoyingly difficult.
> 
> You managed to get backtraces from the other bug. What did you do
> differently? Maybe you can get backtrace from this bug now too?

I think gdb can't access memory due to gentoo kernel patches, so attaching to 
process is not working. Also, I can't change kernel on production server for 
testing purposes.
As I wrote above, core files not present in users's homedir, even with patch 
you provided.

> 
> > I have wrote simple wrapper that removes maildirsize  if the messages
> > were expunged from user's mailbox and if user has non-default quota
> > value. Is there any more convenient  way to force dovecot recalculate
> > quota size ? I have overview IMAP-QOUTA rfc, but seemed that it lacks
> > such imap command, maybe some internal dovecot command exist?
> 
> No, there's no other way.

Clear, thanks.

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Mon, 23 Nov 2009 11:46:35 -0500
Timo Sirainen  wrote:

> On Mon, 2009-11-23 at 09:59 +0200, Nikita Koshikov wrote:
> > With above patch applied, I got this one crash:
> > 
> > Nov 23 09:42:59 IMAP(al...@domain.com): Panic: file mail-index-sync-ext.c: 
> > line 625 (mail_index_sync_ext_hdr_update): assertion failed: 
> > (ext->hdr_offset + offset + size <= map->hdr.header_size)
> 
> Fixed: http://hg.dovecot.org/dovecot-1.2/rev/40a6a70b3146

Finally it works!!! Thanks.

> 
> Also that crash doesn't get fixed by itself, so either delete the
> virtual mailbox's dovecot.index* files or/and apply this patch:
> 
> http://hg.dovecot.org/dovecot-1.2/rev/9a8eb4d68b04
> 

My indexes are located in the memory (:INDEX=MEMORY:) so I don't apply this 
patch. If you need me to test it - please let me know.

> And I finally created 1000 mailboxes myself and tested that it actually
> works now.
> 

Again, Thanks for your work.

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Fri, 06 Nov 2009 18:42:49 -0500
Timo Sirainen  wrote:

> On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
> > I don't understand why gdb says "no debugging symbols found" for some
> > dovecot's executables, all binaries under /usr/libexec/dovecot have
> > "not stripped"
> ..
> > (gdb) bt full
> > #0  0xb7f88749 in ?? () from /lib/libc.so.6
> > No symbol table info available.
> > Cannot access memory at address 0xbf690ff8
> 
> Yeah, something in your system has broken gdb.. Makes the debugging
> annoyingly difficult.
> 

I have wrote simple wrapper that removes maildirsize  if the messages were 
expunged from user's mailbox and if user has non-default quota value. Is there 
any more convenient  way to force dovecot recalculate quota size ? I have 
overview IMAP-QOUTA rfc, but seemed that it lacks such imap command, maybe some 
internal dovecot command exist?

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Fri, 20 Nov 2009 13:53:18 -0500
Timo Sirainen  wrote:

> On Fri, 2009-11-20 at 10:23 +0200, Nikita Koshikov wrote:
> > Nov 20 10:16:00 IMAP(al...@domain.com): Panic: file 
> > mail-transaction-log-append.c: line 31 (log_append_buffer): assertion 
> > failed: ((type & MAIL_TRANSACTION_)
> 
> Oh, missed that one: http://hg.dovecot.org/dovecot-1.2/rev/fa8a438c64ce
> 

With above patch applied, I got this one crash:

Nov 23 09:42:59 IMAP(al...@domain.com): Panic: file mail-index-sync-ext.c: line 
625 (mail_index_sync_ext_hdr_update): assertion failed: (ext->hdr_offset + 
offset + size <= map->hdr.header_size)
Nov 23 09:42:59 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2521] 
-> imap [0x80d2592] -> imap [0x80d1ef9] -> imap [0x80c06ef] -> 
imap(mail_index_sync_record+0x210) [0x80af010] -> 
imap(mail_index_sync_map+0x2b9) [0x80afd39] -> imap(mail_index_map+0x3c4) 
[0x80a4f34] -> imap(mail_index_sync_commit+0xc2) [0x80ad552] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7deebc6] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c) 
[0xb7df16ac] -> imap(mailbox_sync+0x35) [0x80975a5] -> 
imap(cmd_select_full+0x3d8) [0x8062fe8] -> imap(cmd_select+0x19) [0x8063729] -> 
imap [0x80659ac] -> imap [0x8065a53] -> imap(client_handle_input+0x2d) 
[0x8065bad] -> imap(client_input+0x5f) [0x80664ef] -> 
imap(io_loop_handler_run+0xe0) [0x80da680] -> imap(io_loop_run+0x20) 
[0x80d9b20] -> imap(main+0x5dc) [0x806ee7c] -> 
/lib/libc.so.6(__libc_start_main+0xe5) [0xb7e3c725] -> imap [0x805e2b1]
Nov 23 09:43:00 dovecot: Error: child 19671 (imap) killed with signal 6 (core 
dumped)

Gdb trace in attachment.

GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...done.
Loaded symbols for /lib/libbz2.so.1
Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done.
Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Core was generated by `imap'.
Program terminated with signal 6, Aborted.
[New process 19671]
#0  0xe424 in __kernel_vsyscall ()
(gdb) bt full
#0  0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7e50660 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0xb7e51e98 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x080d2535 in default_fatal_finish (type=, status=0) at failures.c:160
backtrace = 0xb7d72a48 "imap [0x80d2521] -> imap [0x80d2592] -> imap [0x80d1ef9] -> imap [0x80c06ef] -> imap(mail_index_sync_record+0x210) [0x80af01.
#4  0x080d2592 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80ea1cc "file %s: line %d (%s): assertion failed: (%s)", 
args=0xbfd9ac24 "4x\017\bq\002") at failures.c:443
No locals.
#5  0x080d1ef9 in i_panic (format=0x80ea1cc "file %s: line %d (%s): assertion failed: (%s)") at failures.c:207
No locals.
#6  0x080c06ef in mail_index_sync_ext_hdr_update (ctx=0xbfd9ad2c, offset

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Thu, 19 Nov 2009 18:53:19 -0500
Timo Sirainen  wrote:

> On Thu, 2009-11-19 at 18:31 -0500, Timo Sirainen wrote:
> > On Thu, 2009-11-19 at 09:10 +0200, Nikita Koshikov wrote:
> > > Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite: 
> > > size=32824
> > > Nov 19 08:57:34 IMAP(al...@domain.com): Warning: 
> > > mail_index_update_header_ext: ext_id=2 offset=0 size=32824
> > > Nov 19 08:57:34 IMAP(al...@domain.com): Warning: 
> > > log_append_ext_hdr_update: size=65536
> > 
> > Thanks. Pretty simple fix after all that I missed:
> > http://hg.dovecot.org/dovecot-1.2/rev/3e1ca490dde0
> 
> Except .. although that above patch helps a bit, it still breaks after
> header size goes to 64k and fixing that requires changing index file
> format a bit. This should help there:
> http://hg.dovecot.org/dovecot-1.2/rev/e5d38150be58
> 

Sorry, but with version 1.2.8 the crash is still here:

Nov 20 10:16:00 IMAP(al...@domain.com): Panic: file 
mail-transaction-log-append.c: line 31 (log_append_buffer): assertion failed: 
((type & MAIL_TRANSACTION_)
Nov 20 10:16:00 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2411] 
-> imap [0x80d2482] -> imap [0x80d1de9] -> imap [0x80b4a9f] -> imap(mail_tran]
Nov 20 10:16:00 dovecot: Error: child 29596 (imap) killed with signal 6 (core 
dumped)

I have tried delete old indexes, but this didn't help.

current gdb-trace is attached.GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...done.
Loaded symbols for /lib/libbz2.so.1
Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done.
Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Core was generated by `imap'.
Program terminated with signal 6, Aborted.
[New process 29596]
#0  0xe424 in __kernel_vsyscall ()
(gdb) bt full
#0  0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7e2b660 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0xb7e2ce98 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x080d2425 in default_fatal_finish (type=, status=0) at failures.c:160
backtrace = 0xb7d5d6c0 "imap [0x80d2411] -> imap [0x80d2482] -> imap [0x80d1de9] -> imap [0x80b4a9f] -> imap(mail_transaction_log_append+0x68d) [0x80b556d] -> imap [0x80aa039] -> imap(mail_index_sync_commit+0xa5) [0x80ad535]"...
#4  0x080d2482 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80ea0cc "file %s: line %d (%s): assertion failed: (%s)", 
args=0xbff736b4 "�U\017\b\037") at failures.c:443
No locals.
#5  0x080d1de9 in i_panic (format=0x80ea0cc "file %s: line %d (%s): assertion failed: (%s)") at failures.c:207
No locals.
#6  0x080b4a9f in log_append_buffer (ctx=0xbff737a8, buf=0xb7d4d670, hdr_buf=0x0, type=29596) at mail-transaction-log-append.c:31
hdr = {size = 196419352, type = 32824}
hdr_size = 
hdr_pos = 
__PRET

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Wed, 18 Nov 2009 11:55:40 -0500
Timo Sirainen  wrote:

> On Wed, 2009-11-18 at 10:04 +0200, Nikita Koshikov wrote:
> 
> > #10 0x080d03d2 in buffer_write (_buf=0xb7e1a670, pos=67100436, 
> > data=0xbb53ae0, data_size=32824) at buffer.c:63
> > No locals.
> > #11 0x080d04f6 in buffer_append (buf=0xb7e1a670, data=0xbb53ae0, 
> > data_size=32824) at buffer.c:168
> > No locals.
> > #12 0x080b54cc in mail_transaction_log_append (t=0xb205218, 
> > log_file_seq_r=0xbf8541f0, log_file_offset_r=0xbf8541e0) at 
> > mail-transaction-log-append.c:333
> 
> Interesting. If you still have the core and do:
> 
> fr 12
> p *hdr
> p offset
> p hdrs[ext_id]
> 
With old core(not sure if it's exactly the same):

(gdb) fr 12
#12 0x080b54cc in mail_transaction_log_append (t=0xb205218, 
log_file_seq_r=0xbf8541f0, log_file_offset_r=0xbf8541e0) at 
mail-transaction-log-append.c:333
warning: Source file is more recent than executable.
333 } else {
(gdb) p *hdr
No symbol "hdr" in current context.
(gdb) p offset
No symbol "offset" in current context.
(gdb) p hdrs[ext_id]
No symbol "hdrs" in current context.

> What does it print? Also could you try what it logs with the attached patch?
> 
Here is logs, from new crash:

Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite: size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: mail_index_update_header_ext: 
ext_id=2 offset=0 size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: log_append_ext_hdr_update: 
size=65536
Nov 19 08:57:35 IMAP(al...@domain.com): Panic: data stack: Out of memory when 
allocating 268435472 bytes
Nov 19 08:57:35 IMAP(al...@agromat.ua): Error: Raw backtrace: imap [0x80d23e1] 
-> imap [0x80d2452] -> imap [0x80d1db9] -> imap [0x80d0b8d] -> imap [0x80d1007] 
-> imap [0x80db988] -> imap [0x80cff3a] -> imap(buffer_write+0xd2) [0x80d0432] 
-> imap(buffer_append+0x26) [0x80d0556] -> 
imap(mail_transaction_log_append+0x624) [0x80b5504] -> imap [0x80aa059] -> 
imap(mail_index_sync_commit+0xa5) [0x80ad555] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7e6cc06] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c) 
[0xb7e6f71c] -> imap(mailbox_sync+0x35) [0x8097595] -> 
imap(cmd_select_full+0x3d8) [0x8062fb8] -> imap(cmd_select+0x19) [0x80636f9] -> 
imap [0x806597c] -> imap [0x8065a23] -> imap(client_handle_input+0x2d) 
[0x8065b7d] -> imap(client_input+0x5f) [0x80664bf] -> 
imap(io_loop_handler_run+0xe0) [0x80da540] -> imap(io_loop_run+0x20) 
[0x80d99e0] -> imap(main+0x5dc) [0x806ee4c] -> 
/lib/libc.so.6(__libc_start_main+0xe5) [0xb7eba725] -> imap [0x805e281]
Nov 19 08:57:36 dovecot: Error: child 16124 (imap) killed with signal 6 (core 
dumped)

From new core file:
(gdb) fr 12
#12 0x080b5504 in mail_transaction_log_append (t=0xb205218, 
log_file_seq_r=0xbf9162b0, log_file_offset_r=0xbf9162a0) at 
mail-transaction-log-append.c:337
337 mail-transaction-log-append.c: No such file or directory.
in mail-transaction-log-append.c
(gdb) p *hdr
No symbol "hdr" in current context.
(gdb) p offset
No symbol "offset" in current context.
(gdb) p hdrs[ext_id]
No symbol "hdrs" in current context.

And attached current gdb-trace.
(gdb) bt full
#0  0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7ece660 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0xb7ecfe98 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x080d23f5 in default_fatal_finish (type=, status=0) at failures.c:160
backtrace = 0x92700478 "imap [0x80d23e1] -> imap [0x80d2452] -> imap [0x80d1db9] -> imap [0x80d0b8d] -> imap [0x80d1007] -> imap [0x80db988] -> imap [0x80cff3a] -> imap(buffer_write+0xd2) [0x80d0432] -> imap(buffer_append+0x"...
#4  0x080d2452 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80f9b9c "data stack: Out of memory when allocating %u bytes", 
args=0xbf915fb4 "\020") at failures.c:443
No locals.
#5  0x080d1db9 in i_panic (format=0x80f9b9c "data stack: Out of memory when allocating %u bytes") at failures.c:207
No locals.
#6  0x080d0b8d in mem_block_alloc (min_size=134217728) at data-stack.c:291
block = (struct stack_block *) 0x0
prev_size = 
alloc_size = 268435456
#7  0x080d1007 in t_malloc_real (size=, permanent=true) at data-stack.c:352
block = (struct stack_block *) 0x6
ret = 
alloc_size = 134217728
#8  0x080db988 in pool_data_stack_realloc (pool=0xbb4c808, mem=0xb28b4018, old_size=67108857, new_size=134217728) at mempool-datastack.c:127
No locals.
#9  0x080cff3a in buffer_alloc (buf=0xbb4c818, size=134217728) at buffer.c:32
__PRETTY_FUNCTION__ = "buffer_alloc"

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Wed, 18 Nov 2009 06:18:06 -0500
Charles Marcus  wrote:

> On 11/18/2009, Nikita Koshikov (koshi...@gmail.com) wrote:
> > Today, I have updated to version 1.2.7 and I was able to get
> > gdb-backtrace with it. This is clean installation without any patches
> > applied.
> 
> Probably a good idea to always provide dovecot -n output too...
> 

It has been showed in my first post. Since that time config file didn't change.

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

> I guess it's just exponentially increasing the buffer size then until it
> reaches mail_process_size.
> 
> > I recompile dovecot binaries with debug symbols, but seemed that gdb 
> > backtrace is broken.
> 
> Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to
> imap process while it's still running? So something like:
> 
>  - open the account so imap process starts
>  - gdb -p 
>  - gdb command: c
>  - cause imap to crash -> gdb should stop
>  - gdb command: bt full
> 

Today, I have updated to version 1.2.7 and I was able to get gdb-backtrace with 
it. This is clean installation without any patches applied.

Error-log:
Nov 18 09:55:37 IMAP(al...@domain.com): Panic: data stack: Out of memory when 
allocating 268435472 bytes
Nov 18 09:55:37 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2381] 
-> imap [0x80d23f2] -> imap [0x80d1d59] -> imap [0x80d0b2d] -> imap [0x80d0fa7] 
-> imap [0x80db928] -> imap [0x80cfeda] -> imap(buffer_write+0xd2) [0x80d03d2] 
-> imap(buffer_append+0x26) [0x80d04f6] -> 
imap(mail_transaction_log_append+0x61c) [0x80b54cc] -> imap [0x80aa029] -> 
imap(mail_index_sync_commit+0xa5) [0x80ad525] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7ea8bc6] -> 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c) 
[0xb7eab6ac] -> imap(mailbox_sync+0x35) [0x8097595] -> 
imap(cmd_select_full+0x3d8) [0x8062fb8] -> imap(cmd_select+0x19) [0x80636f9] -> 
imap [0x806597c] -> imap [0x8065a23] -> imap(client_handle_input+0x2d) 
[0x8065b7d] -> imap(client_input+0x5f) [0x80664bf] -> 
imap(io_loop_handler_run+0xe0) [0x80da4e0] -> imap(io_loop_run+0x20) 
[0x80d9980] -> imap(main+0x5dc) [0x806ee4c] -> 
/lib/libc.so.6(__libc_start_main+0xe5) [0xb7ef6725] -> imap [0x805e2
 81]
Nov 18 09:55:37 dovecot: Error: child 9173 (imap) killed with signal 6 (core 
dumped)

Gdb-backtrace is attached.r...@mail alice 0:0 # gdb /usr/libexec/dovecot/imap core 
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...done.
Loaded symbols for /lib/libbz2.so.1
Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done.
Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Core was generated by `imap'.
Program terminated with signal 6, Aborted.
[New process 9173]
#0  0xe424 in __kernel_vsyscall ()
(gdb) bt full
#0  0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1  0xb7f0a660 in raise () from /lib/libc.so.6
No symbol table info available.
#2  0xb7f0be98 in abort () from /lib/libc.so.6
No symbol table info available.
#3  0x080d2395 in default_fatal_finish (type=, status=0) at failures.c:160
backtrace = 0x92700478 "imap [0x80d2381] -> imap [0x80d23f2] -> imap [0x80d1d59] -> imap [0x80d0b2d] -> imap [0x80d0fa7] -> imap [0x80db928] -> imap [0x80cfeda] -> imap(buffer_write+0xd2) [0x80d03d2] -> imap(buffer_append+0x"...
#4  0x080d23f2 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80f9adc "data stack: Out of memory when allocating %u bytes", 
args=0xbf853ef4 "\020") at failures.c:443
No locals.
#5  0x080d1d59 in i_panic (format=0x80f9adc "da

Re: [Dovecot] Web-Interface for Dovecot-Sieve?

[Nikita Koshikov]

> OK, good.  I'm glad to be corrected, that is starting to look promising.
> 
> Compare http://www.horde.org/ingo/screenshots/rule.png .  For now, ingo
> still appears to be more capable-- e.g. I find it essential to be able to
> mark messages as "Seen" as I file them.  But roundcube is a good product
> under steady development, and I won't be at all surprised if they catch up
> or surpass ingo.
> 

Try to use sieverules plugin from http://www.tehinterweb.co.uk/roundcube/.
(http://www.tehinterweb.co.uk/roundcube/plugins/sieverules.jpg) it has more 
features implemented than managesieve.

> I've been using ingo and am happy with it, but to use it you have to
> install and configure the whole horde framework-- a good amount of work,
> and the only part of it I want is ingo.
> 

Re: [Dovecot] Virtual mailboxes not always up to date?

[Nikita Koshikov]

On Mon, 09 Nov 2009 08:03:59 +0800
Patrick Nagel  wrote:

> Hi,
> 
> I have been experimenting with the Virtual mailboxes plugin [1] recently.
> 
> This is my setup: the following dovecot-virtual file exists in a virtual
> mailbox called "todo":
> 
> -- dovecot-virtual --
> *
> -Trash
> -Trash/*
> -Spam
> -Spam/*
>   OR KEYWORD $TODO KEYWORD todo
> -
> 
> so if any mail in any mailbox except Trash+Spam has the keyword "$TODO"
> (that's what KMail sets when you mark a message as "action item") or "todo"
> (that's what Thunderbird sets when you put the Todo tag on a message), it
> should be shown in that 'todo' virtual mailbox.
> 
> This works for the first time after I edit the dovecot-virtual file and
> then access the virtual mailbox (all mails I expect to be shown in the todo
> mailbox are there) - but further accesses always show the same content,
> even though I removed the keyword from one of the mails (and I can confirm
> that the keyword has actually been removed by dovecot, since the filename
> doesn't contain the letter corresponding to the keyword anymore).
> 
> It seems that changing the dovecot-virtual file triggers the execution of
> the "search program" - but shouldn't simply accessing the virtual mailbox
> also trigger it? Or is there a (undocumented?) timeout between search
> program executions?

> namespace:
>   type: private
>   separator: .
>   prefix: virtual.
>   location: virtual:~/.maildir/virtual
>   list: yes
>   subscriptions: yes

Try to add :INDEX=MEMORY to location setting.

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Thu, 29 Oct 2009 12:38:22 -0400
Timo Sirainen  wrote:

> On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote:
> > On Wed, 28 Oct 2009 14:17:52 -0400
> > Timo Sirainen  wrote:
> > 
> > > On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote:
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out 
> > > > > > of memory when allocating 268435472 bytes
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc 
> > > > > > detected *** imap: double free or corruption (!prev): 0x0812ba00 ***
> > > 
> > > Oh, the double free is because of broken handling of out-of-memory
> > > error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes
> > > that.
> > 
> > Sorry, but this patch didn't help.
> 
> Not with the "out of memory", but did it get rid of the "double free or
> corruption" error?
> 
> > > Hmm. So I guess there's no memory corruption causing this, but I don't
> > > really see why it would try to allocate that much memory. 268435472 in
> > > hex is 0x1010, which is an interesting number but doesn't really
> > > help much either.
> > 268435456 is 256M which is max mail_process_size config setting, maybe this 
> > values is in use ? 
> 
> I guess it's just exponentially increasing the buffer size then until it
> reaches mail_process_size.
> 
> > I recompile dovecot binaries with debug symbols, but seemed that gdb 
> > backtrace is broken.
> 
> Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to
> imap process while it's still running? So something like:
> 
>  - open the account so imap process starts
>  - gdb -p 
>  - gdb command: c
>  - cause imap to crash -> gdb should stop
>  - gdb command: bt full
> 

I found I way, to switch crash under telnet session and attach gdb to process, 
but this didn't produce valid trace:

GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
Attaching to process 29952
Reading symbols from /usr/libexec/dovecot/imap...(no debugging symbols 
found)...done.
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...
(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0xb7dccac0 (LWP 29952)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...(no debugging 
symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libbz2.so.1

(no debugging symbols found)
0xe424 in __kernel_vsyscall ()
(gdb) cont
Continuing.
(no debugging symbols found)

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb7dccac0 (LWP 29952)]
0xe424 in __kernel_vsyscall ()
(gdb) quit
The program is running.  Quit anyway (and detach it)? (y or n) yes
Detaching from program: /usr/libexec/dovecot/imap, process 29952

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Thu, 05 Nov 2009 12:52:08 -0500
Timo Sirainen  wrote:

> On Sat, 2009-10-31 at 10:42 +0200, Nikita Koshikov wrote:
> > The patch helped with quotas, but it raises new error - "segmentation error"
> 
> gdb backtrace would be helpful. But how to actually get it could be a
> bit difficult, since I guess kernel thinks expire-tool is a setuid
> program.
> 
> There are two ways I guess:
> 
> a) Apply attached diff1 and see if core dump gets written. Once you get
> a core, use "gdb expire-tool core", "bt full".
This didn't create core file.

> 
> b) If not, apply diff2 and start expire-tool. Then look up its pid and
> run "gdb -p ", "cont", wait for crash, "bt full".

I don't understand why gdb says "no debugging symbols found" for some dovecot's 
executables, all binaries under /usr/libexec/dovecot have "not stripped"

GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu".
Attaching to process 26407
Reading symbols from /usr/libexec/dovecot/expire-tool...(no debugging symbols 
found)...done.
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...
(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0xb7f02ac0 (LWP 26407)]
Loaded symbols for /lib/libpthread.so.0
(no debugging symbols found)
0xe424 in __kernel_vsyscall ()
(gdb) cont
Continuing.
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7f02ac0 (LWP 26407)]
0xb7f88749 in ?? () from /lib/libc.so.6
(gdb) bt full
#0  0xb7f88749 in ?? () from /lib/libc.so.6
No symbol table info available.
Cannot access memory at address 0xbf690ff8
(gdb) quit
The program is running.  Quit anyway (and detach it)? (y or n) y
Detaching from program: /usr/libexec/dovecot/expire-tool, process 26407

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

>
> The patch helped with quotas, but it raises new error - "segmentation
> error"
>
>
After some more tests, I find out that crash happens, while trying to
proceed  the third user, and the user's quota and other settings not
important.

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Fri, 30 Oct 2009 13:16:47 -0400
Timo Sirainen  wrote:

> On Fri, 2009-10-30 at 14:03 +0200, Nikita Koshikov wrote:
> > > > The  problem that some users have their own quotas stored in ldap. 
> 
> See if the attached patch helps? I'm a bit afraid that it could also
> break something. If it does, there's really no easy way to get this
> fixed before v2.0.
> 

Note:
I rechecked expire-tool behavior with version 1.2.4 and it's working the same 
way as 1.2.6, 
so my problem have more long history, than I thought.

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Thu, 29 Oct 2009 12:38:22 -0400
Timo Sirainen  wrote:

> On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote:
> > On Wed, 28 Oct 2009 14:17:52 -0400
> > Timo Sirainen  wrote:
> > 
> > > On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote:
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out 
> > > > > > of memory when allocating 268435472 bytes
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc 
> > > > > > detected *** imap: double free or corruption (!prev): 0x0812ba00 ***
> > > 
> > > Oh, the double free is because of broken handling of out-of-memory
> > > error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes
> > > that.
> > 
> > Sorry, but this patch didn't help.
> 
> Not with the "out of memory", but did it get rid of the "double free or
> corruption" error?
> 
> > > Hmm. So I guess there's no memory corruption causing this, but I don't
> > > really see why it would try to allocate that much memory. 268435472 in
> > > hex is 0x1010, which is an interesting number but doesn't really
> > > help much either.
> > 268435456 is 256M which is max mail_process_size config setting, maybe this 
> > values is in use ? 
> 
> I guess it's just exponentially increasing the buffer size then until it
> reaches mail_process_size.
> 
> > I recompile dovecot binaries with debug symbols, but seemed that gdb 
> > backtrace is broken.
> 
> Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to
> imap process while it's still running? So something like:
> 
>  - open the account so imap process starts
>  - gdb -p 
>  - gdb command: c
>  - cause imap to crash -> gdb should stop
>  - gdb command: bt full
> 

One more note:
version 1.2.4 also has this bug. 

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Fri, 30 Oct 2009 13:16:47 -0400
Timo Sirainen  wrote:

> On Fri, 2009-10-30 at 14:03 +0200, Nikita Koshikov wrote:
> > > > The  problem that some users have their own quotas stored in ldap. 
> 
> See if the attached patch helps? I'm a bit afraid that it could also
> break something. If it does, there's really no easy way to get this
> fixed before v2.0.
> 

The patch helped with quotas, but it raises new error - "segmentation error"

Here details:
# /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh 
Info: Loading modules from directory: /usr/lib/dovecot/imap
Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so

-->koshiko...@domain.com - have own quota in ldap, expire-tool is working fine:

Info: expire: pattern=Trash type=expunge secs=2592000
Info: expire: pattern=Spam type=expunge secs=2592000
Info: auth input: quota_rule=*:bytes=629145600
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/koshikov.n
Info: auth input: mail=maildir:~/data
Info: Quota root: name=Mailbox quota backend=maildir args=
Info: Quota rule: root=Mailbox quota mailbox=* bytes=629145600 messages=0
Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=62914560 (10%) 
messages=0
Info: Quota warning: bytes=566231040 (90%) messages=0 
command=/etc/dovecot/plugins/quota_warning.sh 90
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, 
inbox=/data/mail/domain.com/koshikov.n/data
Info: Namespace: type=private, prefix=Company/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2

--> bunkin...@domain.com have default quota value, expire-tool is working fine:

Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/bunkina.l
Info: auth input: mail=maildir:~/data
Info: Quota root: name=Mailbox quota backend=maildir args=
Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0
Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) 
messages=0
Info: Quota warning: bytes=471859200 (90%) messages=0 
command=/etc/dovecot/plugins/quota_warning.sh 90
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/bunkina.l/data, index=, control=, 
inbox=/data/mail/domain.com/bunkina.l/data
Info: Namespace: type=private, prefix=Company/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/bunkina.l/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2

--> podburt...@domain.com - have default quota, this account cause segmentation 
error, while running expire-tool:

Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/podburtniy
Info: auth input: mail=maildir:~/data
Info: Quota root: name=Mailbox quota backend=maildir args=
Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0
Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) 
messages=0
Info: Quota warning: bytes=471859200 (90%) messages=0 
command=/etc/dovecot/plugins/quota_warning.sh 90
Ошибка сегментирования

The only difference with mailbox podburt...@domain.com that it's ldap mail 
field is different from samaccountname:

podburt...@domain.com
mail: podburt...@domain.com
sAMAccountName: podburtn

koshiko...@domain.com
mail: koshiko...@domain.com
sAMAccountName: koshikov.n

But, I'm using only mail/maxstorage/useraccountcontrol attributes for mail 
staff.
Users login to their accounts using full e-mail address and AD password.

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Thu, 29 Oct 2009 13:10:37 -0400
Timo Sirainen  wrote:

> On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:
> > The  problem that some users have their own quotas stored in ldap. 
> 
> So the quota_rule should be returned by userdb lookup from ldap, right?
> 
One more clean example.
For user koshiko...@domain.com I change quota in AD to 600M(default user quota 
is 512M,). After successful login to imap account, maildirsize is(first 2 
lines):

629145600S
260658345 5484

So, quota value read from ldap database and applied to user's mailbox.
Then I run /usr/sbin/dovecot --exec-mail ext 
/usr/libexec/dovecot/expire-tool.sh --test:

Info: Loading modules from directory: /usr/lib/dovecot/imap
Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Info: Quota root: name=Mailbox quota backend=maildir args=
Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0
Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) 
messages=0
Info: Quota warning: bytes=471859200 (90%) messages=0 
command=/etc/dovecot/plugins/quota_warning.sh 90
Info: expire: pattern=Trash type=expunge secs=2592000
Info: expire: pattern=Spam type=expunge secs=2592000
Info: auth input: quota_rule=*:bytes=629145600
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/koshikov.n
Info: auth input: mail=maildir:~/data
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, 
inbox=/data/mail/domain.com/koshikov.n/data
Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2
Info: koshiko...@domain.com/Spam: seq=1 uid=1933: Expunge
Info: koshiko...@domain.com/Spam: timestamp 1256902423 (Fri Oct 30 13:33:43 
2009) -> 1256903611 (Fri Oct 30 13:53:31 2009)

Quota_rule are different from auth section and quota root\rule section. After 
expire-tool expunges e-mails, the maildirsize file become:

524288000S
260686038 5487

Which is default quota_rule value, not from ldap database.

Re: [Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

On Thu, 29 Oct 2009 13:10:37 -0400
Timo Sirainen  wrote:

> On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:
> > The  problem that some users have their own quotas stored in ldap. 
> 
> So the quota_rule should be returned by userdb lookup from ldap, right?

Yes, and it's working fine with deliver and general imap process, it only 
failed(reset to general quota_rule) after expire-rool.sh script finished his 
duties.

> 
> > Info: auth input: uid=8
> > Info: auth input: gid=12
> > Info: auth input: home=/data/mail/domain.com/koshikov.n
> > Info: auth input: mail=maildir:~/data
> 
> This should have listed quota_rule.
> 
The problematic user is sr, and it has quota_rule option(running with --test)

(Thu Oct 29 07:45:18 2009)
Info: auth input: quota_rule=*:bytes=1048576
^ returned from ldap
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/sr
Info: auth input: mail=maildir:~/data
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/sr/data, index=, control=, 
inbox=/data/mail/domain.com/sr/data
Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/sr/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2
Info: s...@domain.com/Spam: seq=1 uid=405: Expunge
Info: s...@domain.com/Spam: seq=2 uid=406: Expunge
Info: s...@domain.com/Spam: seq=3 uid=407: Expunge
Info: s...@domain.com/Spam: timestamp 1256710764 (Wed Oct 28 08:19:24 2009) -> 
125698 (Sat Oct 31 11:06:40 2009)

> > If dovecot -n output is needed, please let me know.
> 
> If the above doesn't help, then yeah, dovecot -n and dovecot-ldap.conf
> contents.
> 
dovecot -n:
# 1.2.6: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.11.1 
log_path: /var/log/dovecot/dovecot-error.log
info_log_path: /var/log/dovecot/dovecot.log
protocols: imaps managesieve
ssl_cert_file: /etc/ssl/dovecot/imaps.crt
ssl_key_file: /etc/ssl/dovecot/imaps.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting: Server ready.
login_processes_count: 20
login_max_processes_count: 512
mail_max_userip_connections(default): 20
mail_max_userip_connections(imap): 20
mail_max_userip_connections(managesieve): 10
first_valid_uid: 8
last_valid_uid: 8
first_valid_gid: 12
last_valid_gid: 12
mail_drop_priv_before_exec: yes
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): quota imap_quota trash expire zlib autocreate virtual
mail_plugins(imap): quota imap_quota trash expire zlib autocreate virtual
mail_plugins(managesieve): 
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve
imap_client_workarounds(default): delay-newmail
imap_client_workarounds(imap): delay-newmail
imap_client_workarounds(managesieve): 
namespace:
  type: private
  separator: /
  location: maildir:~/data
  inbox: yes
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: Company/
  location: virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
  list: yes
lda:
  postmaster_address: postmas...@domain.com
  hostname: mail.domain.com
  mail_plugins: quota trash expire sieve virtual
  quota_full_tempfail: yes
  sendmail_path: /usr/sbin/sendmail
  auth_socket_path: /var/run/dovecot/auth-master
  log_path: /var/log/dovecot/dovecot-deliver.log
  info_log_path: /var/log/dovecot/dovecot-deliver.log
  global_script_path: /etc/dovecot/sieve/default.sieve
  sieve_global_dir: /etc/dovecot/sieve
auth default:
  mechanisms: plain login
  default_realm: domain.com
  cache_size: 10240
  cache_negative_ttl: 0
  user: dovecot_auth
  master_user_separator: *
  worker_max_count: 50
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/master.pwd
master: yes
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  passdb:
driver: ldap
args: /etc/dovecot/dovecot-ldap.conf
  userdb:
driver: prefetch
  userdb:
driver: ldap
args: /etc/dovecot/dovecot-userdb-ldap.conf
  userdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  socket:
type: listen
clien

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Thu, 29 Oct 2009 12:38:22 -0400
Timo Sirainen  wrote:

> On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote:
> > On Wed, 28 Oct 2009 14:17:52 -0400
> > Timo Sirainen  wrote:
> > 
> > > On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote:
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out 
> > > > > > of memory when allocating 268435472 bytes
> > > > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc 
> > > > > > detected *** imap: double free or corruption (!prev): 0x0812ba00 ***
> > > 
> > > Oh, the double free is because of broken handling of out-of-memory
> > > error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes
> > > that.
> > 
> > Sorry, but this patch didn't help.
> 
> Not with the "out of memory", but did it get rid of the "double free or
> corruption" error?

The valgrind error is the same. I'll attach current valgrid output.
Also, I have noticed, that valgrind writes vgcore.* files in user's homedir, I 
attach gdb backtrace from that file. 

> 
> > > Hmm. So I guess there's no memory corruption causing this, but I don't
> > > really see why it would try to allocate that much memory. 268435472 in
> > > hex is 0x1010, which is an interesting number but doesn't really
> > > help much either.
> > 268435456 is 256M which is max mail_process_size config setting, maybe this 
> > values is in use ? 
> 
> I guess it's just exponentially increasing the buffer size then until it
> reaches mail_process_size.
> 
> > I recompile dovecot binaries with debug symbols, but seemed that gdb 
> > backtrace is broken.
> 
> Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to
> imap process while it's still running? So something like:
> 
>  - open the account so imap process starts
>  - gdb -p 
>  - gdb command: c
>  - cause imap to crash -> gdb should stop
>  - gdb command: bt full
> 
The problem, that webmail don't keep permanent connection, so when I login - 
there are about 2-3 imap-login\disconnected log entries. And when I try to get 
directory listing the new process created for this operation and then crushes.
Can gdb work in non-interactive mode ? so I can write wrapper to handle debug 
output ?

Also I enable imap_debug feature on webmail client, as you say - it stop 
working after client trying to access "All" virtual folder. The appropriate 
imap-session log part is also attached.

Note,
I can't reproduce panic with Thunderbird, this happens in roundcube only.

All virtual folder config is:

cat /var/mail/virtual/.all/dovecot-virtual
#All messages in all folders
*
  all


vcore-gdb.trace
Description: Binary data


dovecot-error.valgrind
Description: Binary data
=cut==
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "1"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Drafts"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasChildren) "/" "Dealers"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Dealers/Kapriz"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Dealers/1 Inbox"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Dealers/2 Outbox"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Spam"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Trash"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "INBOX"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\Noselect \HasChildren) "/" "Company"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Company/all"
[30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) "/" "Company/favorites"
[30-Oct-2009 10:30:11 +0200]: S: lmb OK List completed.
[30-Oct-2009 10:30:12 +0200]: C: lsb LSUB "" "*"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Sent"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Drafts"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Spam"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "1"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "INBOX"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Business correspondence/Inbox"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Business correspondence/Outbox"
[30-Oct-2009 10:30:12 +0200]: S: * LSUB () "/" "Trash"
[30-Oct-2009 10:30:12 +0200]: S: 

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Wed, 28 Oct 2009 14:17:52 -0400
Timo Sirainen  wrote:

> On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote:
> > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of 
> > > > memory when allocating 268435472 bytes
> > > > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected 
> > > > *** imap: double free or corruption (!prev): 0x0812ba00 ***
> 
> Oh, the double free is because of broken handling of out-of-memory
> error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes
> that.

Sorry, but this patch didn't help.

> 
> > > It's anyway related to one of your virtual maiboxes. I suppose you have
> > > a virtual mailbox that includes messages from all real mailboxes?
> > 
> > Yes, I have two virtual mailboxes:
> > all - contains all messages
> > favorites - contains flagged letters
> > 
> > But this user didn't subscribe to any of them.
> 
> Backtrace shows that it crashes while trying to refresh a virtual
> mailbox.
> 
> Hmm. So I guess there's no memory corruption causing this, but I don't
> really see why it would try to allocate that much memory. 268435472 in
> hex is 0x1010, which is an interesting number but doesn't really
> help much either.
268435456 is 256M which is max mail_process_size config setting, maybe this 
values is in use ? 

> 
> Can you make sure debugging symbols aren't stripped from imap binary
> (i.e. "file /usr/.../dovecot/imap" says "not stripped", this is the
> default when compiling Dovecot from sources) and get gdb backtrace?
> http://dovecot.org/bugreport.html Then I could see exactly where it's
> crashing and figuring out this would be a lot easier.
I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace 
is broken.

r...@mail dovecot-1.2.6 0:0 # file /usr/libexec/dovecot/imap
/usr/libexec/dovecot/imap: ELF 32-bit LSB executable, Intel 80386, version 1 
(SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped

r...@mail dovecot-1.2.6 0:130 # gdb /usr/libexec/dovecot/imap 
/data/mail/domain.com/gozhda.a/core 
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...(no debugging 
symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libbz2.so.1
Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1

(no debugging symbo

Re: [Dovecot] sieve and fileinto encoding

[Nikita Koshikov]

On Wed, 28 Oct 2009 14:11:08 +0300
Михаил Захаренко  wrote:

> Nikita Koshikov пишет:
> > On Wed, 28 Oct 2009 11:20:08 +0300
> > Михаил Захаренко  wrote:
> >   
> >> Nikita Koshikov пишет:
> >> 
> >>> On Wed, 28 Oct 2009 00:14:50 +0300
> >>> proton-sss  wrote:
> >>>   
> >>>> Hello All!
> >>>> I`m using dovecot 1.2.6
> >>>> I have folder named in russian "САПП", and when in filter i wrote:
> >>>>  fileinto "САПП";
> >>>> all works well.
> >>>>
> >>>> Now i find 2 web applications to manage sieve filters (horde-ingo and 
> >>>> squirrelmail/avelsieve) - then creates filter like this:
> >>>> if header :contains "Subject" "САПП"
> >>>> {
> >>>> fileinto  "&BCEEEAQfBB8-";
> >>>> stop;
> >>>> }
> >>>>
> >>>> And creates another folder ".&-BCEEEAQfBB8-", but GOOD folder name 
> >>>> already exist ".&BCEEEAQfBB8-".
> >>>>
> >>>> Please help to understand why dovecots deliver make this happen.
> >>>>
> >>>> Best regards
> >>>> Michael
> >>>> 
> >>> Take a look at http://dovecot.org/list/dovecot/2009-July/041690.html
> >>>
> >>> Also, roundcube with latest sieverules plugin can handle utf8-sieve 
> >>> directories in correct way.
> >>>   
> >> As i understood only way is patch scriptgenerator to delete conversation 
> >> from utf- 8. Is i`m rigth ?
> >> 
> > Sieverules plugin have config setting for this:
> > // Sieve RFC says that we should use UTF-8 endcoding for mailbox names,
> > // but some implementations does not covert UTF-8 to modified UTF-7.
> > // set to null for default behaviour
> > $sieverules_config['folder_encoding'] = 'UTF-8';
> >
> > As I know, others implementations leak this feature, so you should hack 
> > sources.
> >   
> is it roundcube plugin?
> 
Yes

> in last version i don`t find this option in config (or source).
> Please help to find appropriate version of plugin.
>
http://www.tehinterweb.co.uk/roundcube/
 

Re: [Dovecot] sieve and fileinto encoding

[Nikita Koshikov]

On Wed, 28 Oct 2009 11:20:08 +0300
Михаил Захаренко  wrote:

> Nikita Koshikov пишет:
> > On Wed, 28 Oct 2009 00:14:50 +0300
> > proton-sss  wrote:
> >
> >   
> >> Hello All!
> >> I`m using dovecot 1.2.6
> >> I have folder named in russian "САПП", and when in filter i wrote:
> >>  fileinto "САПП";
> >> all works well.
> >>
> >> Now i find 2 web applications to manage sieve filters (horde-ingo and 
> >> squirrelmail/avelsieve) - then creates filter like this:
> >> if header :contains "Subject" "САПП"
> >> {
> >> fileinto  "&BCEEEAQfBB8-";
> >> stop;
> >> }
> >>
> >> And creates another folder ".&-BCEEEAQfBB8-", but GOOD folder name 
> >> already exist ".&BCEEEAQfBB8-".
> >>
> >> Please help to understand why dovecots deliver make this happen.
> >>
> >> Best regards
> >> Michael
> >> 
> >
> > Take a look at http://dovecot.org/list/dovecot/2009-July/041690.html
> >
> > Also, roundcube with latest sieverules plugin can handle utf8-sieve 
> > directories in correct way.
> >   
> As i understood only way is patch scriptgenerator to delete conversation 
> from utf- 8. Is i`m rigth ?
> 
Sieverules plugin have config setting for this:
// Sieve RFC says that we should use UTF-8 endcoding for mailbox names,
// but some implementations does not covert UTF-8 to modified UTF-7.
// set to null for default behaviour
$sieverules_config['folder_encoding'] = 'UTF-8';

As I know, others implementations leak this feature, so you should hack sources.

> Best Regards
> Michael
> 

Re: [Dovecot] mailbox count folders issues

[Nikita Koshikov]

On Tue, 27 Oct 2009 17:50:22 -0400
Timo Sirainen  wrote:

> On Tue, 2009-10-27 at 09:48 +0200, Nikita Koshikov wrote:
> > I have one user, which have a lot of folders (not it is 414 without INBOX). 
> > When he tries to create one more folder with random name - the folder 
> > successfully created, but LIST command invokes out of memory to imap 
> > process.
> > 
> > From the logs:
> > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory 
> > when allocating 268435472 bytes
> > Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** 
> > imap: double free or corruption (!prev): 0x0812ba00 ***
> 
> This is bad. Can you reproduce it easily? Could you cause this crash
> under valgrind? For example:
> 
> protocol imap {
>   mail_executable = /usr/local/bin/imap.sh
> }
> 
> and /usr/local/bin/imap.sh:
> 
> #!/bin/sh
> 
> if [ $USER = "gozhd...@domain.com" ]; then
>   exec /usr/bin/valgrind /usr/libexec/dovecot/imap
> else
>   exec /usr/libexec/dovecot/imap
> fi
> 
> Valgrind will hopefully then log to Dovecot's log what exactly is the
> problem.
> 
In attachement full valgrind output for user's session. 
Steps I perform:
1)Login with admin master password to user's mailbox(from webmail)
2)create new directory
3)try to get directories list

> It's anyway related to one of your virtual maiboxes. I suppose you have
> a virtual mailbox that includes messages from all real mailboxes?

Yes, I have two virtual mailboxes:
all - contains all messages
favorites - contains flagged letters

But this user didn't subscribe to any of them.Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Memcheck, a memory error detector.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Using LibVEX rev 1884, a library for dynamic binary translation.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== For more details, rerun with: -v
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== 
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== 
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 59 from 2)
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== malloc/free: in use at exit: 129 bytes in 3 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== malloc/free: 165 allocs, 162 frees, 105,524 bytes allocated.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== For counts of detected errors, rerun with: -v
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== searching for pointers to 3 not-freed blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== checked 80,788 bytes.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== 
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== LEAK SUMMARY:
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==definitely lost: 41 bytes in 1 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==  possibly lost: 0 bytes in 0 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==still reachable: 88 bytes in 2 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== suppressed: 0 bytes in 0 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== Rerun with --leak-check=full to see details of leaked memory.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Memcheck, a memory error detector.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Using LibVEX rev 1884, a library for dynamic binary translation.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== For more details, rerun with: -v
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== 
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644== 
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644== ERROR SUMMARY: 0 er

[Dovecot] expire plugin + --exec-mail in 1.2.6 version

[Nikita Koshikov]

Hello list,

After upgrade to dovecot version 1.2.6 I have noticed some quota 
malfunctions(users become often overquota). Looking at logs closely I have 
found that quota value resets to general dovecot-config(
quota = maildir:Mailbox quota
quota_rule = *:storage=500M
quota_rule2 = Trash:storage=10%%
), after dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh command. 
The  problem that some users have their own quotas stored in ldap. 

For example, user s...@domain.com have maildirsize before running 
expire-tool(1G value from ldap):
# head -2 /data/mail/domain.com/sr/data/maildirsize 
1048576S
836423205 8285

Then I run expire-tool with mail_debug enabled:
# /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh 
Info: Loading modules from directory: /usr/lib/dovecot/imap
Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Info: Quota root: name=Mailbox quota backend=maildir args=
Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0
Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) 
messages=0
Info: Quota warning: bytes=471859200 (90%) messages=0 
command=/etc/dovecot/plugins/quota_warning.sh 90
Info: expire: pattern=Trash type=expunge secs=2592000
Info: expire: pattern=Spam type=expunge secs=2592000
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/koshikov.n
Info: auth input: mail=maildir:~/data
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, 
inbox=/data/mail/domain.com/koshikov.n/data
Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2
Info: koshiko...@domain.com/Spam: seq=1 uid=1869: Expunge
Info: koshiko...@domain.com/Spam: seq=2 uid=1870: Expunge
Info: koshiko...@domain.com/Spam: seq=3 uid=1871: Expunge
Info: koshiko...@domain.com/Spam: seq=4 uid=1872: Expunge
Info: koshiko...@domain.com/Spam: seq=5 uid=1873: Expunge
Info: koshiko...@domain.com/Spam: seq=6 uid=1874: Expunge
Info: koshiko...@domain.com/Spam: seq=7 uid=1875: Expunge
Info: koshiko...@domain.com/Spam: seq=8 uid=1876: Expunge
Info: koshiko...@domain.com/Spam: seq=9 uid=1877: Expunge
Info: koshiko...@domain.com/Spam: seq=10 uid=1878: Expunge
Info: koshiko...@domain.com/Spam: seq=11 uid=1879: Expunge
Info: koshiko...@domain.com/Spam: seq=12 uid=1880: Expunge
Info: koshiko...@domain.com/Spam: timestamp 1256618680 (Tue Oct 27 06:44:40 
2009) -> 1256716674 (Wed Oct 28 09:57:54 2009)
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/fsb
Info: auth input: mail=maildir:~/data
Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes
Info: maildir: data=~/data
Info: maildir++: root=/data/mail/domain.com/fsb/data, index=, control=, 
inbox=/data/mail/domain.com/fsb/data
Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, 
list=yes, subscriptions=no
Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
Info: maildir++: root=/var/mail/virtual, index=, control=, inbox=
Info: Namespace : Using permissions from /data/mail/domain.com/fsb/data: 
mode=0700 gid=-1
Info: trash plugin: Added 'Trash' with priority 1
Info: trash plugin: Added 'Spam' with priority 2
Info: f...@domain.com/Trash: seq=1 uid=578: Expunge
Info: f...@domain.com/Trash: seq=2 uid=579: Expunge
Info: f...@domain.com/Trash: seq=3 uid=580: Expunge
Info: f...@domain.com/Trash: seq=4 uid=581: Expunge
Info: f...@domain.com/Trash: seq=5 uid=582: Expunge
Info: f...@domain.com/Trash: seq=6 uid=583: Expunge
Info: f...@domain.com/Trash: seq=7 uid=584: Expunge
Info: f...@domain.com/Trash: seq=8 uid=585: Expunge
Info: f...@domain.com/Trash: seq=9 uid=586: Expunge
Info: f...@domain.com/Trash: seq=10 uid=587: Expunge
Info: f...@domain.com/Trash: seq=11 uid=588: Expunge
Info: f...@domain.com/Trash: seq=12 uid=589: Expunge
Info: f...@domain.com/Trash: seq=13 uid=590: Expunge
Info: f...@domain.com/Trash: timestamp 1256627284 (Tue Oct 27 09:08:04 2009) -> 
1256732991 (Wed Oct 28 14:29:51 2009)
Info: auth input: uid=8
Info: auth input: gid=12
Info: auth input: home=/data/mail/domain.com/arsen
Info: auth i

Re: [Dovecot] sieve and fileinto encoding

[Nikita Koshikov]

On Wed, 28 Oct 2009 00:14:50 +0300
proton-sss  wrote:

> Hello All!
> I`m using dovecot 1.2.6
> I have folder named in russian "САПП", and when in filter i wrote:
>  fileinto "САПП";
> all works well.
> 
> Now i find 2 web applications to manage sieve filters (horde-ingo and 
> squirrelmail/avelsieve) - then creates filter like this:
> if header :contains "Subject" "САПП"
> {
> fileinto  "&BCEEEAQfBB8-";
> stop;
> }
> 
> And creates another folder ".&-BCEEEAQfBB8-", but GOOD folder name 
> already exist ".&BCEEEAQfBB8-".
> 
> Please help to understand why dovecots deliver make this happen.
> 
> Best regards
> Michael

Take a look at http://dovecot.org/list/dovecot/2009-July/041690.html

Also, roundcube with latest sieverules plugin can handle utf8-sieve directories 
in correct way.

[Dovecot] mailbox count folders issues

[Nikita Koshikov]

Hello list, 

I have one user, which have a lot of folders (not it is 414 without INBOX). 
When he tries to create one more folder with random name - the folder 
successfully created, but LIST command invokes out of memory to imap process.

From the logs:
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory 
when allocating 268435472 bytes
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: 
double free or corruption (!prev): 0x0812ba00 ***
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: === Backtrace: =
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: /lib/libc.so.6[0xb7e8a714]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: 
/lib/libc.so.6(cfree+0x9c)[0xb7e8bfcc]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: imap(t_pop+0x7a)[0x80f0a1a]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: 
imap(t_pop_check+0xf)[0x80f0b9f]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: imap[0x80f199d]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: imap[0x80f2285]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: imap[0x80f1bd9]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: imap[0x80f099d]
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: Sending log messages too 
fast, throttling..
Oct 26 15:26:38 dovecot: Error: child 9497 (imap) killed with signal 6 (core 
dumps disabled)
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: imap[0x80f0e17]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: imap[0x80fccc8]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: imap[0x80efd3a]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
imap(buffer_write+0xd2)[0x80f0242]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
imap(buffer_append+0x26)[0x80f0366]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
imap(mail_transaction_log_append+0x5dc)[0x80d457c]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: imap[0x80c90d9]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
imap(mail_index_sync_commit+0xa5)[0x80cc625]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so[0xb7de7cee]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c)[0xb7dea7cc]
Oct 26 15:26:39 IMAP(gozhd...@domain.com): Error: 
imap(mailbox_sync+0x35)[0x80b6335]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(cmd_select_full+0x3d8)[0x8065088]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(cmd_select+0x19)[0x80657e9]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: imap[0x8067a9c]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: imap[0x8067b35]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(client_handle_input+0x2d)[0x8067ced]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(client_input+0x5f)[0x80686af]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(io_loop_handler_run+0xe0)[0x80fae60]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
imap(io_loop_run+0x20)[0x80fa2e0]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: imap(main+0x5ea)[0x807116a]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: 
/lib/libc.so.6(__libc_start_main+0xe5)[0xb7e35725]
Oct 26 15:26:40 IMAP(gozhd...@domain.com): Error: imap[0x80602b1]
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: === Memory map: 
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 08048000-08129000 r-xp 
 08:02 385446 /usr/libexec/dovecot/imap
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 08129000-0812a000 r--p 
000e 08:02 385446 /usr/libexec/dovecot/imap
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 0812a000-0812c000 rw-p 
000e1000 08:02 385446 /usr/libexec/dovecot/imap
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 0812c000-0bbbd000 rw-p 
0812c000 00:00 0  [heap]
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 9260-92621000 rw-p 
9260 00:00 0 
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: 92621000-9270 ---p 
92621000 00:00 0 
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: b280d000-b2819000 r-xp 
 08:02 263779 /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: b2819000-b281a000 r--p 
b000 08:02 263779 /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: b281a000-b281b000 rw-p 
c000 08:02 263779 /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Oct 26 15:26:41 IMAP(gozhd...@domain.com): Error: b2822000-b7d88000 rw-p 
b2822000 00:00 0 
Oct 26 15:26:42 IMAP(gozhd...@domain.com): Error: b7d88000-b7d9 r--s 
 08:11 24109678   
/data/mail/domain.com/gozhda.a/data/dovecot.index.cache
Oct 26 15:26:42 IMAP(gozhd...@domain.com): Error: b7d9-b7d95000 r--s 
 08:11 24109800   
/data/mail/domain.com/gozhda.a/data/.Supplier.Furniture.Prima 
Ridhacitra.Inbox/dovecot.index.cache
Oct 26 15:26:42 IMAP(gozhd...@domain.com): Error: b7d95000-b7d9a000 r--s 
 08:11 24117515   
/data/mail

Re: [Dovecot] Dovecot deliver with AD LDAP userdb

[Nikita Koshikov]

On Wed, 14 Oct 2009 16:00:54 -0400
Mark Schaub  wrote:

> Good Afternoon,
> 
> I have my virtual users stored in an Active Directory database. As far
> as mail info goes the 2 important fields are: mail (their primary
> email address) and otherMailbox (a multivalue attribute containing
> their mail aliases). Right now all email addresses belong in the same
> domain and there are no immediate plans to change this.
> 
> If I use Postfix to do the lookups and delivery only using Dovecot for
> passdb Pam driver and a static userdb then everything delivers fine to
> mail and otherMailbox addresses. I have been working on switching over
> to Dovecot's deliver process in order to integrate sieve capability. I
> have configured userdb ldap and passdb ldap in Dovecot and have set up
> master and client sockets. Before I describe my problem let me show
> you my configuration (as I think the dovecot-ldap.conf in particular
> needs to be explained). Here are the important parts of dovecot.conf:
> 
> auth default {
>   socket listen {
> master {
> path = /usr/var/run/dovecot/auth-master
> mode = 0666
> user = vmail
> group = vmail
> }
> 
> client {
> path = /var/spool/postfix/private/auth
> mode = 0666
> user = postfix
> group = postfix
> }
>  }
> 
>  passdb ldap {
> args = /etc/dovecot-ldap.conf
>   }
>  userdb ldap {
> # Path for LDAP configuration file, see doc/dovecot-ldap-example.conf
> args = /etc/dovecot-ldap.conf
>   }
> 
> and my dovecot -n:
> 
> lda:
>   postmaster_address: postmas...@sau24.org
>   mail_plugins: sieve
> auth default:
>   mechanisms: plain login
>   debug: yes
>   passdb:
> driver: ldap
> args: /etc/dovecot-ldap.conf
>   userdb:
> driver: ldap
> args: /etc/dovecot-ldap.conf
>   socket:
> type: listen
> client:
>   path: /var/spool/postfix/private/auth
>   mode: 438
>   user: postfix
>   group: postfix
> master:
>   path: /usr/var/run/dovecot/auth-master
>   mode: 438
>   user: vmail
>   group: vmail
> plugin:
>   sieve: ~/.dovecot.sieve
>   sieve_dir: ~/sieve
>   sieve_storage: ~/sieve
> 
> Further, here is my dovecot-ldap.conf:
> 
> hosts = delldb.sau24.org:3268
> base = dc=sau24, dc=org
> ldap_version = 3
> dn = cn=Mail User,cn=Users,dc=sau24,dc=org
> dnpass = secret
> auth_bind = yes
> scope = subtree
> user_attrs = sAMAccountName=home=/home/vmail/%$,=uid=501,=gid=501
> pass_attrs = sAMAccountName=user
> user_filter = 
> (&(objectclass=person)(|(sAMAccountName=%n)(mail...@sau24.org)(othermailbox...@sau24.org)))
> pass_filter = 
> (&(objectclass=person)(|(sAMAccountName=%n)(mail...@sau24.org)(othermailbox...@sau24.org)))
> 
> Now, you'll notice that the user_filter and pass_filter have the
> domain statically placed after the mail and othermailbox attributes.
> This reason for this is that this ldap configuration is used for both
> authentication purposes and local delivery, so its not always being
> passed the domain info (for authentication it gets the users IP
> address). Since all my users are on the same domain I can statically
> set this here. This setup authenticates users to imap and pop (users
> use their sAMAccountName as their username when they log in), and
> pulls their correct home directory. Checking mail is not a problem at
> all.
> 
> Further, this setup will deliver mail correctly to mail addressed to
> the value in the "mail" attribute. The problem seems to be in the
> otherMailbox attribute, it cannot lookup the users sAMAccountName
> based off of the otherMailbox field. Here is what it looks like in the
> logs for mail sent to my "mail" attribute (mark.sch...@sau24.org):
> 
> Oct 14 12:36:19 auth(default): Info: master in: USER1
> mark.sch...@sau24.org   service=deliver
> Oct 14 12:36:19 auth(default): Info: ldap(mark.sch...@sau24.org): user
> search: base=dc=sau24, dc=org scope=subtree
> filter=(&(objectclass=user)(|(sAMAccountName=mark.schaub)(mail=mark.sch...@sau24.org)(othermailbox=mark.sch...@sau24.org)))
> fields=sAMAccountName
> Oct 14 12:36:19 auth(default): Info: ldap(mark.sch...@sau24.org):
> result: sAMAccountName(home=/home/vmail/%$)=/home/vmail/mschaub
> Oct 14 12:36:19 auth(default): Info: master out: USER   1
> mark.sch...@sau24.org   home=/home/vmail/mschaubuid=501
> gid=501
> Oct 14 12:36:19 deliver(mark.sch...@sau24.org): Info: sieve:
> msgid=<6b71e3a70910140936v4e8cd6fdwd847a1721a9d9...@mail.gmail.com>:
> stored mail into mailbox 'INBOX'
> 
> But this is what I get when I try to send to an address listed in my
> otherMailbox field (t...@sau24.org):
> 
> Oct 14 12:30:58 auth(default): Info: master in: USER1
> t...@sau24.org  service=deliver
> Oct 14 12:30:58 auth(default): Info: ldap(t...@sau24.org): user
> search: base=dc=sau24, dc=org scope=subtree
> filter=(&(objectclass=user)(|(sAMAccountName=tech)(mail=t...@sau24.org)(othermailbox=t...@sau24.org)))
> fields=sAMAccountName
> Oct 14 12

Re: [Dovecot] expire plugin: columns not uniq

[Nikita Koshikov]

On Mon, 24 Aug 2009 13:33:15 -0400
Timo Sirainen  wrote:

> On Mon, 2009-08-24 at 13:27 -0400, Timo Sirainen wrote:
> > > CREATE TRIGGER mergeexpire BEFORE INSERT ON expires FOR EACH ROW
> > > BEGIN 
> > >   UPDATE expires SET expire_stamp=NEW.expire_stamp 
> > >   WHERE username = NEW.username AND mailbox = NEW.mailbox; 
> > >   SELECT raise(ignore) 
> > >   WHERE (SELECT 1 FROM expires WHERE username = NEW.username AND 
> > > mailbox = NEW.mailbox) IS NOT NULL;
> > > END;
> > > 
> > > This seem some kind crudely to me, but it's working.
> > 
> > That looks correct to me. 
> 

Sorry for such lately response, I have just returned from vacation.

> Oh, except there's a small race condition if the first row is being
> added at the same time by two processes. That's why the PostgreSQL
> trigger is so much more complex. But it should be somewhat rare I
> guess..
> 
> Unless SQLite has some locks that prevent that? Anyway I was thinking
> that two processes run the UPDATE part of the trigger and then both try
> to INSERT. One of them succeeds and the other one fails. But the row
> gets added anyway and the timestamp is the same anyway, so it probably
> doesn't matter all that much, just logs an error.

As far as I know, sqlite permits to write database changes(INSERT,UPDATE or 
DELETE) to only 1 process at the same time and when process is updating data - 
sqlite table is locking. This might be OS queue processing, because  sqlite 
have no master process for handling such racing. In my understanding, if two 
identical INSERTs will be passed to database(and the record already exists) - 
trigger's UPDATE will be run twice, but consistently.
My admissions can be faulty and sqlite experts should shed some light on this 
situation.
The above trigger is working about a week in my setup - I have checked error 
log for this period - there is nothing related to database issues.

Re: [Dovecot] expire plugin: columns not uniq

[Nikita Koshikov]

No, in general.

> If one wants some data in a DB, but update if already present.
>
> If you re-programm the TRIGGER in the client, you need to lock the whole
> table, if you want to overcome all race conditions and also don't want any
> errors logged (at least with postgres).
>

Thanks for explanation.
I'm not so aware in databases, but if trigger will be programmed on client
side - this will make code more portable(in case of database choicing) and
what is the less evil -  locking or portability...
In any case, thanks for reply, it helps me much.

Re: [Dovecot] expire plugin: columns not uniq

[Nikita Koshikov]

> Search the Wiki about Postgres and the requirement of a TRIGGER.
> I guess that sqlite requires something like this.

Maybe someone family with sqlite will say how to make this trigger better and 
constrain check field existence before doing UPDATE

CREATE TRIGGER mergeexpire BEFORE INSERT ON expires FOR EACH ROW
BEGIN 
UPDATE expires SET expire_stamp=NEW.expire_stamp 
WHERE username = NEW.username AND mailbox = NEW.mailbox; 
SELECT raise(ignore) 
WHERE (SELECT 1 FROM expires WHERE username = NEW.username AND 
mailbox = NEW.mailbox) IS NOT NULL;
END;

This seem some kind crudely to me, but it's working.

Re: [Dovecot] expire plugin: columns not uniq

[Nikita Koshikov]

On Fri, 21 Aug 2009 10:48:42 +0200 (CEST)
Steffen Kaiser  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> On Wed, 19 Aug 2009, Nikita Koshikov wrote:
> 
> > How can I solve this ? and why expire plugin is trying to use INSERT, 
> > instead of UPDATE ?
> 
> Search the Wiki about Postgres and the requirement of a TRIGGER.
> I guess that sqlite requires something like this.
>
Thank for your answer.
I'll dig in this direction.
 
> Unfortunately, there is no UPDATE OR INSERT in SQL and it is hard to 
> simulate.

You mean in dovecot expire plugin or where ?

> 
> Bye,
> 
> - -- 
> Steffen Kaiser
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.6 (GNU/Linux)
> 
> iQEVAwUBSo5fbnWSIuGy1ktrAQKQdQgAs1fioq/x86JNb9L7SvGnU9wPOUICilsI
> RcY+PrsMzGxdRShlnDhEIJGRlbzr7jHDhDhMfej+TGP9XqBFPqBphVxT8JIJmGXz
> Z4qEvwLoyuSKxWikLxVLaslDPYPu2tjmvKt+HWaLgev7ecwFF8NoKBHhWV6fR3x9
> o/xVVcj6hM38SkiGc6+t27OC5ch6Tq5Vqmau4QIB7fkN8bA8IoBU17rM0eW4k80m
> 7nKTVsPqFcDKlmes2d5vJVlCYg+NCy69RVIZEFQwOJahBjveSvJsXWnhks9e0nEk
> PmtMPMUFHOvfe+4KfK2GFxAPCznOSrxfgMuUewXnENCgLtXrox+zPA==
> =8vpx
> -END PGP SIGNATURE-

[Dovecot] expire plugin: columns not uniq

[Nikita Koshikov]

Hello list,

My dovecot setup have expire plugin enabled. It is working fine: new records 
added to database(sqlite), expire-tool successfully expunges old mail, but when 
expire-tool try to update database record for processed user, I get error in 
logs:

r...@mail dovecot 0:0 # /usr/sbin/dovecot --exec-mail ext 
/usr/libexec/dovecot/expire-tool.sh --test
Info: koshiko...@domain.com/Spam: timestamp 1250622406 (Tue Aug 18 22:06:46 
2009) -> 1250684287 (Wed Aug 19 15:18:07 2009)


in logs:
Aug 19 13:29:11 dict: Error: sqlite: exec(INSERT INTO expires 
(expire_stamp,username,mailbox) VALUES 
('1250684287','koshiko...@domain.com','Spam')) failed: columns username, 
mailbox are not unique (19)
Aug 19 13:29:11 dict: Error: sql dict: commit failed: 1҅�t+�]�Ћu�}��]Ð�t&


This is my expire dict config file:

connect = /var/mail/expire.db
#v1.2
map {
pattern = shared/expire/$user/$mailbox
table = expires
value_field = expire_stamp

fields {
username = $user
mailbox = $mailbox
}
}

expire plugin related dovecot sections:
dict {
  expire = sqlite:/etc/dovecot/plugins/expire.conf
}
plugin {
  expire = Trash 30 Spam 20
  expire_dict = proxy::expire
}

Version of sqlite is 3.6.16, dovecot-1.2.3.

How can I solve this ? and why expire plugin is trying to use INSERT, instead 
of UPDATE ?

Re: [Dovecot] virtual plugin and ACL

[Nikita Koshikov]

On Fri, 07 Aug 2009 15:23:32 -0400
Timo Sirainen  wrote:

> That's because in private namespaces user owns the mails, and
> "authenticated" doesn't reduce the user's privileges. You could use
> "owner" instead.
> 
> Also I don't think you should use ACLs at all here. It's easier and more
> secure to just make /var/mail/virtual non-writable to imap process. For
> example change file/dir owners to root and make them world-readable.

Thank you, Timo.

Both variants are working fine for me.

[Dovecot] virtual plugin and ACL

[Nikita Koshikov]

Hello list, 

We are using virtual plugin for to provide users two additional folders:
Company/all - it contains all messages in user's mailbox
Company/favorites - here is flagged messages.

Here is namespace part of config file:
namespace private {
prefix = Company/
separator = /
location = virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++
subscriptions = no
}

Everything works fine, but problem arises when someone rename or delete virtual 
mailbox. With above config, 1 user make changes to his mailbox, but changes 
distribute for entire domain.

First, I have tried to add :CONTROL=/data/mail/%d/%n/data/virtual to location 
part of virtual namespace, but I have no luck with such settings. Directory 
/data/mail/%d/%n/data/virtual not even been created.

Then I tried to setup ACL for virtual mailbox. Adding "acl" to mail_plugins in 
imap and lda section and acl=vfile to plugins config. Under /var/mail/virtual 
in each mailbox I create dovecot-acl file contaning:
user=koshikov.n lrwstiekxa
authenticated lrwstipe

But this didn't work.

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE 
AUTH=PLAIN AUTH=LOGIN] Server ready.
a login test qwe
a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT 
THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE 
UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES 
WITHIN CONTEXT=SEARCH QUOTA] Logged in
a list "" *
* LIST (\HasNoChildren) "/" "Sent"
* LIST (\HasNoChildren) "/" "Drafts"
* LIST (\HasNoChildren) "/" "Spam"
* LIST (\HasNoChildren) "/" "Trash"
* LIST (\HasNoChildren) "/" "INBOX"
* LIST (\Noselect \HasChildren) "/" "Company"
* LIST (\HasNoChildren) "/" "Company/all"
* LIST (\HasNoChildren) "/" "Company/favorites"
a OK List completed.
a select Company/all
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] 
Flags permitted.
* 13 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1249459221] UIDs valid
* OK [UIDNEXT 14] Predicted next UID
* OK [NOMODSEQ] No permanent modsequences
a OK [READ-WRITE] Select completed.
a getacl
a BAD Error in IMAP command GETACL: Unknown command.
a rename Company/all Company/test
a OK Rename completed.

After "rename completed" in /var/mail/virtual new file dovecot-acl-list is 
appear with content:
1249457074 test

How can I get workaround for this situation ? I just need to restrict my users 
renaming and deleting virtualboxes.

Re: [Dovecot] sieve delivery to utf folders

[Nikita Koshikov]

On Fri, 31 Jul 2009 09:06:04 +0200
Stephan Bosch  wrote:

> Well, I guess the final situation is caused by the fact that the folder 
> name is encoded in mUTF7 two times.
> 
> At my end the following Sieve script works as expected:
> 
> require "fileinto";
> 
> fileinto "тест";
> 
> Result:
> 
> Jul 31 09:01:02 xi dovecot: deliver(stephan): sieve: 
> msgid=<20090729182144.4c9ba...@jimbo>: stored mail into mailbox 
> '&BEIENQRBBEI-'
> 
> I guess you are doing:
> 
> require "fileinto";
> 
> fileinto "&BEIENQRBBEI-";
> 
> Result:
> 
> Jul 31 09:04:53 xi dovecot: deliver(stephan): sieve: 
> msgid=<20090729182144.4c9ba...@jimbo>: stored mail into mailbox 
> '&-BEIENQRBBEI-'
> 
> So, use the "тест" UTF-8 string in your scripts directly.

Thank you, Stephan.
The problem was with managesieve plugin to rouncube webmail, which construct 
"fileinto" rules with non-decoding utf-7 values. I should fix this in php.

Thanks again.

> 
> Regards,
> 
> -- 
> Stephan Bosch
> step...@rename-it.nl

Re: [Dovecot] sieve delivery to utf folders

[Nikita Koshikov]

On Thu, 30 Jul 2009 17:06:35 +0200
Stephan Bosch  wrote:

> You are using mUTF-7 in folder names. According to the Sieve 
> specification, you must use the UTF-8 version. The fileinto command 
> internally translates this to the mUTF-7 equivalent, so that the IMAP 
> server sees it as expected. This is a change of about half a year ago:
> 
> http://hg.rename-it.nl/dovecot-1.2-sieve/rev/fc0395d50d04
>
How can I set utf-8 as folder names ? my $LANG is ru_RU.UTF-8 now.
I might misunderstand something, but why client created folder name in mUTF-7 
and sieve fileinto internally converted mUTF-7 name are not the same ?

/usr/libexec/dovecot/imap-utf7 -r "тест"
&BEIENQRBBEI-   // this is like clients create mail

/usr/libexec/dovecot/imap-utf7 "&BEIENQRBBEI-"
тест//this decoding fine

/usr/libexec/dovecot/imap-utf7 "&-BEIENQRBBEI-"
&BEIENQRBBEI-   //this is like sieve created folder 
looks in the mail-client programs.

[Dovecot] sieve delivery to utf folders

[Nikita Koshikov]

Hello list,

Today, my users have noticed, that delivery to folders with non-english names 
is broken. For example, if I create russian directory, it correct encoded and 
display into mail-client program and my webmail:
Encoded:Normal view
.INBOX.&BEIENQRBBEI-русс

This is part of sieve script, which routing delivery to russian folder. 

elsif allof (header :contains "Subject" "qwe")
{
fileinto "INBOX/&BEIENQRBBEI-";
}

But, when actualy delivery take place, in logs I have

Jul 30 17:13:49 deliver(koshiko...@domain.com): Info: sieve: 
msgid=<4a71a9e6.5040...@domain.com>: stored mail into mailbox 
'INBOX/&-BEIENQRBBEI-

And folder .INBOX.&-BEIENQRBBEI- appears in users maildir. This folder can't be 
unencoded by mail-client program and user see exactly "&-BEIENQRBBEI-" under 
INBOX directory(also normaly encoded directory "&BEIENQRBBEI-" --> "русс" 
displays correct). Something breaks encoding and adds sing "-" to all words in 
utf encoded directory names.

I can't tell exactly when this problem appear, but I remember that I have 
tested delivery to russian directories by sieve scripts and all worked fine.

My current dovecot installation is 1.2.2, dovecot-sieve-0.1.9 and 
managesieve-0.11.7

Re: [Dovecot] sieve rules processing

[Nikita Koshikov]

On Thu, 30 Jul 2009 10:38:51 +0200
Stephan Bosch  wrote:

> Nikita Koshikov schreef:
> > On Thu, 30 Jul 2009 01:29:26 +0200
> > Stephan Bosch  wrote:
> >> I am not exactly sure what you mean. Why are you providing an explicit 
> >> keep command when you want to sort the message further? 
> > Without keep action - mail just redirected to specified mailbox, but it 
> > doesn't stores into my inbox.
> [..]
> > This is my current rule list:
> > 
> > require ["fileinto","envelope","copy"];
> > # rule:[redirect]
> > if anyof (true)
> > {
> > redirect :copy "al...@domain.com";
> > }
> > # rule:[Cron]
> > elsif anyof (header :contains "Subject" "Cron")
> > {
> > fileinto "INBOX/Cron";
> > }
> >  
> [..]
> > But this mail save into INBOX, for user koshiko...@domain.com, (the second 
> > rule is not working), in logs I have:
> > Jul 30 10:29:04 deliver(koshiko...@domain.com): Info: sieve: 
> > msgid=<4a714b09.9030...@domain.com>: forwarded to 
> > Jul 30 10:29:04 deliver(koshiko...@domain.com): Info: sieve: 
> > msgid=<4a714b09.9030...@domain.com>: stored mail into mailbox 'INBOX'
> > Jul 30 10:29:04 deliver(al...@domain.com): Info: sieve: 
> > msgid=<4a714b09.9030...@domain.com>: stored mail into mailbox 'INBOX'
> > 
> > So :copy acting for me like "keep". Do you have any ideas, why this might 
> > happen ?
> 
> Yes, your second rule is an elsif, meaning that it is only executed when 
> the first if-condition is not 'true'. Since that first rule has a forced 
> 'true' result, the second rule is absolutely never executed. You script 
> should be:
> 
> require ["fileinto","copy"];
> 
> # rule:[redirect]
> if true
> {
>redirect :copy "al...@domain.com";
> }
> 
> # rule:[Cron]
> if header :contains "Subject" "Cron"
> {
>fileinto "INBOX/Cron";
> }
> 
> Note that the anyof() commands are only necessary when you have multiple 
> tests per if command. Also, I removed the envelope require, since it is 
> not used.

Thank you, Stephan.
Your example rules works like a charm.
> 
> Regards,
> 
> -- 
> Stephan Bosch
> step...@rename-it.nl

Re: [Dovecot] sieve rules processing

[Nikita Koshikov]

On Thu, 30 Jul 2009 10:32:20 +0200
Tom Hendrikx  wrote:

> You are using a if/elseif/else statement. When the if clause (which is
> always true) is matched, the stuff in the elsif/else clauses is discarded.
> 
> You'll need to use 2 separate if statements to make this work.
> 
Thank you for explanation, Tom. Dividing rule list to two separate statements 
helped.

> --
> Regards,
>   Tom
> 

Re: [Dovecot] sieve rules processing

[Nikita Koshikov]

On Thu, 30 Jul 2009 01:29:26 +0200
Stephan Bosch  wrote:

Thanks for your reply, Stephan.

> I am not exactly sure what you mean. Why are you providing an explicit 
> keep command when you want to sort the message further? I would think 
> the following example script does what you need:
> 
Without keep action - mail just redirected to specified mailbox, but it doesn't 
stores into my inbox.

> ==
> require "fileinto";
> require "copy";
> 
> # Temporary rule
> if true {
>   # Forward a copy of all messages to gmail account
>   redirect :copy "koshi...@gmail.com";
> }
> 
> # File spam into spam folder and stop further processing
> if header :contains "X-spam" "yes" {
>   fileinto "Spam";
>   stop;
> }
> 
> # Mailing list
> if header :contains "list-id" "ietf-mta-filters.imc.org" {
>   fileinto "Lists.Sieve";
> }
> 
> # Implicit keep will store message in INBOX if it was not
> # stored already by the fileinto commands above. The :copy
> # argument to the redirect command prevents that command
> # from cancelling the implicit keep action.
> ==
> 
> I hope this helps you along.
> 
This is my current rule list:

require ["fileinto","envelope","copy"];
# rule:[redirect]
if anyof (true)
{
redirect :copy "al...@domain.com";
}
# rule:[Cron]
elsif anyof (header :contains "Subject" "Cron")
{
fileinto "INBOX/Cron";
}
 
Here is the message's headers:
Return-path: 
Envelope-to: koshiko...@domain.com
Delivery-date: Thu, 30 Jul 2009 10:29:04 +0300
Received: from [192.168.1.160]
by mail.domain.com with ESMTPSA (TLSv1:AES256-SHA:256) id 1MWQ4W-0007bH-38
(envelope-from )
for koshiko...@domain.com; Thu, 30 Jul 2009 10:29:04 +0300
Message-ID: <4a714b09.9030...@domain.com>
Date: Thu, 30 Jul 2009 10:26:01 +0300
From: "b...@domain.com" 
User-Agent: Thunderbird 2.0.0.22 (X11/20090605)
MIME-Version: 1.0
To: =?UTF-8?B?0JrQvtGI0LjQutC+0LIg0J3QuNC60LjRgtCw?=
 
Subject: Cron
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

test

But this mail save into INBOX, for user koshiko...@domain.com, (the second rule 
is not working), in logs I have:
Jul 30 10:29:04 deliver(koshiko...@domain.com): Info: sieve: 
msgid=<4a714b09.9030...@domain.com>: forwarded to 
Jul 30 10:29:04 deliver(koshiko...@domain.com): Info: sieve: 
msgid=<4a714b09.9030...@domain.com>: stored mail into mailbox 'INBOX'
Jul 30 10:29:04 deliver(al...@domain.com): Info: sieve: 
msgid=<4a714b09.9030...@domain.com>: stored mail into mailbox 'INBOX'

So :copy acting for me like "keep". Do you have any ideas, why this might 
happen ?

> Regards,
> 
> -- 
> Stephan Bosch
> step...@rename-it.nl

[Dovecot] sieve redirect action

[Nikita Koshikov]

Hello Stephan,

First of all, thank you for dovecot-sieve implementation.

I see that you are adding X-Sieve header to all outgoing e-mail in 
cmd-redirect.c 

/* Prepend sieve version header (should not affect signatures) */
rfc2822_header_field_write(f, "X-Sieve", SIEVE_IMPLEMENTATION);

How do you look at adding one more header, for example X-Sieve-Forward-From:

Now, when redirect action is in use - mail forwards using sendmail command line 
interface. The problem that there is no suitable way to know which user's 
script has generate redirect. Received: header may contain envelope receiver 
but this is valid if only 1 "rcpt to:" command was specified on SMTP session.
This invokes problem when forwarding mail to non-local domains, breaking such 
things like SPF. I'd like to setup SRS for fixing this, but I don't know 
sender("sieve local user") mail. Adding one header with valid local user e-mail 
address will fix this.
Google add 2 headers for this -  X-Forwarded-To: and X-Forwarded-For: .

What do you think about this ? 

[Dovecot] sieve rules processing

[Nikita Koshikov]

Hello list,

My question: is sieve support some kind of "continue" action, to explicit tell 
server processing rules below current one(if current has triged)?

Situation:
I have few rules, which fileinto's to different directories. All is working 
fine, but if I want redirect all my mail for some period of time to another 
mailbox, I'm adding rule like  this, to be processing first.

if anyof (true)
{
redirect "koshi...@gmail.com";
keep;
}

Mail redirecting, but keep state store messages to INBOX. All I want - to 
process other rules, which will move message to the right imap directory.
This situation also happens with vacation sieve-action.
In sieve rfc I didn't find any workaround to that actions.

Maybe someone know how to make sieve working in described way?
 

Re: [Dovecot] virtual plugin question

[Nikita Koshikov]

On Sun, 26 Jul 2009 19:52:30 -0400
Timo Sirainen  wrote:

> On Wed, 2009-07-22 at 22:39 +0300, Nikita Koshikov wrote:
> > > There's no -g parameter given to gcc. The emerge overrides CFLAGS by
> > > adding -march=i686 etc. stuff in there, which also removes -g parameter.
> > > Try to figure out some way to add it back.
> > 
> > (gdb) bt full
> > #0  0xb7d80080 in virtual_storage_sync_init () from 
> > /usr/lib/dovecot/imap/lib20_virtual_plugin.so
> > No symbol table info available.
> 
> Almost :) Only the virtual plugin now was missing symbols. Anyway, I
> think this fixes the crashes:
> http://hg.dovecot.org/dovecot-1.2/rev/88d4e83fcea5
> 

Thank you, Timo.
With dovecot version 1.2.2 this bug has gone away.

Re: [Dovecot] Gentoo overlay?

[Nikita Koshikov]

On Sat, 25 Jul 2009 17:43:47 -0400
Charles Marcus  wrote:

> Does anyone know of a decent gentoo overlay that keeps dovecot+
> reasonably up to date?
> 
> Thanks,

Take a look on http://bugs.gentoo.org/show_bug.cgi?id=276480.
You can simply add this ebuild to your local overlay tree. Here is more info 
about this http://www.gentoo-wiki.info/HOWTO_Installing_3rd_Party_Ebuilds.

Re: [Dovecot] virtual plugin question

[Nikita Koshikov]

> There's no -g parameter given to gcc. The emerge overrides CFLAGS by
> adding -march=i686 etc. stuff in there, which also removes -g parameter.
> Try to figure out some way to add it back.

(gdb) bt full
#0  0xb7d80080 in virtual_storage_sync_init () from 
/usr/lib/dovecot/imap/lib20_virtual_plugin.so
No symbol table info available.
#1  0x08070294 in imap_sync_init (client=0x8134a40, box=0x813d480, 
imap_flags=IMAP_SYNC_FLAG_SAFE, flags=0) at imap-sync.c:146
ctx = (struct imap_sync_context *) 0x83809b8
__PRETTY_FUNCTION__ = "imap_sync_init"
#2  0x0807057a in cmd_sync_delayed (client=0x8134a40) at imap-sync.c:591
cmd = 
first_expunge = 
first_nonexpunge = 
__PRETTY_FUNCTION__ = "cmd_sync_delayed"
#3  0x08067b64 in client_handle_input (client=0x8134a40) at client.c:728
ret = false
remove_io = false
handled_commands = true
__PRETTY_FUNCTION__ = "client_handle_input"
#4  0x0806838f in client_input (client=0x8134a40) at client.c:752
cmd = 
output = (struct ostream *) 0x8134bf4
bytes = 
__PRETTY_FUNCTION__ = "client_input"
#5  0x080fa720 in io_loop_handler_run (ioloop=0x812e9b0) at ioloop-epoll.c:208
ctx = (struct ioloop_handler_context *) 0x812eab8
event = (const struct epoll_event *) 0x812eaf8
list = (struct io_list *) 0x8134c78
io = (struct io_file *) 0x8134c50
tv = {tv_sec = 0, tv_usec = 395850}
t_id = 2
msecs = 
ret = 1
i = 0
j = 0
call = 
#6  0x080f9ba0 in io_loop_run (ioloop=0x812e9b0) at ioloop.c:338
No locals.
#7  0x08070e15 in main (argc=Cannot access memory at address 0xe
) at main.c:323
No locals.
(gdb) 

Re: [Dovecot] virtual plugin question

[Nikita Koshikov]

I'm find out how to reproduce the bug.
Config setting  contains "location = virtual:/var/mail/virtual" for all users.

1. User1 (with subscribed virtual mailboxes) login and check his virtual dirs.
2. User2 login to server, subscribe to the same virtual mailboxes and checks 
them.
3. User1 trying to check his virtual mailbox get segmentation fault.

> Yes, since each user's virtual mailbox would look different the index  
> files have to be also separate for each user. Otherwise they'll just  
> be recreated every time.
> 
> location = virtual:/var/mail/virtual:INDEX=~/virtual
> 
> Or something like that.

After I posted my message, I changed location to include INDEX option for 
testing and server works without any errors(about 4 hours with using virtual 
mailboxes), so adding INDEX fix the problem.

> 
> > (no debugging symbols found)
> ..
> > (gdb) bt full
> > #0  0xb7f6157c in ?? () from /usr/lib/dovecot/imap/ 
> > lib20_virtual_plugin.so
> > No symbol table info available.
> > #1  0x083895d8 in ?? ()
> 
> Still, it shouldn't crash. This backtrace with only "??" isn't useful.  
> Having debugging symbols would help a lot in fixing this crash.
> 

My current dovecot installation include --enable-devel-checks, but every core 
file that I got was with this ?? signs. Maybe I attach core file to wrong 
binary, I have tried with imap and imap-login - both looks the same.

[Dovecot] virtual plugin question

[Nikita Koshikov]

Hello list,

After enabling virtual plugin in dovecot config, I got imap segmentation faults 
from time to time.
I can't say what event trigs this behavior, but in logs I see:

Jul 22 13:55:01 dovecot: Error: child 20908 (imap) killed with signal 11 (core 
dumped)
Jul 22 13:56:43 dovecot: Error: child 19466 (imap) killed with signal 11 (core 
dumped)
Jul 22 14:06:16 dovecot: Error: child 22629 (imap) killed with signal 11 (core 
dumped)
Jul 22 14:08:17 dovecot: Error: child 22879 (imap) killed with signal 11 (core 
dumped)

Here is my config section, which I added to enable virtual plugin:
namespace private {
prefix = Company/
separator = /
location = virtual:/var/mail/virtual
subscriptions = no
}
and "virtual" to mail_plugins in imap and lda section. Full dovecot -n config 
provided at the end of this message.
I'm not sure, if it is save to enable virtual location for all users using the 
same path, but the main goal for me - to create dovecot-virtual files once for 
all users. Maybe INDEX-es should be personalize for this setup work correct ?

ls -la /var/mail/virtual/
drwxr-xr-x 2 mail mail 4096 Июл 22 15:43 all
drwxr-xr-x 2 mail mail 4096 Июл 22 15:37 favorites
-rw-r--r-- 1 mail mail   14 Июл 22 13:53 .virtual-subscriptions

cat /var/mail/virtual/all/dovecot-virtual 
#all emails
*
  all
cat /var/mail/virtual/favorites/dovecot-virtual 
#Mark messages
*
 flagged

Core dump:

gdb /usr/libexec/dovecot/imap /data/mail/domain.com/koshikov.n/core 
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)

warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...
(no debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...(no 
debugging symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...(no debugging 
symbols found)...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...
(no debugging symbols found)...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libbz2.so.1
(no debugging symbols found)
Core was generated by `imap'.
Program terminated with signal 11, Segmentation fault.
[New process 29801]
#0  0xb7f6157c in ?? () from /usr/lib/dovecot/imap/lib20_virtual_plugin.so
(gdb) bt full
#0  0xb7f6157c in ?? () from /usr/lib/dovecot/imap/lib20_virtual_plugin.so
No symbol table info available.
#1  0x083895d8 in ?? ()
No symbol table info available.
#2  0x0573 in ?? ()
No symbol table info available.
#3  0xbfbeada0 in ?? ()
No symbol table info available.
#4  0xbfbeadc8 in ?? ()
No symbol table info available.
#5  0xbfbeadeb in ?? ()
No symbol table info available.
#6  0x08389da4 in ?? ()
No symbol table info available.
#7  0xbfbeadb8 in ?? ()
No symbol table info available.
#8  0x08389da8 in ?? ()
No symbol table info available.
#9  0x in ?? ()
No symbol table info available.

dovecot -n:
# 1.2.1: /etc/dovecot/dovecot.conf
Warning: Growing pool 'settings' with: 8192
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.11.1 
log_path: /var/log/dovecot/dovecot-error.log
info_log_path: /var/log/dovecot/dovecot.log
protocols: imaps managesieve
ssl_cert_file: /etc/ssl/dovecot/imaps.crt
ssl_key_file: /etc/ssl/dovecot/imaps.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
logi

Re: [Dovecot] [Bug] dovecot deliver v1.2.1 segfaults

[Nikita Koshikov]

On Tue, 21 Jul 2009 12:50:05 +0200
Stephan Bosch  wrote:

> Nikita Koshikov wrote:
> > On Mon, 20 Jul 2009 15:24:40 -0400
> > Timo Sirainen  wrote:
> > 
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > 0xb7ec4713 in strlen () from /lib/libc.so.6
> > (gdb) bt
> > #0  0xb7ec4713 in strlen () from /lib/libc.so.6
> > #1  0xb7e0d742 in sieve_address_parse_envelope_path () from 
> > /usr/lib/dovecot/lda/lib90_sieve_plugin.so
> > #2  0xb7e1ef94 in ?? () from /usr/lib/dovecot/lda/lib90_sieve_plugin.so
> > #3  0x in ?? ()
> > 
> We found the bug. It is fixed in the Mercurial repository. The fix is 
> part of a rather large change that I need to test properly first. It 
> will be incorporated asap in the next release.
> 
> Regards,
> 
> Stephan

Thanks for your work and help, Stephan.
Waiting impatiently for new relese.

Re: [Dovecot] [Bug] dovecot deliver v1.2.1 segfaults

[Nikita Koshikov]

On Mon, 20 Jul 2009 15:24:40 -0400
Timo Sirainen  wrote:

> On Mon, 2009-07-20 at 22:05 +0300, Nikita Koshikov wrote:
> > Starting program: /usr/libexec/dovecot/deliver -d koshiko...@domain.com -p 
> > /root/mail.eml
> ..
> > Program exited with code 0113.
> 
> 0113 = 75 = EX_TEMPFAIL = deliver should have logged an error message to
> log file.
> 
Thank you for reply, Timo

yes, in logs I found permision issue, after chown to mail user, I got:

GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
(gdb) r
Starting program: /usr/libexec/dovecot/deliver -d koshiko...@domain.com -p 
/root/mail.eml
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
Warning: Growing pool 'Plugin strings' with: 1024
Warning: Growing pool 'Plugin strings' with: 2048
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program received signal SIGSEGV, Segmentation fault.
0xb7ec4713 in strlen () from /lib/libc.so.6
(gdb) bt
#0  0xb7ec4713 in strlen () from /lib/libc.so.6
#1  0xb7e0d742 in sieve_address_parse_envelope_path () from 
/usr/lib/dovecot/lda/lib90_sieve_plugin.so
#2  0xb7e1ef94 in ?? () from /usr/lib/dovecot/lda/lib90_sieve_plugin.so
#3  0x in ?? ()


> > > http://www.dovecot.org/bugreport.html
> > > 
> > I have tried to get it, but nothing appear in user's home directory
> 
> One of the most important things is to get "ulimit -c unlimited" run
> before your MTA. Or alternatively instead of executing deliver directly,
> have it execute a script:
> 
> #!/bin/sh
> ulimit -c unlimited
> /usr/libexec/dovecot/deliver $*
> 

Re: [Dovecot] [Bug] dovecot deliver v1.2.1 segfaults

[Nikita Koshikov]

Thanks for your respond, Stephan.
Here is information, that maybe will be usefull for debugging.

On Mon, 20 Jul 2009 18:53:15 +0200
Stephan Bosch  wrote:

> Since it fails hereafter it would not surprise me when this is a Sieve 
> problem. Could you provide:
> 
> 1) The Sieve scripts involved in the problem (anonymized if you care):
> 
> /etc/dovecot/sieve/default.sieve 
> /data/mail/domain.com/koshikov.n/.dovecot.sieve

/etc/dovecot/sieve/default.sieve

require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
  fileinto "Spam";
} 

/data/mail/domain.com/koshikov.n/.dovecot.sieve

require ["fileinto","envelope"];
# rule:[Cron]
if anyof (header :contains "Subject" "Cron")
{
fileinto "INBOX.Cron";
}
# rule:[Logwatch]
elsif anyof (header :contains "Subject" "Logwatch for")
{
fileinto "INBOX.Logwatch";
}
# rule:[Postmaster errors]
elsif anyof (address :contains "From" "MAILER-DAEMON")
{
fileinto "INBOX.Postmaster";
}
# rule:[Exim statistic]
elsif anyof (header :contains "Subject" "Exim statistic")
{
fileinto "INBOX.Mailstat";
}
# rule:[Arpwatch]
elsif anyof (envelope :contains "From" "arpwa...@guard.domain.com")
{
fileinto "INBOX.Arpwatch";
}
# rule:[backup]
elsif anyof (header :contains "Subject" "backup report")
{
fileinto "INBOX.Backup";
}

> 
> 2) An e-mail message that is known to trigger this failure
>
In attachment you can find exim -Mvh and exim -Mvb format message.

> > deliver[9476]: segfault at 0 ip b7e16713 sp bfa0ba2c error 4 in 
> > libc-2.9.so[b7da4000+13d000]
> > deliver[11572]: segfault at 0 ip b7f41713 sp bfb38b4c error 4 in 
> > libc-2.9.so[b7ecf000+13d000]
> > deliver[12433]: segfault at 0 ip b7e7e713 sp bfb73b8c error 4 in 
> > libc-2.9.so[b7e0c000+13d000]
> 
> If your Dovecot is compiled with debugging symbols enabled you can 
> obtain a useful trace by executing deliver in gdb:
> 
> gdb --args /usr/lib/dovecot/deliver -p mail.eml
> 
> Type 'r' and press enter and wait until it crashes. Type 'bt' to obtain 
> the back trace.
Here is a problem with sending parameters to deliver

 gdb --args /usr/libexec/dovecot/deliver -p /home/admin/message.eml 
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-linux-gnu"...
(no debugging symbols found)
(gdb) r
Starting program: /usr/libexec/dovecot/deliver -p /home/admin/message.eml
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
Fatal: destination user parameter (-d user) not given

If I add -d "koshiko...@domain.com" 
(gdb) r
Starting program: /usr/libexec/dovecot/deliver -d koshiko...@domain.com -p 
/root/mail.eml
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
Warning: Growing pool 'Plugin strings' with: 1024
Warning: Growing pool 'Plugin strings' with: 2048
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)

Program exited with code 0113.
(gdb) bt
No stack.

Here is mine configure options
./configure --prefix=/usr --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu 
--mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share 
--sysconfdir=/etc --localstatedir=/var/lib --localstatedir=/var 
--sysconfdir=/etc/dovecot --enable-header-install --with-ioloop=best 
--with-libcap --with-db --enable-devel-checks --without-gssapi --with-ldap 
--without-mysql --with-pam --without-pgsql --with-sqlite --without-vpopmail 
--with-ssl=openssl


> 
> Alternatively, you can obtain a core dump, but I must say that I have 
> little experience with producing one. Other people on this list can help 
> you with this. Basic information is available at:
> 
> http://www.dovecot.org/bugreport.html
> 
I have tried to get it, but nothing appear in user's home directory

> > plugin:
> [...]
> >   sieve: ~/.dovecot.sieve
> >   sieve_dir: ~/sieve
> >   sieve_extensions: +imapflags +notify
> I see that you have enabled deprecated notify support. What version are 
> you using? The notify support provided by the last release (0.1.8) is 
> not compatible with CMUSieve and it is therefore not (yet) useful to 
> enable this.

This two plugins was enbled for testing purposes only. I disabled them, but the 
problem is still here.
I'm using sieve-0.1.8 version now.

Also, I just have seen two more messages in exim's queue, that exited with 
segmentation faults, so problem not in system filter. If you need this two 
mail-examples please tell. They also exited after sieve section in logs.


> 
> Regards,
> 
> -- 
> Stephan Bosch
> step...@rename-it.nl


message.eml
Description: Binary data

[Dovecot] [Bug] dovecot deliver v1.2.1 segfaults

[Nikita Koshikov]

Hello list,

I have noticed strange dovecot behavior with exim system-filter.
I'm using exim+dovecot setup. My exim configuration have system filter enable, 
which restrict some attachments, like exe, vbs etc. When message with forbidden 
attachment received, exim accept the message, then construct 
bounce-message(Reject reason + original message) and returns it to sender. 
Trying to deliver such messages, I always get segmentation faults and message 
stayed in exim's queue.
Others common delivers work fine. This only happens with bouncing system-filter 
messages.

Steps to reproduce:
1) Create message with forbiden attachment
2) Send message.
3) Get segmentation fault.

Here is more details:
exim -d+all -M 1MSn7r-0003hN-BG (this command runs delivery from exim's queue)

13:14:23 11570 > koshiko...@domain.com <
13:14:23 11570 locking /var/spool/exim/db/retry.lockfile
13:14:23 11570 locked /var/spool/exim/db/retry.lockfile
13:14:23 11570 EXIM_DBOPEN(/var/spool/exim/db/retry)
13:14:23 11570 returned from EXIM_DBOPEN
13:14:23 11570 opened hints database /var/spool/exim/db/retry: flags=O_RDONLY
13:14:23 11570 dbfn_read: key=T:koshiko...@domain.com
13:14:23 11570 no retry record exists
13:14:23 11570 search_tidyup called
13:14:23 11571 changed uid/gid: local delivery to koshikov.n 
 transport=local_delivery
13:14:23 11571   uid=8 gid=12 pid=11571
13:14:23 11571   auxiliary group list: 
13:14:23 11571   home=NULL current=/
13:14:23 11571 set_process_info: 11571 delivering 1MSn7r-0003hN-BG to 
koshikov.n using local_delivery
13:14:23 11571 local_delivery transport entered
13:14:23 11571 direct command:
13:14:23 11571   argv[0] = /usr/libexec/dovecot/deliver
13:14:23 11571   argv[1] = -e
13:14:23 11571   argv[2] = -d
13:14:23 11571   argv[3] = $local_p...@$domain
13:14:23 11571 expanding: $local_p...@$domain
13:14:23 11571result: koshiko...@domain.com
13:14:23 11571 direct command after expansion:
13:14:23 11571   argv[0] = /usr/libexec/dovecot/deliver
13:14:23 11571   argv[1] = -e
13:14:23 11571   argv[2] = -d
13:14:23 11571   argv[3] = koshiko...@domain.com
13:14:23 11571 Writing message to pipe
13:14:23 11571 writing data block fd=11 size=0 timeout=3600
13:14:23 11573 set_process_info: 11573 reading output from 
|/usr/libexec/dovecot/deliver -e -d $local_p...@$domain
13:14:23 11571 writing data block fd=11 size=814 timeout=3600
13:14:23 11571 writing data block fd=11 size=0 timeout=3600
13:14:23 11571 local_delivery transport yielded 2
13:14:23 11571 search_tidyup called
13:14:23 11570 local_delivery transport returned FAIL for koshiko...@domain.com
13:14:23 11570 post-process koshiko...@domain.com (2)
13:14:23 11570 LOG: MAIN
13:14:23 11570   : local_delivery transport output: 
Warning: Growing pool 'Plugin strings' with: 1024
13:14:23 11570 LOG: MAIN
13:14:23 11570   ** koshiko...@domain.com F=<> R=ldap_accept T=local_delivery: 
Child process of local_delivery transport (running command 
"/usr/libexec/dovecot/deliver -e -d $local_p...@$domain") was
terminated by signal 11 (Segmentation fault)
13:14:23 11570  deliveries are done 
13:14:23 11570 changed uid/gid: post-delivery tidying
13:14:23 11570   uid=8 gid=12 pid=11570
13:14:23 11570   auxiliary group list: 
13:14:23 11570 set_process_info: 11570 tidying up after delivering 
1MSn7r-0003hN-BG
13:14:23 11570 Processing retry items
13:14:23 11570 Succeeded addresses:
13:14:23 11570 koshiko...@domain.com: no retry items
13:14:23 11570 Failed addresses:
13:14:23 11570 Deferred addresses:
13:14:23 11570 koshiko...@domain.com: no retry items
13:14:23 11570 koshiko...@domain.com: no retry items
13:14:23 11570 end of retry processing
13:14:23 11570 LOG: MAIN
13:14:23 11570   Frozen (delivery error message)
13:14:23 11570 delivery deferred: update_spool=1 header_rewritten=0
13:14:23 11570 Writing spool header file
13:14:23 11570 Size of headers = 707
13:14:23 11570 end delivery of 1MSn7r-0003hN-BG
13:14:23 11570 search_tidyup called
13:14:23 11570 search_tidyup called
13:14:23 11570  Exim pid=11570 terminating with rc=0 



dovecot-deliver log, while trying to deliver this message:

==> /var/log/dovecot/dovecot-deliver.log <==
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: Loading modules from 
directory: /usr/lib/dovecot/lda
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: Module loaded: 
/usr/lib/dovecot/lda/lib10_quota_plugin.so
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: Module loaded: 
/usr/lib/dovecot/lda/lib11_trash_plugin.so
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: Module loaded: 
/usr/lib/dovecot/lda/lib20_expire_plugin.so
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: Module loaded: 
/usr/lib/dovecot/lda/lib90_sieve_plugin.so
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: auth input: uid=8
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: auth input: gid=12
Jul 20 13:19:53 deliver(koshiko...@domain.com): Info: auth input: 

Re: [Dovecot] exim4 dovecot sasl problem

[Nikita Koshikov]

On Mon, 20 Jul 2009 00:07:21 -0700 (PDT)
Tristan Rex Santiago  wrote:

> 
> what flag do i need to set in recompiling exim4 to enable dovecot-sasl 
> support? i tried --enable-dovecot-sasl but exim4 still reported dovecot 
> driver error.
> 
AUTH_DOVECOT=yes in Makefile

http://www.exim.org/exim-html-4.68/doc/html/spec_html/ch04.html

> many thanks
> --- On Fri, 7/17/09, Tristan Rex Santiago  wrote:
> 
> From: Tristan Rex Santiago 
> Subject: exim4 dovecot sasl problem
> To: dovecot@dovecot.org
> Cc: tansa...@gmail.com
> Date: Friday, 17 July, 2009, 4:36 PM
> 
> hi!
> 
> i'm trying to do smtp authentication using dovecot and exim4 by following 
> this guide http://wiki.dovecot.org/HowTo/ EximAndDovecotSASL.
> 
> 
> for the exim4 part, i got an
> 
> authenticator dovecot: cannot find authenticator driver "dovecot"
>  error after i entered the codes in /etc/exim4/exim4.conf.template and 
> reloaded exim4 with the update-exim4.conf command.
> 
> i'm running debian lenny.
> 
> here's the authentication section of exim4.conf.template:
> 
>  begin authenticators
> 
> cram_md5:
> 
>   driver = cram_md5
> 
>   public_name = CRAM-MD5
> 
> 
>   client_name = ${extract{1}{:}{${lookup{$
> host}nwildlsearch{CONFDIR/ passwd.client}{$value}fail}}}
>   client_secret = ${extract{2}{:}{${lookup{$ host}nwildlsearch{CONFDIR/ 
> passwd.client}{$value}fail}}}
> 
> PASSWDLINE=${sg{\
> 
>     ${lookup{$host}nwildlsearch{ CONFDIR/passwd.client}{$value} 
> fail}\
>         }\
>         {\\N[\\^]\\N}\
>         {^^}\
>         }
> 
> plain:
>   driver = plaintext
>   public_name = PLAIN
> 
> .ifndef AUTH_CLIENT_ALLOW_NOTLS_ PASSWORDS
>   client_send = "<; ${if !eq{$tls_cipher}{}\
>     {^${extract{1}{:}{PASSWDLINE}} \
>          ^${sg{PASSWDLINE}{\\N([^:]+:)( .*)\\N}{\\$2}}\
>            }fail}"
> 
> .else
>   client_send = "<; ^${extract{1}{:}{PASSWDLINE}}\
>             ^${sg{PASSWDLINE}{\\N([^:]+:)( .*)\\N}{\\$2}}"
> .endif
> 
> login:
>   driver = plaintext
>   public_name = LOGIN
> .ifndef AUTH_CLIENT_ALLOW_NOTLS_ PASSWORDS
> 
>   # Return empty string if not non-TLS AND looking up $host in passwd-file
>   # yields a non-empty string; fail otherwise.
>   client_send = "<; ${if and{\
>   {!eq{$tls_cipher}{}}\
> 
>   {!eq{PASSWDLINE}{}}\
>  }\
>   {}fail}\
>  ; ${extract{1}{::}{PASSWDLINE}}\
>          ; ${sg{PASSWDLINE}{\\N([^:]+:)(. *)\\N}{\\$2}}"
> 
> .else
>   # Return empty string if looking up $host in passwd-file yields a
>   # non-empty string; fail otherwise.
>   client_send = "<; ${if !eq{PASSWDLINE}{}\
>   {}fail}\
>  ; ${extract{1}{::}{PASSWDLINE}}\
> 
>          ; ${sg{PASSWDLINE}{\\N([^:]+:)(. *)\\N}{\\$2}}"
> .endif
> 
> dovecot_login:
>   driver = dovecot
>   public_name = LOGIN
>   server_socket = /var/run/dovecot/auth-client
>   server_set_id = $auth1
> 
> 
> dovecot_plain:
>   driver = dovecot
>   public_name = PLAIN
>   server_socket = /var/run/dovecot/auth-client
>   server_set_id = $auth1
> 
> 
> Design your own exclusive Pingbox today! 
>  It's easy to create your personal chat space on your blogs
> 
> 
>   Connect instantly with more friends on your blog and personal website? 
> Create your latest Pingbox today! http://ph.messenger.yahoo.com/pingbox

Re: [Dovecot] exim4 dovecot sasl problem

[Nikita Koshikov]

On Fri, 17 Jul 2009 01:36:32 -0700 (PDT)
Tristan Rex Santiago  wrote:

> hi!
> 
> i'm trying to do smtp authentication using dovecot and exim4 by following 
> this guide http://wiki.dovecot.org/HowTo/EximAndDovecotSASL.
> 
> 
> for the exim4 part, i got an
> 
> authenticator dovecot: cannot find authenticator driver "dovecot"
>  error after i entered the codes in /etc/exim4/exim4.conf.template and 
> reloaded exim4 with the update-exim4.conf command.
> 
> i'm running debian lenny.

run exim -bV , search for string "Authenticators:" and check if "dovecot" is 
present there. If not, you need to recompile you exim with dovecot-sasl support.

> 
> here's the authentication section of exim4.conf.template:
> 
> begin authenticators
> 
> cram_md5:
> 
>   driver = cram_md5
> 
>   public_name = CRAM-MD5
> 
> 
>   client_name = ${extract{1}{:}{${lookup{$
> host}nwildlsearch{CONFDIR/passwd.client}{$value}fail}}}
>   client_secret = 
> ${extract{2}{:}{${lookup{$host}nwildlsearch{CONFDIR/passwd.client}{$value}fail}}}
> 
> PASSWDLINE=${sg{\
> 
>     
> ${lookup{$host}nwildlsearch{CONFDIR/passwd.client}{$value}fail}\
>         }\
>         {\\N[\\^]\\N}\
>         {^^}\
>         }
> 
> plain:
>   driver = plaintext
>   public_name = PLAIN
> 
> .ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
>   client_send = "<; ${if !eq{$tls_cipher}{}\
>     {^${extract{1}{:}{PASSWDLINE}}\
>          ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}\
>            }fail}"
> 
> .else
>   client_send = "<; ^${extract{1}{:}{PASSWDLINE}}\
>             ^${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
> .endif
> 
> login:
>   driver = plaintext
>   public_name = LOGIN
> .ifndef AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS
> 
>   # Return empty string if not non-TLS AND looking up $host in passwd-file
>   # yields a non-empty string; fail otherwise.
>   client_send = "<; ${if and{\
>   {!eq{$tls_cipher}{}}\
> 
>   {!eq{PASSWDLINE}{}}\
>  }\
>   {}fail}\
>  ; ${extract{1}{::}{PASSWDLINE}}\
>          ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
> 
> .else
>   # Return empty string if looking up $host in passwd-file yields a
>   # non-empty string; fail otherwise.
>   client_send = "<; ${if !eq{PASSWDLINE}{}\
>   {}fail}\
>  ; ${extract{1}{::}{PASSWDLINE}}\
> 
>          ; ${sg{PASSWDLINE}{\\N([^:]+:)(.*)\\N}{\\$2}}"
> .endif
> 
> dovecot_login:
>   driver = dovecot
>   public_name = LOGIN
>   server_socket = /var/run/dovecot/auth-client
>   server_set_id = $auth1
> 
> 
> dovecot_plain:
>   driver = dovecot
>   public_name = PLAIN
>   server_socket = /var/run/dovecot/auth-client
>   server_set_id = $auth1
> 
> 
> 
>   How can I fight insomnia? Discover how on Yahoo! Answers. 
> http://ph.answers.yahoo.com

Re: [Dovecot] dovecot v1.2.1 expire-tool with mail_log enabled

[Nikita Koshikov]

On Mon, 13 Jul 2009 20:28:33 +0200
e-frog  wrote:

> Nikita Koshikov wrote:
> > Hello list,
> > 
> > Sorry, if it has been asked, but searching throught list not give possitive 
> > answers.
> > 
> > After upgrade to dovecot version 1.2.1 expire-tool stoped working. I have 
> > changed database format and config file to appropriative 1.2 version but:
> > 
> > expire-tool.sh fails with:
> > # dovecot -c /etc/dovecot/dovecot.conf --exec-mail ext 
> > /usr/libexec/dovecot/expire-tool.sh
> > 
> 
> You are already using the wrapper for expire-tool ... to remove the 
> imap_quota plugin?
> 
> > Info: Loading modules from directory: /usr/lib/dovecot/imap
> > Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so
> > Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so
> > Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so
> > Info: Module loaded: /usr/lib/dovecot/imap/lib20_fts_plugin.so
> > Error: dlopen(/usr/lib/dovecot/imap/lib20_mail_log_plugin.so) failed: 
> > /usr/lib/dovecot/imap/lib20_mail_log_plugin.so: undefined symbol: 
> > imap_write_flags
> > Fatal: Couldn't load required plugins
> > 
> > When I diasabled mail_log plugin from imap section, everything start 
> > working, as expected and expire-tool delete old mail.
> > 
> 
> I think you also need to remove the mail_log plugin from the environment:
> 
> expire-tool.sh:
> ...
> MAIL_PLUGINS=${MAIL_PLUGINS//imap_quota/}
> MAIL_PLUGINS=${MAIL_PLUGINS//mail_log/}
> ...
> 

This helps, Thank you.

Now my expire-tool.sh looks like:
#!/bin/bash
MAIL_PLUGINS=${MAIL_PLUGINS//imap_quota/} 
MAIL_PLUGINS=${MAIL_PLUGINS//mail_log/}
exec ${0%.sh} "$@"

Maybe wiki should be updated to consider this trick ?

[Dovecot] dovecot v1.2.1 expire-tool with mail_log enabled

[Nikita Koshikov]

Hello list,

Sorry, if it has been asked, but searching throught list not give possitive 
answers.

After upgrade to dovecot version 1.2.1 expire-tool stoped working. I have 
changed database format and config file to appropriative 1.2 version but:

expire-tool.sh fails with:
# dovecot -c /etc/dovecot/dovecot.conf --exec-mail ext 
/usr/libexec/dovecot/expire-tool.sh

Info: Loading modules from directory: /usr/lib/dovecot/imap
Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so
Info: Module loaded: /usr/lib/dovecot/imap/lib20_fts_plugin.so
Error: dlopen(/usr/lib/dovecot/imap/lib20_mail_log_plugin.so) failed: 
/usr/lib/dovecot/imap/lib20_mail_log_plugin.so: undefined symbol: 
imap_write_flags
Fatal: Couldn't load required plugins

When I diasabled mail_log plugin from imap section, everything start working, 
as expected and expire-tool delete old mail.

dovecot -n gives:
# 1.2.1: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.11.1 
log_path: /var/log/dovecot/dovecot-error.log
info_log_path: /var/log/dovecot/dovecot.log
protocols: imaps managesieve
ssl_cert_file: /etc/ssl/dovecot/imaps.crt
ssl_key_file: /etc/ssl/dovecot/imaps.key
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable(default): /usr/libexec/dovecot/imap-login
login_executable(imap): /usr/libexec/dovecot/imap-login
login_executable(managesieve): /usr/libexec/dovecot/managesieve-login
login_greeting: Server ready.
login_processes_count: 10
login_max_processes_count: 512
first_valid_uid: 8
last_valid_uid: 8
first_valid_gid: 12
last_valid_gid: 12
mail_executable(default): /usr/libexec/dovecot/imap
mail_executable(imap): /usr/libexec/dovecot/imap
mail_executable(managesieve): /usr/libexec/dovecot/managesieve
mail_plugins(default): quota imap_quota trash fts fts_squat expire zlib mail_log
mail_plugins(imap): quota imap_quota trash fts fts_squat expire zlib mail_log
mail_plugins(managesieve): 
mail_plugin_dir(default): /usr/lib/dovecot/imap
mail_plugin_dir(imap): /usr/lib/dovecot/imap
mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve
auth default:
  mechanisms: plain login
  cache_size: 10240
  cache_negative_ttl: 0
  user: dovecot_auth
  master_user_separator: *
  worker_max_count: 50
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/master.pwd
master: yes
  passdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  passdb:
driver: ldap
args: /etc/dovecot/dovecot-ldap.conf
  userdb:
driver: prefetch
  userdb:
driver: ldap
args: /etc/dovecot/dovecot-userdb-ldap.conf
  userdb:
driver: passwd-file
args: /etc/dovecot/passdb/users.pwd
  socket:
type: listen
client:
  path: /var/run/dovecot/auth-client
  mode: 432
  user: mail
  group: dovecot_auth
master:
  path: /var/run/dovecot/auth-master
  mode: 384
  user: mail
  group: mail
plugin:
  quota_warning: storage=95%% /etc/dovecot/plugins/quota_warning.sh 95
  quota: maildir:Mailbox quota
  quota_rule: *:storage=500M
  quota_rule2: Trash:storage=10%%
  trash: /etc/dovecot/plugins/dovecot-trash.conf
  expire: Trash 1 Spam 30
  expire_dict: proxy::expire
  sieve: ~/.dovecot.sieve
  sieve_storage: ~/sieve
  sieve_extensions: +imapflags +notify
dict:
  expire: sqlite:/etc/dovecot/plugins/expire.conf

cat /etc/dovecot/plugins/expire.conf
#driver=sqlite
connect = /var/mail/expire.db

#v1.2
map {
pattern = shared/expire/$user/$mailbox
table = expires
value_field = expire_stamp

fields {
username = $user
mailbox = $mailbox
}
}

If you need any addition information please tell.