Re: [Dovecot] Assertion failed with imapc after upgrading Dovecot from 2.1.7 to 2.2.9
Hi !
I would like to jump to version 2.2.9 instead of 2.1.7 to avoid maybe
hundred of segfault by day but my problem with the assertion is always here.
Anyone has an idea to resolve it ?
Sylvain
2014-01-07 Sylvain
> Hi !
>
> I have an old Courier IMAP and in front of it, I have put a proxy cache
> with Dovecot/imapc.
>
> I use Debian Wheezy (stable) which package Dovecot in version 2.1.7.
> I have tested the upgrade to Debian Jessie (testing) which package Dovecot
> in version 2.2.9 but an assertion is thrown :
>
> dovecot: imap(xxx): Panic: file imapc-list.c: line 499
> (imapc_list_delete_unused_indexes): assertion failed: (strncmp(vname,
> fs_list->ns->prefix, fs_list->ns->prefix_len) == 0)
>
> I have checked source code and have seen that if *imapc_list_prefix* is
> not set, assertion will not be "walked". It's works but special inbox
> aren't detected correctly in email clients.
> If I understand the meaning of *vname* variable, it is because our
> Courier IMAP send us "INBOX" which is the value of my *imapc_list_prefix*and
> thus, assertion is thrown.
>
> Here some details of my tests :
>
> Courier IMAP :
>
> * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE
> THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION
> STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc.
> See COPYING for distribution information.
> a login xxx xxx
> a OK LOGIN Ok.
> a list "" "*"
> * LIST (\HasNoChildren) "." "INBOX.Drafts"
> * LIST (\HasNoChildren) "." "INBOX.Trash"
> * LIST (\HasNoChildren) "." "INBOX.test"
> * LIST (\HasNoChildren) "." "INBOX.Sent"
> * LIST (\HasNoChildren) "." "INBOX.Junk"
> * LIST (\Unmarked \HasChildren) "." "INBOX"
> a OK LIST completed
>
> Dovecot version 2.1.7 :
>
> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
> a login xxx xxx
> a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT
> CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC
> ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE]
> Logged in
> a list "" "*"
> * LIST (\HasChildren) "." "INBOX"
> * LIST (\HasNoChildren \Drafts) "." "INBOX.Drafts"
> * LIST (\HasNoChildren \Trash) "." "INBOX.Trash"
> * LIST (\HasNoChildren) "." "INBOX.test"
> * LIST (\HasNoChildren \Sent) "." "INBOX.Sent"
> * LIST (\HasNoChildren \Junk) "." "INBOX.Junk"
> a OK List completed.
>
> Dovecot version 2.2.9 :
>
> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
> a login xxx xxx
> a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN
> CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in
> a list "" "*"
> Connection closed by foreign host.
>
> And the dovecot configuration relative to the inbox :
>
> imapc_list_prefix = INBOX
> namespace inbox {
> inbox = yes
> separator = .
> prefix = INBOX.
> }
>
> Any help will be welcome :)
>
> Sylvain
>
Re: [Dovecot] Shared mailbox ACL
On 28/01/2014, at 6:37 PM, Steffen Kaiser wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On Fri, 24 Jan 2014, Alex Ferrara wrote:
>
>> I have been able to get the inbox of the shared mailbox to appear in
>> Thunderbird, but I would like to allow all subfolders to have the same ACLs.
>> Is there a way to do this without having an ACL entry for each folder in the
>> dovecot-acl file?
>
> No, currently you need one file per mailbox.
Thanks for the pointer Steffen. Since I am migrating from an established Cyrus
installation, I thought that I would be smart about this and make a little
script. I have made the script available at the bottom of this message if
anyone is interested.
>
>> I also cannot seem to create folders
>> under the shared inbox.
>
> If the accessing user has the "k"-permission, there might be problems with
> the unix file permissions. Latter logs errors. I think I remember something
> that you cannot give ACLs to the top most root of some mail storages, e.g. in
> Maildir Maildir/dovecot-acl applies to the INBOX and there is no file for the
> "/" or something like that. Can you create a new mailbox below, say, the
> INBOX?
This is still a problem for me. I can create folders on subfolders of the
shared folder, but not under the shared folder itself. I have set
"mail_shared_explicit_inbox = no" so the shared folder is the inbox.
If I change "mail_shared_explicit_inbox = yes" then I can create folders
underneath the inbox, but still not in the top level. Folders that are already
there from the migration work fine. The current configuration I have does not
have an inbox per shared mailbox, so I would prefer to replicate that if
possible.
I am a little unsure what to try next as I have turned on debug logging and
there is no entry in the logs when I unsuccessfully try to create a folder
under the top level.
dupacl.sh
#!/bin/bash
IFS_BAK=$IFS
IFS="
"
MAILROOT='/mnt/mail/mailboxes'
MAILUSER='vmail'
MAILGROUP='vmail'
SHAREDFILE='/etc/dovecot/shared-mailboxes'
function usage {
echo -e "Dovecot ACL tool\n"
echo -e "The purpose of this tool is to clone the ACL of the inbox to all
child folders\n"
echo -e "$0 [ mailbox | -all | -new ]"
exit 1
}
function cloneacl {
MAILBOX="$MAILROOT/$1/Maildir"
ACLFILE="$MAILBOX/dovecot-acl"
if [ -d "$MAILBOX" ]; then
echo "Working on mailbox $1..."
# Make sure we have an ACL file
if [ -f "$ACLFILE" ]; then
if [ "$2" == "-new" ]; then
echo "Found ACL file in mailbox $1. Skipping"
return
fi
else
echo "ACL file does not exist. Creating one"
echo "group=$1 lrwstipekxa" > "$ACLFILE"
chown $MAILUSER:$MAILGROUP "$ACLFILE"
fi
# Copy the ACL file to sub directories
for dir in `find $MAILBOX -type d -name ".*"`; do
cp -av $MAILBOX/dovecot-acl "$dir/"
done
# Remove the dovecot-acl-list to make the mailboxes appear
if [ -f "$ACLFILE-list" ]; then
rm "$ACLFILE-list"
fi
else
echo "Mailbox $1 does not exist"
exit 1
fi
}
if [ "$1" == "-all" -o "$1" == "-new" ]; then
SHARED=`grep shared/ "$SHAREDFILE" | cut -d / -f 4`
for mailbox in $SHARED; do
cloneacl $mailbox $1
done
elif [ $1 ];then
cloneacl $1 -single
else
usage
fi
IFS=$IFS_BAK
IFS_BAK=
Re: [Dovecot] Architecture for large Dovecot cluster
Just a guess, but I expect the difference is because NetApp de-dupes by checksumming blocks and mark whole blocks as duplicates if they have the same checksum. True, the start of the message is always at byte 0, but because of different header length per user for the same message (different mail address with different lengths) the body will never start at the same byte. Oh yes, that's right. I confused maildir format with sendmail queue files that separates header and body. There is still some similarlity for mass mailouts to the same mail domain: they will have almost identical headers +/- message IDs and a few bytes here and there, but as you say, SiS is the way to go for deduping bulky message attachments. Joseph Tam
Re: [Dovecot] recipient_delimiter -- feature request
Quoting Patrick Domack :
To make it work the current way, is simple.
I would love for it though, to lookup and see if an mailbox exists
with the delim first, then if it doesn't lookup just the username
part.
Quoting Andrew Ray :
Hi,
It would be nice if the recipient_delimiter configuration option
supported multiple recipient delimiters as Postfix now does.
e.g. now if I set: recipient_delimiter = +- in Postfix, then
test-extens...@sample.com and test+extens...@sample.com are both
recognized.
But if I set: recipient_delimiter = +- in Dovecot, neither of the
above are recognized and instead test+-extens...@sample.com is valid.
Well, presently this feature isn't that important to me, but I
imagine it could be a source of confusion for people who are
migrating from say qmail to postfix (as I am doing) and notice
Postfix's cool ability to support both types of delimiters and
assume Dovecot will as well (as I did).
Thanks,
Andrew Ray
--
Andrew Ray
+1 404-418-5443
+421 (0) 917 832 253
http://guns.freedomlives.net/
Well, to answer my own wishlist, this isn't by no means complete, as I
only use lmtp, so I didn't bother to patch lda, but logins already
worked good the way it was, so just needed lmtp to match with delim
first, then try again without it.
--- dovecot-2.2.10/src/lmtp/commands.c 2013-08-21 16:30:17.0 -0400
+++ dovecot-2.2.10/src/lmtp/commands.c 2014-01-28 20:18:12.303577376 -0500
@@ -387,7 +387,7 @@
return;
domain = strchr(address, '@');
- p = strstr(address, client->unexpanded_lda_set->recipient_delimiter);
+ p = strpbrk(address, client->unexpanded_lda_set->recipient_delimiter);
if (p != NULL && (domain == NULL || p < domain)) {
/* user+detail@domain */
*username_r = t_strdup_until(*username_r, p);
@@ -536,7 +536,7 @@
memset(&input, 0, sizeof(input));
input.module = input.service = "lmtp";
- input.username = username;
+ input.username = address;
input.local_ip = client->local_ip;
input.remote_ip = client->remote_ip;
input.local_port = client->local_port;
@@ -545,6 +545,16 @@
ret = mail_storage_service_lookup(storage_service, &input,
&rcpt.service_user, &error);
+if (ret<=0 && strlen(detail)>0) {
+ input.username = username;
+ ret = mail_storage_service_lookup(storage_service, &input,
+ &rcpt.service_user, &error);
+ } else if(ret>0) {
+ username=t_strdup(address);
+ if(strlen(detail)>0)
+ detail = "";
+ }
+
if (ret < 0) {
prefix = t_strdup_printf(ERRSTR_TEMP_USERDB_FAIL_PREFIX,
username);
Re: [Dovecot] post-login scripting failing with "denied access to user" v2.1.7
Thanks Daniel, That worked well. The default spam filter is now created properly. I have placed sieve filters in the parent directory of the Maildir, otherwise the .dovecot.sieve file is erroneously interpreted as a maildir by IMAP clients. The way I have used parent directories ($HOME/../) in the script results in a warning from manage sieve but I think it is harmless as the filter work fine: dovecot: managesieve(t...@fqdn.net): Warning: sieve-storage: Active sieve script symlink /home/vmail/fqdn.net/test/Maildir/../.dovecot.sieve is broken: invalid/unknown path to storage (points to /home/vmail/fqdn.net/test/Maildir/../sieve/managesieve.sieve). Is there a way to suppress this warning? Frank On 28/01/2014, at 4:10 AM, Daniel Parthey wrote: > Hi Frank, > > Am 27.01.2014 07:57, schrieb Frank: >> After adding this setting, login fails and gives the error below. >> It certainly looks like a permission issue, but I have checked my script is >> executable and owned by vmail:vmail. The bash script executes fine as the >> mail user. >> The other possibility is that dovecot is denying access to the $HOME >> variable, but that seems unlikely. >> Nevertheless, I am stuck! >> >> Log shows: >> -- >> dovecot: master: Dovecot v2.1.7 starting up (core dumps disabled) >> dovecot: auth-worker(17522): mysql(database1): Connected to database >> postfixadmin >> dovecot: imap-login: Login: user=, method=PLAIN, >> rip=10.0.3.120, lip=10.0.1.180, mpid=17524, TLS, session=<+blqxM> >> dovecot: imap(te...@fqdn.net): Post-login script denied access to user >> te...@fqdn.net > > http://wiki2.dovecot.org/PostLoginScripting > > You need to hand over the connection to the original imap handler, > your script may not simple exit, but needs to exec its argument line: > > exec "$@" > > Kind regards > Daniel > > -- > Dipl.-Inf. Daniel Parthey > System Engineer > Metaways Infosystems GmbH > Pickhuben 2, D-20457 Hamburg > > E-Mail: d.part...@metaways.de > Web:http://www.metaways.de > > Metaways Infosystems GmbH - Sitz: D-22967 Tremsbüttel > Handelsregister: Amtsgericht Lübeck HRB 4508 AH > Geschäftsführung: Hermann Thaele, Lüder-H.Thaele
Re: [Dovecot] recipient_delimiter -- feature request
To make it work the current way, is simple.
--- a/src/lmtp/commands.c 2013-02-05 18:31:36.0 -0500
+++ b/src/lmtp/commands.c 2014-01-28 18:15:01.011677816 -0500
@@ -387,7 +387,7 @@
return;
domain = strchr(address, '@');
- p = strstr(address, client->unexpanded_lda_set->recipient_delimiter);
+ p = strpbrk(address, client->unexpanded_lda_set->recipient_delimiter);
if (p != NULL && (domain == NULL || p < domain)) {
/* user+detail@domain */
*username_r = t_strdup_until(*username_r, p);
I would love for it though, to lookup and see if an mailbox exists
with the delim first, then if it doesn't lookup just the username part.
Quoting Andrew Ray :
Hi,
It would be nice if the recipient_delimiter configuration option
supported multiple recipient delimiters as Postfix now does.
e.g. now if I set: recipient_delimiter = +- in Postfix, then
test-extens...@sample.com and test+extens...@sample.com are both
recognized.
But if I set: recipient_delimiter = +- in Dovecot, neither of the
above are recognized and instead test+-extens...@sample.com is valid.
Well, presently this feature isn't that important to me, but I
imagine it could be a source of confusion for people who are
migrating from say qmail to postfix (as I am doing) and notice
Postfix's cool ability to support both types of delimiters and
assume Dovecot will as well (as I did).
Thanks,
Andrew Ray
--
Andrew Ray
+1 404-418-5443
+421 (0) 917 832 253
http://guns.freedomlives.net/
[Dovecot] recipient_delimiter -- feature request
Hi, It would be nice if the recipient_delimiter configuration option supported multiple recipient delimiters as Postfix now does. e.g. now if I set: recipient_delimiter = +- in Postfix, then test-extens...@sample.com and test+extens...@sample.com are both recognized. But if I set: recipient_delimiter = +- in Dovecot, neither of the above are recognized and instead test+-extens...@sample.com is valid. Well, presently this feature isn't that important to me, but I imagine it could be a source of confusion for people who are migrating from say qmail to postfix (as I am doing) and notice Postfix's cool ability to support both types of delimiters and assume Dovecot will as well (as I did). Thanks, Andrew Ray -- Andrew Ray +1 404-418-5443 +421 (0) 917 832 253 http://guns.freedomlives.net/
[Dovecot] Problem to setup replication
Hi,
i tried to setup replication since some days but I'am stuck now.
My conf:
# 2.2.10: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.3.1.el6.x86_64 x86_64 CentOS release 6.5 (Final)
dsync_remote_cmd = ssh -l%{login} %{host} /usr/bin/doveadm dsync-server -u%u
-l%{lock_timeout} -n%{namespace}
log_path = /var/log/dovecot
mail_gid = 5000
mail_home = /var/vmail/%d/%n
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
mail_uid = 5000
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
mbox_write_locks = fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
mail_replica = remote:vm...@server1.domain.dom
replication_full_sync_interval = 1 hours
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = imap sieve
service aggregator {
fifo_listener replication-notify-fifo {
user = vmail
}
unix_listener replication-notify {
user = vmail
}
}
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666
user = postfix
}
unix_listener auth-master {
mode = 0600
user = vmail
}
unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
}
}
service managesieve-login {
inet_listener sieve_deprecated {
port = 2000
}
}
service replicator {
process_min_avail = 1
unix_listener replicator-doveadm {
mode = 0600
user = vmail
}
}
ssl = required
ssl_cert =
Re: [Dovecot] Auth socket can't listen using ssl
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 28 Jan 2014, Anthony Bourguignon wrote: Le Monday 27 January 2014 à 12:08 -0500, Steve Litt a écrit : I read in some docs somewhere that Dovecot automatically trusts anything on localhost. If you're telnetting into it from the same physical computer that hosts the port, try telnetting into it from a different physical computer with a different IP address and see if you can still telnet in. I've tried from localhost and another computer. In both tries, the connection is made without ssl. Hmm, maybe "internal" sockets do not utilize SSL at all? Just IMAP/POP/ManageSieve? - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUueGOHD1/YhP6VMHAQJNNgf/YhJvRaC5OgqCE2TJzEcy6WvFDB7bKIdU +8tVn66o1p5cAQYXZWb917otLnuejrO8RVLe5fAOb06Olo1eIbSbhJZv4JISS3OW adYZ468TdYT1Qdjmwbzo0lchZPlA2JHoRMi4EfxFcJN6rUte5XdkSds92ZhOIUvK /yAaBhsXYmF84n/24OqwT65b+IY2uJTggZR80tW+RhzAlUIR0D44oOCw8d7k2w3Y u545oolzHQiUvT1NLNedzjpneTfuV9ZukK9c/W/9mgUkXmNBykhzRsLbsJcOoOwP re59kSzgVssNlsTEFimUajU/RvoUs591AY+LVwk3mPBDQ6iJTDRwng== =tKyc -END PGP SIGNATURE-
Re: [Dovecot] Architecture for large Dovecot cluster
Joseph Tam wrote: > Sven Hartge wrote: >> Interesting datapoint: NetApp Deduplication did only recover about 1% >> of storage space with mdbox-based mail storage, while on an >> maildir-based mail storage, the rate was about 15%. (This was tested >> with a copy of real user data, so is accurate for my workload.) > Just a guess, but I expect the difference is because NetApp de-dupes > by checksumming blocks and mark whole blocks as duplicates if they > have the same checksum. > The message body has the same block offset in maildir (i.e. the start > of a message is at byte 0), whereas mdbox might align message body > anywhere in a block, so you might have 512 different block > configurations for the same message. True, the start of the message is always at byte 0, but because of different header length per user for the same message (different mail address with different lengths) the body will never start at the same byte. In the end, a slight compression (gzip 3) via Dovecot resulted in better space savings than compression and deduplication via NetApp. The most space can obviously saved via SiS of attachements in dovecot, but to be frank, this feature scares me a bit. Grüße, Sven. -- Sigmentation fault. Core dumped.
Re: [Dovecot] Auth socket can't listen using ssl
Le Monday 27 January 2014 à 12:08 -0500, Steve Litt a écrit : > I read in some docs somewhere that Dovecot automatically > trusts anything on localhost. If you're telnetting into it from the same > physical computer that hosts the port, try telnetting into it from a > different physical computer with a different IP address and see if you > can still telnet in. I've tried from localhost and another computer. In both tries, the connection is made without ssl. But thanks for the tip
