Re: Headsup on feature removal
-- Original Message -- [...] To start, the following features are likely to be removed in next few releases of Dovecot. [...] - mailbox alias plugin Like autocreate, autosubscribe, and expire - Is there a built-in feature that makes this plugin obsolete? --- Daniel
Re[2]: Namespace problem? duplicated folders...
Try this:
namespace inbox {
hidden = no
inbox = yes
list = yes
mailbox Drafts {
auto = no
special_use = \Drafts
}
mailbox Junk {
auto = no
special_use = \Junk
}
mailbox Sent {
auto = no
special_use = \Sent
}
mailbox "Sent Messages" {
auto = no
special_use = \Sent
}
mailbox Trash {
auto = no
special_use = \Trash
}
prefix = INBOX/
separator = /
subscriptions = no
type = private
}
namespace subscriptions {
hidden = yes
list = no
subscriptions = yes
type = private
}
---
Daniel
-- Original Message --
From: "Gregory Sloop"
To: "Dovecot Mailing List"
Sent: 1/23/2020 12:46:54 PM
Subject: Re: Namespace problem? duplicated folders...
Anyone?
A tip on where/what to look at, even?
I've read the docs on namespaces and I'm not sure how I could do this
[or not do it, in this case] from the docs.
I realize I didn't include the version I'm running in the prior post
too.
It is; 2.2.22
I'm sure this is related to name-spaces, but for some reason [brain
damage perhaps?
:)
] I can't seem to figure out how to fix it.
In TBird, for example, I have a folder tree that looks like this
Inbox -|
Folder-A
Folder-B
Folder-C
Folder-A
Folder-B
Folder-C
And Folder-A, Folder-B, Folder-C are the same folders, just shown
twice. [In two different hierarchies.]
I'm using mbox files to store mail. [If that matters.]
Here's what I have in my conf files for namespace defs
namespace {
inbox = yes
#hidden = yes
prefix = INBOX/
separator = /
}
namespace inbox {
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Trash {
special_use = \Trash
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
}
I've had this problem for literally years, and it's not been that big
of a deal (mostly irritating) - and I've tried to "fix" it before
without any success. But now it's causing some issues and I'd really
like to get this nagging issue gone. Since it's been so long since I
last tried to fix it, I really can't recall any of settings changes I
made in an attempt to fix it. I'd guess this is a trivial issue, but if
so, can someone point me in the right direction.
Thanks in advance!
-Greg
--
Gregory Sloop, Principal: Sloop Network & Computer Consulting
Voice: 503.251.0452 x121
EMail: gr...@sloop.net
http://www.sloop.net
---
Re: Help with migration to Dovecot from raw files
On 2020-03-28 23:54, Stefano Antonelli wrote:
I've attached my script in case it's helpful to anyone in a similar
situation. It took quite a bit of effort to get this right, highly
depends on the format of the original mailstore, and I've only tested
this on one smaller mailbox (<400 messages). And I wasn't concerned
with any client having to download everything again since this was a
fresh import.
In any case, it worked for me.
Err had a few issues on a larger 4gb mailbox:
1. small bug in the script, new script attached. Change:
# Grab the flags from the filename.
F=`echo $fname | cut -c 26-27`
to:
# Grab the flags from the filename.
F=0x`echo $fname | cut -c 26-27`
without the 0x in front, the masking operation interprets $F as octal.
2. there are cases where the old mail server wrote files with identical
timestamps even though the dates of the email are clearly different. I
presume this was just buggy behaviour and could explain why I couldn't
successfully download via imap.
201011072323050332-0106028C3D2E9D97.imap
1289200985.M00P12345.icewarp,S=5614,W=5614:2,RS
Received: Tue, 14 Sep 2010 19:44:47 -0700
201011072323050330-0106028A3D2C9676.imap
1289200985.M00P12345.icewarp,S=2317,W=2317:2,RS
Received: Sun, 12 Sep 2010 18:51:44 -0700
I may consider fixing this.
3. for some reason there were some off by one file sizes that resulted
in errors like this:
dovecot.index.cache: UID 278: Broken physical size in mailbox ...
failed: Cached message size smaller than expected (1972 < 1973
,box=INBOX, UID=278)
In my case 298 out of 22023 files were affected. Script to fix this is
after using the import-from-icewarp-files.sh script is attached. I
couldn't find a good way to find only maildir messages by filename.
Seems difficult to parse. When I renamed the files, I used icewarp as
the server name and that's how I searched for files:
FILES=`find /path/to/mail/user -name \*\.icewarp\,\*`
If there's a better way to do this, I would like to know for future
reference.
Thanks,
Stef
#!/bin/bash
icnt=0
cnt=0
# I don't know of a good way to find only maildir mail files.
FILES=`find /path/to/mail/user -name \*\.icewarp\,\*`
for f in $FILES; do
fname=${f##*/}
path=${f%/*}
# echo "in : $fname"
let "icnt+=1"
# Split the filename into string parts
# for reassembly later and to get the size
remainder=$fname
part1="${remainder%%=*}"; remainder="${remainder#*=}"
size="${remainder%%,*}"; remainder="${remainder#*,}"
remainder="${remainder#*=}"
wsize="${remainder%%:*}"; remainder="${remainder#*:}"
rsize=`stat -c%s $f`
# Don't continue processing if rsize is empty or if size is ok.
[[ -z "$rsize" ]] && continue
[[ "$size" == "$rsize" ]] && continue
echo "in : $fname"
# echo "sizes: S=$size,W=$wsize real: $rsize"
out="$part1=$rsize,W=$rsize:$remainder"
echo "out : $out"
# Check operation before doing the move.
echo mv $path/$fname $path/$out
# mv $path/$fname $path/$out
let "cnt+=1"
done
echo $cnt of $icnt files affected.
#!/bin/bash
# This Script
# !! DOES NOT HANDLE SPACES IN FILE OR DIRECTORY NAMES !!
#
# You will need to go to the mailstore path and change any folder
# names with spaces to something without spaces. Leave "inbox"
# as is; that's an icewarp default mailbox. This script was
# not tested with any spaces or special characters
# (ie '.', '-', or '_') in the imap folder name.
FILES=`find /path/to/icewarp/mailstore -name \*.imap`
SCRAP=$PWD/scrap
ODIR=$PWD/output
PID=12345
SNAME=icewarp
# Icewarp filename specifics:
# 200612171259060055-0102000535916763.imap
# TT-ZZFF.imap
# 35916763 (first line of email)
# - year, month, day
# TT - hour, minute, seconds
# - unknown
# Z- always seems to be 1
# FF - flags
# - appears to be a sequence number
# - appears to be the Message-Id in hex.
#
# Icewarp flags:
# * OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Draft \Deleted)]
# Not documented. Discovered through querying the icewarp server
# and observing returned flags vs filename stored on disk. The flags
# are likely bitmasks of bytes 26 and 27 of the filename:
SEEN=0x02 # 0010 /Seen
ANSD=0x04 # 0100 /Answered
FLGD=0x08 # 1000 /Flagged
DRFT=0x10 # 0001 /Draft
DLTD=0x40 # 0100 /Deleted or (no flags)
# Dovecot filename specifics:
# ,S=: contains the file size.
# ,W=: contains the file's RFC822.SIZE,
# ie. the file size with linefeeds being CR+LF characters.
# If the message was stored with CR+LF linefeeds,
# and are the same.
#
# Dovecot flags:
#Flag "P" (passed): the user has resent/forwarded/bounced
# this message to someone else.
#Flag "R" (replied): the user has replied to this message.
#Flag
Re: mail_crypt crashing randomly during LMTP
On 29. Mar 2020, at 15.15, Andi Krischer wrote: > We have a really strange bug with our dovecot setup. After weeks and month of > debugging we’re finally reached the point when we ask you for help. > > Our basic setup is quite simple: a single dovecot 2.3.10 (0da0eff44) install > on Debian 10 with LDAP as user/passdb and sieve, imap_sieve and mail_crypt as > mail_plugins. Was this happening with old Dovecot versions, or did it start after v2.3.10 upgrade? > This is what the log says in the moment of crashing: > dovecot: lmtp(53852): Panic: Module context mail_crypt_user_module missing > *** backtrace *** (see: https://pastebin.com/YCiFtxmy) It shows mailbox_free() in the backtrace, so it's probably crashing in mail_crypt_mailbox_close()'s MAIL_CRYPT_CONTEXT(box). But looking at the code, I can't see how that could be possible. Maybe there is some kind of memory corruption or something. > dovecot: lmtp(53852): Fatal: master: service(lmtp): child 53852 killed with > signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Could you get core dumps enabled? See the link. And from the core dump send at least: bt full fr 6 (or maybe 5 or 7, whichever makes the following command work) p *box Also please send your doveconf -n output.
mail_crypt crashing randomly during LMTP
Hello everybody,
hope you’re all well!
We have a really strange bug with our dovecot setup. After weeks and month of
debugging we’re finally reached the point when we ask you for help.
Our basic setup is quite simple: a single dovecot 2.3.10 (0da0eff44) install on
Debian 10 with LDAP as user/passdb and sieve, imap_sieve and mail_crypt as
mail_plugins.
For the mail_crypt we have a special rule which allows us to enable crypting
for each user individually.
See this line from dovecot-ldap.conf.ext:
pass_attrs =
userPassword=password,=userdb_mail_plugins=%{if;%{real_lport};eq;993;imap_sieve;}
%{if;%{real_lport};eq;143;imap_sieve;} notify mail_log
%{if;%{ldap:encryptMail};eq;TRUE;mail_crypt;
},=userdb_mail_crypt_private_password=%w
As you can see for IMAP connections we enable imap_sieve and if the LDAP
attribute “encryptMail” is set to TRUE we enable “mail_crypt” as mail_plugin.
This works fine except that around 5-10 times a day the dovecot server crashes
during a LMTP connection. It crashes in the last steps of LMTP which result
that the mail get delivered to Maildir but Postfix doesn’t get a positive
response and delivers the mail again after a few minutes. So all of our users
get some mails twice or even more times.
This is what the log says in the moment of crashing:
dovecot: lmtp(53852): Panic: Module context mail_crypt_user_module missing
*** backtrace *** (see: https://pastebin.com/YCiFtxmy)
dovecot: lmtp(53852): Fatal: master: service(lmtp): child 53852 killed with
signal 6 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps)
Here I’ve pasted the LTMP session in mail.log: https://pastebin.com/A4ughNUF
Obviously it looks like it has something to do with our mail_crypt setup. But
the thing is that although we’re monitoring this for months we cannot find a
condition which cause this. It seems to happen more frequent for mails with lot
of people in To or CC (large headers), but definitively not all of them…
Some more things we’ve already thought on:
First my guess was that the mail_crypt plugin isn’t loaded in some case but in
the backtrace are things from “lib10_mail_crypt_plugin.so”, so it should be
loaded, right?
On the other hand a non-crashing LTMP session looks exactly the same in log
except it has three additional lines in mail.log:
Debug: Loading modules from directory: /usr/lib/dovecot/modules
Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so
My college already had a look in the source code, but this didn’t make us much
smarter. Here his references:
mail context is defined here:
https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac612b/src/plugins/mail-crypt/mail-crypt-plugin.c#L453
and removed again here:
https://github.com/dovecot/core/blob/81b5b188c478ec36bea8bda8fcad1e5f32ac612b/src/plugins/mail-crypt/mail-crypt-plugin.c#L397
Does any of the experts on this list have an idea what could cause these
crashes and how to fix them?
Thanks in advance!
Best,
Andi
