Re: [Dovecot] Proxying, pertinent values and features, SNI
On Thu, 4 Apr 2013 22:21:43 +0300 Timo Sirainen wrote:
> On 3.4.2013, at 10.59, Christian Balzer wrote:
>
> > I'm looking into deploying dovecot as a proxy, currently using
> > perdition. Have been using dovecot on the actual servers for years,
> > nearly a decade. So far just 1.x, but for the proxy it will have to be
> > 2.x (2.1.7 is the current Debian version), as the trigger for this
> > change is the need to support multiple SSL certificates.
> >
> > All that happens on the proxy seems to be handled by the login
> > processes, so that is why we're not seeing anything useful in the
> > process titles or with doveadm, right?
> > And from past comments by Timo I guess that adding such functionality
> > isn't on his to-do list at all.
>
> doveadm proxy list
>
That will teach me to look at man pages. ^o^
Internal help all the way, man pages are for chums. ^o^
Thanks!
> > A configurable capabilities string for POP would be quite welcome, but
> > at least nothing is different between the 1.x backends and the 2.x
> > proxy in that protocol.
>
> v2.2 backends actually add some new POP3 capabilities. I guess there
> could be such a setting, although it's a bit annoying to develop..
>
I guess so, but that will really make it an universally deployable proxy
and help people transitioning to dovecot from other environments, too.
[snip]
>
> > I presume to best support all(?) clients out there is to have
> > "local_name" sections for SNI first and then "local" sections for IP
> > address based certs. It is my understanding that SNI needs to be
> > requested by the client, so aside from client bugs (nah, those don't
> > exist ^o^) every client should get an appropriate response for TLS.
> > Has anybody done a setup like that already?
>
> If you have separate IPs for each sertificate, you don't need to
> support/configure SNI, so local {} blocks are enough.
>
I know that, the idea was/is to determine how many (connects and clients)
do a proper TLS/SNI negotiation if offered.
However are these even differently logged by dovecot? I suspect not.
Regards,
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Fusion Communications
http://www.gol.com/
Re: [Dovecot] Sync errors trying to sync old mbox inboxes to dovecot
Timo,
I tried importing the anonymized file, and there were no errors. So, it's
something with the actual data. UTF-8 issues?
While I can't send you a raw customer's mailbox, one of the failures is a
'role' account mailbox, with no critical/useful data in it.
Here is a link (expires in 30 days) to download the mailbox, it's 107M.
https://truck.it/p/HNlzl9hWbO
The error I'm getting is
dsync(webmail): Error: Sync failed for mbox file
/var/opt/migrate/users/webmail/inbox: seq=4482 uid=4482 uid_broken=0 originally
needed 10 bytes, now needs 12 bytes
The seq and uid are different, depending on the mailbox being migrated. Some
of the problem mailboxes only report a single failure, and some report multiple
failures (where the seq/uid are different for each failure). For a given
mailbox, the seq and uid reported seem to always be consistent.
I create a directory called migrate, copy the file in there and name it inbox
The command I'm using is
Dsync -u webmail mirror mbox:migrate/
Here's my 2.2.rc3 doveconf -n output
# 2.2.rc3: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)
auth_debug = yes
auth_debug_passwords = yes
auth_default_realm =
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/dovecot-debug.log
disable_plaintext_auth = no
dotlock_use_excl = no
lda_mailbox_autocreate = yes
lock_method = dotlock
log_path = /var/log/dovecot.log
mail_access_groups = mail
mail_debug = yes
mail_fsync = always
mail_location = mdbox:~/mail:INDEX=~/index
mail_plugins = quota zlib
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date
mbox_read_locks = dotlock
mbox_write_locks = dotlock
mdbox_rotate_interval = 1 weeks
mdbox_rotate_size = 20 M
mmap_disable = yes
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
autocreate = Trash
autocreate2 = Junk
autocreate3 = Drafts
autocreate4 = Sent
autocreate5 = INBOX
autosubscribe = Trash
autosubscribe2 = Junk
autosubscribe3 = Drafts
autosubscribe4 = Sent
autosubscribe5 = INBOX
mail_max_userip_connections = 10
managesieve_max_line_length = 65536
quota = dict:User quota::file:%h/dovecot.quota
quota2_rule = Trash:storage=+10%%
quota3_rule = Junk:storage=+20%%
quota_rule = *:storage=100M:messages=10
recipient_delimiter = +
sieve_before = /var/spool/mail/global.sieve/
zlib_save = gz
zlib_save_level = 9
}
pop3_reuse_xuidl = yes
protocols = imap pop3 sieve lmtp
service auth {
inet_listener auth {
port = 113
}
unix_listener auth-userdb {
user = nobody
}
}
service lmtp {
inet_listener lmtp {
port = 24
}
process_min_avail = 10
service_count = 1
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
process_min_avail = 0
service_count = 1
vsz_limit = 64 M
}
ssl = no
ssl_cert = mailto:t...@iki.fi]
Sent: Sunday, April 07, 2013 4:36 PM
To: Michael Smith
Cc: dovecot@dovecot.org
Subject: Re: [Dovecot] Sync errors trying to sync old mbox inboxes to dovecot
On 6.4.2013, at 0.10, Michael Smith wrote:
> I am trying to 'dsync mirror' old Qpopper/mbox inboxes to our new
> Dovecot/mdbox servers. For most of the several thousand accounts, it works
> without issue. But there are a handful of accounts that are generating the
> following errors.
>
> dsync(joe): Error: Sync failed for mbox file
> /var/opt/migrate/users/joe/inbox: seq=981 uid=981 uid_broken=0 originally
> needed 9 bytes, now needs 11 bytes
>
> dsync(john): Error: Sync failed for mbox file
> /var/opt/migrate/users/john/inbox: seq=115028 uid=115028 uid_broken=0
> originally needed 34 bytes, now needs 35 bytes
These are some generic mbox bugs. I think the error goes away if you just
repeat the run?
> dsync(john): Error: Sync failed for mbox file
> /var/opt/migrate/users/john/inbox: seq=114779 uid=114779 uid_broken=0
> originally needed 34 bytes, now needs 35 bytes
> dsync(john): Error: Sync failed for mbox file
> /var/opt/migrate/users/john/inbox: seq=114778 uid=114778 uid_broken=0
> originally needed 34 bytes, now needs 35 bytes
.. or are these all separate dsync runs giving different errors?
Anyway, if you can send me an mbox file where I can reproduce this, I could fix
it. You can use http://dovecot.org/tools/mbox-anonymize.pl script to replace
all the actual data in the mbox file with 'x' chars.
Re: [Dovecot] make check error for Pigeonhole v0.3.4 for Dovecot v2.1.16
On 4/8/2013 8:16 PM, Giam Teck Choon wrote: Hi, When I compile Pigeonhole v0.3.4 with Dovecot v2.1.16 until make check output as below from my rpmbuild output: Test case: ./tests/extensions/mailbox/execute.svtest: 1: Test 'MailboxExists - None exist' SUCCEEDED 2: Test 'MailboxExists - Not all exist' SUCCEEDED 3: Test 'MailboxExists - One exists' SUCCEEDED 4: Test 'MailboxExists - All exist' SUCCEEDED 5: Test ':Create' SUCCEEDED PASS: 5 tests succeeded. Test case: ./tests/extensions/date/basic.svtest: 1: Test 'Defaults' SUCCEEDED 2: Test 'Count' SUCCEEDED 3: Test 'Invalid' SUCCEEDED 4: Test 'Comparison' FAILED: date is invalid: 2009-07-23 FAIL: 1 of 4 tests failed. RPM build errors: make: *** [tests/extensions/date/basic.svtest] Error 1 error: Bad exit status from /var/tmp/rpm-tmp.CIcpqx (%check) Bad exit status from /var/tmp/rpm-tmp.CIcpqx (%check) You're not the first one to notice: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/bc2126771d00 This is one of those genuine phase-of-the-moon kind of bugs :) Hmm, maybe I should make a new release for this problem already. Regards, Stephan.
Re: [Dovecot] LMTP Setup/Design Help Request
On 08-04-2013 22:39, Matthias Hofer wrote:
Hi altogether,
I am switching from dbmail 2.2 to dovecot 2.1.7, and I was able to do
a successful transition of IMAP. But I
had a very hard time with receiving mail, and in the end -- after
loads of reading and debugging -- I broke
down and failed. This is why I am asking you for your kind help.
I only have virtual users and have set
mail_uid = dovemail
mail_gid = dovemail
For authentication, I simply use (for now) a flat file:
passdb {
driver = passwd-file
args = scheme=PLAIN username_format=%u /etc/dovecot/mypasswd.file
}
# cat /etc/dovecot/mypasswd.file
user1:{PLAIN}userpw1
user2:{PLAIN}userpw2
user3:{PLAIN}userpw3
And everything works fine with IMAP. I am completely domain-agnostic
here.
But I was not able to use this fine setup to receive mails with my
postfix. I tried about 13 different
configurations, some with considerable changes dovecot-config, the
passwd-file or postfix, but I could not
find a single variant working.
In this situation, I would be very happy if somebody could give me a
hint, which would be the prudent way to
configure dovecot, LDA/LMTP and postfix based on the setup above.
Thank you very much in advance, and have a great time,
Matthias Hofer
I suppose your postfix is in working state. Note that you do not need
dovecot
to deliver mail with postfix. But it is a good idea. To deliver using
dovecot-lda
my postfix main.cf contains
mail_spool_directory = /Mail/
mailbox_command = /usr/local/libexec/dovecot/dovecot-lda -f "$SENDER"
-a "$RECIPIENT"
Re: [Dovecot] repeating dsync - questions
OK, success: see timings (real nn) below commands. Initial copy of Maildir from live system to test sys (14G of data) rsync -ar --times hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir real 37m Then 1st 'dsync -R backup maildir:~/Maildir' real 828m Then 2nd rsync to pick up new mail - **don't touch existing files** rsync -ar --times --ignore-existing hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir real 3m Then 2nd 'dsync -R backup maildir:~/Maildir' real 12m The --ignore-existing option on the 2nd rsync allows dsync to process the additional emails in a reasonable amount of time. The dovecot-uidlist which is modified in the Maildir by dsync is not written over by the 2nd rsync and therefore the 2nd dsync just processes the added messages. (There were no deletes between rsync runs) Thanks much for your hints and comments. Bob G On Mon, 2013-04-08 at 00:53 +0300, Timo Sirainen wrote: > On 8.4.2013, at 0.10, Bob Gustafson wrote: > > >>> I am still on my quest for a quick way to move mail from a live Maildir > >>> system to a 'soon to be live' sdbox system. > >>> > >>> I copy Maildir to new system using: > >>> rsync -ar --times hoho4:/home/bobgus/Maildir/ /home/bobgus/Maildir > >>> > >>> Then I convert from Maildir to sdbox with: > >>> dsync mirror maildir:~/Maildir > >>> > >>> Then I copy more messages from live system using rsync > >> > >> ^^ that is the mistake > > > > I guess I have a basic misunderstanding of what 'dsync mirror' is doing. > > > > My understanding is that going from Maildir to sdbox, dsync does not > > mess with the data in Maildir. The Maildir metadata is in one form and > > the sdbox metadata is in another form (in the sdbox directory). > > dsync does mess with metadata in the maildir. also with dsync mirror (as > opposed to dsync backup) it can also modify the contents. The main problem > here is: > > 1. dsync sees that a folder A in maildir doesn't have a GUID (because dsync > is just about the only tool that uses it right now), and assigns the mailbox > a new GUID > 2. dsync syncs the mailbox to sdbox with that GUID > 3. rsync comes and wipes out the maildir-uidlist that contained the GUID > 4. second dsync sees that folder A in maildir doesn't have a GUID, and > assigns a new GUID to it > - now maildir has folder A with GUID 1, and sdbox has folder A with GUID 2 > - dsync thinks they are two different folders, and duplicates them as A and > A_2. the A_2 gets also copied back to maildir, because you're using dsync > mirror. This is why the second dsync is slow, it's actually doing all the > work again, and actually doing twice the work since it's copying the mails > from sdbox to maildir as well. > > v2.2 dsync is somewhat smarter and can figure out that they are actually the > same folder A and it simply changes the other's GUID instead of duplicating > all data. > > > No new email messages enter the sdbox system to be 'mirrored' to the > > Maildir system. > > > > I thought of using the 'dsync backup' command, but the sentence "Any > > changes done in destination are discarded." seems to indicate that each > > time 'dsync backup' is done, it starts from the beginning. No > > incremental backup (but this is done in 2.2 ?) > > dsync backup is incremental. it just wipes out any changes done at the other > side (if there happens to be any). > > >>> Then I do the 'dsync mirror maildir:~/Maildir' again > >>> > >>> There were only a few messages that were copied over in the 2nd rsync > >>> pass and it went quickly, but the 2nd dsync pass is taking a long time. > >> > >> The second rsync is overwriting all the metadata changes (mailbox GUIDs > >> most importantly) that the first dsync run did. > > > > Why does dsync mess with the Maildir metadata? Won't that just confuse > > the dovecot running on the Maildir system? > > Incremental dsync doesn't work (well) without additional metadata.
[Dovecot] LMTP Setup/Design Help Request
Hi altogether,
I am switching from dbmail 2.2 to dovecot 2.1.7, and I was able to do a
successful transition of IMAP. But I
had a very hard time with receiving mail, and in the end -- after loads of
reading and debugging -- I broke
down and failed. This is why I am asking you for your kind help.
I only have virtual users and have set
mail_uid = dovemail
mail_gid = dovemail
For authentication, I simply use (for now) a flat file:
passdb {
driver = passwd-file
args = scheme=PLAIN username_format=%u /etc/dovecot/mypasswd.file
}
# cat /etc/dovecot/mypasswd.file
user1:{PLAIN}userpw1
user2:{PLAIN}userpw2
user3:{PLAIN}userpw3
And everything works fine with IMAP. I am completely domain-agnostic here.
But I was not able to use this fine setup to receive mails with my postfix. I
tried about 13 different
configurations, some with considerable changes dovecot-config, the passwd-file
or postfix, but I could not
find a single variant working.
In this situation, I would be very happy if somebody could give me a hint,
which would be the prudent way to
configure dovecot, LDA/LMTP and postfix based on the setup above.
Thank you very much in advance, and have a great time,
Matthias Hofer
Re: [Dovecot] crash with dovecot 2.2: Panic: Buffer full
Le 07/04/2013 22:56, Timo Sirainen a écrit :
> On Sun, 2013-04-07 at 21:47 +0200, to...@starbridge.org wrote:
>> since I've upgraded to 2.2.rc5 (c24783148d78), I've got the previous
>> error again:
>> It's happening on a large mailbox on almost every emails, with Horde 5
>> webmail client (works fine with thunderbird so far)
>> Thanks for your help
>>
>> Apr 7 21:30:58 mx2 dovecot: imap(clean-quarant...@spamguard.fr): Panic:
>> Buffer full (8206 > 8192, pool )
> Thanks, this should fix it:
> http://hg.dovecot.org/dovecot-2.2/rev/bb25fa35d887
>
>
Hi Timo,
It's ok for this error, but now, on a few email, I've got this:
dovecot --version
2.2.rc6 (548e59794f2e)
imap(clean-quarant...@spamguard.fr): Panic: file istream.c: line 153
(i_stream_read): assertion failed: (_stream->skip != _stream->pos)
Apr 8 22:19:26 mx2 dovecot: imap(clean-quarant...@spamguard.fr): Error:
Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x60ada)
[0x7f692abe1ada] -> /usr/lib/dovecot/libdovecot.so.0(+0x60b1e)
[0x7f692abe1b1e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f692aba3415] ->
/usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x1b9) [0x7f692abea2a9]
-> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d)
[0x7f692abea9dd] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0x9b624)
[0x7f692aed8624] ->
/usr/lib/dovecot/libdovecot-storage.so.0(index_mail_get_binary_stream+0x77)
[0x7f692aed8ab7] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_get_binary_stream+0x60)
[0x7f692aeb1640] ->
/usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x9f)
[0x7f692af0db4f] -> dovecot/imap() [0x418f35] -> dovecot/imap()
[0x4175dd] -> dovecot/imap(imap_fetch_more+0x34) [0x418534] ->
dovecot/imap(cmd_fetch+0x309) [0x40e2c9] ->
dovecot/imap(command_exec+0x3c) [0x41621c] -> dovecot/imap() [0x415280]
-> dovecot/imap() [0x41533a] -> dovecot/imap(client_handle_input+0x115)
[0x4155f5] -> dovecot/imap(client_input+0x72) [0x4159a2] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f692abf14a6]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7)
[0x7f692abf22f7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7f692abf0fe8] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7f692aba85a3] -> dovecot/imap(main+0x270) [0x40b940] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f692a815ead]
-> dovecot/imap() [0x40ba9d]
Apr 8 22:19:26 mx2 dovecot: imap(clean-quarant...@spamguard.fr): Fatal:
master: service(imap): child 4369 killed with signal 6 (core dumped)
#0 0x7f692a829475 in raise () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1 0x7f692a82c6f0 in abort () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#2 0x7f692abe1ae8 in default_fatal_finish (type=,
status=status@entry=0) at failures.c:191
backtrace = 0x62d818 "/usr/lib/dovecot/libdovecot.so.0(+0x60ada)
[0x7f692abe1ada] -> /usr/lib/dovecot/libdovecot.so.0(+0x60b1e)
[0x7f692abe1b1e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0)
[0x7f692aba3415] -> /usr/lib/d"...
#3 0x7f692abe1b1e in i_internal_fatal_handler (ctx=0x7fff605d4680,
format=, args=) at failures.c:652
status = 0
#4 0x7f692aba3415 in i_panic (format=format@entry=0x7f692ac06e68
"file %s: line %d (%s): assertion failed: (%s)") at failures.c:263
ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0}
args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area =
0x7fff605d4770, reg_save_area = 0x7fff605d46b0}}
#5 0x7f692abea2a9 in i_stream_read (stream=stream@entry=0x9fa6a0)
at istream.c:153
_stream = 0x9fa640
old_size = 0
ret = -2
__FUNCTION__ = "i_stream_read"
#6 0x7f692abea9dd in i_stream_read_data
(stream=stream@entry=0x9fa6a0, data_r=data_r@entry=0x7fff605d4820,
size_r=size_r@entry=0x7fff605d4828, threshold=threshold@entry=0) at
istream.c:479
ret =
read_more = false
__FUNCTION__ = "i_stream_read_data"
#7 0x7f692aed8624 in blocks_count_lines (full_input=0x9fa6a0,
ctx=0x7fff605d4830) at index-mail-binary.c:319
block_idx = 5
p =
size = 0
data = 0x0
ret =
cur_block = 0x62d488
block_count = 7
cur_offset = 2755
skip =
#8 index_mail_read_binary_to_cache (_mail=_mail@entry=0x66d630,
part=part@entry=0x672190, include_hdr=include_hdr@entry=false,
binary_r=binary_r@entry=0x7fff605d48e6,
converted_r=converted_r@entry=0x7fff605d48e7) at index-mail-binary.c:393
mail = 0x66d630
cache = 0x657b30
ctx = {mail = 0x66d630, input = 0x685c00, has_nuls = false,
converted = true, blocks = {arr = {buffer = 0x62d3d8, element_size =
24}, v = 0x62d3d8, v_modifiable = 0x62d3d8}, copy_start_offset = 113072}
__FUNCTION__ = "index_mail_read_binary_to_cache"
#9 0x7f692aed8ab7 in index_mail_get_binary_stream (_mail=0x66d630,
part=0x672190, include_hdr=false, size_r=0x7fff605d49a8,
lines_r=, binary_r=0x7fff605d499e,
Re: [Dovecot] dovecot 2 variables
Interesting, but this means that we need to have one tree with all users (where each user have additional attribute containing local ip), instead we now have two separate trees because we use this for other authentication purposes. Thank you for sharing your thoughts Steffen On Mon, Apr 8, 2013 at 2:34 PM, Steffen Kaiser < skdove...@smail.inf.fh-brs.de> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > On Sun, 7 Apr 2013, Pavel Dimow wrote: > > Just to be clear my goal is to have multiple domains on multiple addresses >> but use one dovecot instance and to let users logging without @domain >> part. >> > > If %l is expanded in the LDAP query, you could add an attribute with the > local IP address, e.g.: > > pass_filter = (&(objectClass=mailUser)(|(**mail=%Lu)(&(localPart=%Lu)(** > localIP=%l > > (check if parenthesis are balanced) Same with user_filter > > The idea: > > mailUserLDAPItem && ( nameWithDomain || ( nameWithoutDomain && localIP ) ) > > So the user could login with domain on any local port and without domain > on one or more local interfaces. > > - -- Steffen Kaiser > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEVAwUBUWK5PV3r2wJMiz2NAQKNMA**gAovEJQY0nhBwT0E/**d9tEd6wokF5XlVjpc > 15vvc3zbJ9AaQVMz4LHAx1N4Secx+**BP+**UrJLiPPIegGAPbExA4gjI2oC31sEUc**JB > 8iBtRlbVHLn+**pV0DnlG5FBn0KQgIyX1ml+**AafcFrVOq/PcCTQzdWqO0oyLR6PN8a > SGpArMTweVjMpbhiYdR1fqtc5iYEI2**MefO4OjEG0Xxc2KfqzDZqUSZ1H4+**ftPT9a > oA59e2sc3PRjCrjpeA4UbWNC/**ZEDSJFtt01mX3WZ00HVI/+**gMiPZVY7NTeaCC9W4 > rq0dI2J1O/zP2+**yY40KtuZrSt7Kw4b61LXO8Yp/**fJVpGoZzf6HvuWA== > =/+hu > -END PGP SIGNATURE- >
[Dovecot] How to see folders/subfolders/emails through imap
Hello all, I am trying to do something very simple - at least thats what I thought. I have some fs, it contains folders and subfolders with email files ordered like maildir. Now I try to set up dovecot on top simply to let some imap account watch these email files. But I cannot see any folders at all. I can create new folders and see them, but I cannot create subfolders as subdirs like "folder/subfolder". Instead I get "folder.subfolder" dirs on the fs. I tried to set the separator to "/", but that does not help at all. Is there some easy way to configure dovecot to display: ///new/files... /new/files... /new/files... according to fs layout on some imap-client (like thunderbird)? -- Regards, Stephan
Re: [Dovecot] repeating dsync - questions
My need at the moment is only a *one* time deal. I just need to copy/convert all of the messages over to the new system and new (sdbox) format *once*. Then all of the clients can start from zero to build their caches based on the new mail box (not that many clients). --- I will take a look at the Imap copy process - maybe that would be simpler in the long run. No rsync needed (as long as the source system is not changed by the Imap copy process) Thanks much for your comments and suggestions Bob G On Mon, 2013-04-08 at 13:57 +0300, Timo Sirainen wrote: > On 8.4.2013, at 1.32, Bob Gustafson wrote: > > > Ok, how is this for a scheme: > > > > 1) Initially rsync Maildir to destination system > > 2) Do initial 'dsync backup' from Maildir to sdbox on destination system > > 3) Do a 2nd rsync of new Maildir data from live system to destination > > system, but don't copy the old maildir-uidlist still in the Maildir of > > the live system. > > rsync -ar --times --exclude=dovecot-uidlist > system Maildir> > > > > I could also use the --ignore-existing option > > > > In theory, rsync would not touch the dovecot-uidlist file, would not > > touch the existing message files, but would copy over the new messages > > received during the time 'dsync backup' was doing its previous run. > > > > 4) Do a 2nd 'dsync backup' from the Maildir to the sdbox (which hasn't > > changed since the 1st 'dsync backup') > > Without rsync --delete you may end up with duplicates if message flags had > changed during it. > > There's also another potential problem. Since you don't now update the > dovecot-uidlist, the message UIDs may change. Some clients cache messages by > their UID. These clients may lose messages or show wrong messages to users. > So for example if: > > 0. rsync + dsync is run to sdbox. dovecot-uidlist now says that next_uid=123 > 1. Maildir receives mail A that gets assigned UID 123 > 2. Maildir receives mail B that gets assigned UID 124 > 3. User deletes mail A > 4. rsync is run, which copies the new mail B > 5. dsync is run, which notices a new mail B, and assigns it the a new UID 123 > 6. You switch user to new Dovecot > 7. dbox receives a new mail C, and gets assigned UID 124 > 8. User's client is now pretty much completely confused about what UIDs 123 > and 124 contain. User may see different mails as subject and body. User may > not even see the mail B anymore without a client cache rebuild. > > Yet another possibility would be to use dsync to migrate the mails using IMAP > protocol rather than from Maildir directly: > http://wiki2.dovecot.org/Migration/Dsync >
Re: [Dovecot] dsync replication: quota counted twice
On Mon, 8 Apr 2013 15:14:18 +0300 Timo Sirainen wrote: > > ok, thanks for the clarification. Using two different tables does > > the trick. I'll probably also have to disable the quota-warning > > scripts on one of the servers. Currently, both servers run the > > script when a account goes overquota -> two hate-mails in the > > user's Inbox. > > Ah. Hmm. This should solve it: > http://hg.dovecot.org/dovecot-2.2/rev/54cdc5b6727c > Thanks a lot..this seems to work, only server1 sends a quota warning now. maillog shows an (empty) error, but the script runs anyway: Apr 8 15:16:38 fumailsynctest1 dovecot: quota-warning: Error: close failed in file object destructor: Apr 8 15:16:38 fumailsynctest1 dovecot: quota-warning: Error: Error in sys.excepthook: Apr 8 15:16:38 fumailsynctest1 dovecot: quota-warning: Error: Apr 8 15:16:38 fumailsynctest1 dovecot: quota-warning: Error: Original exception was:
[Dovecot] Dovecot Configuration
Can somebody, please, point me in the right direction in finding a COMPLETE and EXACT definition of the Dovecot configuration hierarchy. In other words what configuration options exist both at the top level but also at each subconfiguration elements (ex. sections etc). While my C is a bit rusty, I can still make sense of source code and with a bit of effort I will figure it out. I have been going through the src/config/all-settings.c but still have large gaps. A hint in the right direction on how the config works might help clarify and speed things up. Thanks Andreas Kasenides
Re: [Dovecot] Problem with fts lucene, on solaris 10
On Monday 08 April 2013, Timo Sirainen wrote:
> On 8.4.2013, at 12.19, Pigi wrote:
>
> > Program received signal SIGSEGV, Segmentation fault.
> > 0xfebd65dc in _malloc_unlocked () from /lib/libc.so.1
> > (gdb) bt full
> > #0 0xfebd65dc in _malloc_unlocked () from /lib/libc.so.1
> > No symbol table info available.
> > #1 0xfebd66e4 in realloc () from /lib/libc.so.1
> > No symbol table info available.
> > #2 0xff1d9a70 in pool_system_realloc (pool=0xff21576c, mem=0xff21576c,
> > old_size=0, new_size=8192) at mempool-system.c:115
> > No locals.
> > #3 0xff1cda28 in i_realloc (mem=0x0, old_size=0, new_size=8192) at
> > imem.c:14
> > No locals.
>
> Looks to me like memory corruption, which is just about impossible to debug
> if you don't have valgrind or a similar tool (and I don't think Solaris has
> any such free tools).
Yes, you're right. There aren't ( already packaged ) tool for valgrinding or
similar.
> Do you happen to get any other crashes where it doesn't crash in malloc code?
> You showed previously that it was crashing with SIGBUS somewhere.
For what I can see, form dovecot.log I have a couple of error:
Apr 08 12:02:18 indexer: Error: Indexer worker disconnected, discarding 1
requests for
Apr 08 12:02:18 imap(): Error: indexer failed to index mailbox INBOX
Apr 08 12:02:18 indexer-worker(): Fatal: master: service(indexer-worker):
child 27242 killed with signal 11 (core not dumped)
Apr 08 12:02:19 indexer-worker(): Error: lucene index
/home//Maildir/lucene-indexes: IndexWriter() failed (#1): Lock obtain timed
out
Apr 08 12:03:46 indexer: Error: Indexer worker disconnected, discarding 1
requests for
Apr 08 12:03:46 imap(): Error: indexer failed to index mailbox INBOX
Apr 08 12:03:46 indexer-worker(): Fatal: master: service(indexer-worker):
child 27445 killed with signal 10 (core not dumped)
Apr 08 12:06:57 indexer: Error: Indexer worker disconnected, discarding 1
requests for
Apr 08 12:06:57 imap(): Error: indexer failed to index mailbox INBOX
Apr 08 12:06:57 indexer-worker(): Fatal: master: service(indexer-worker):
child 27761 killed with signal 11 (core not dumped)
Apr 08 12:08:33 indexer: Error: Indexer worker disconnected, discarding 1
requests for
Here a couple or more of gdb"ed" process, but tied to SEGV ( I almost suspect
that when gdbing I always catch a SEGV ) :
==
First gdb
==
No symbol table info available.
#1 0xfebd6884 in realloc () from /lib/libc.so.1
No symbol table info available.
#2 0xfe83d270 in lucene::util::StringReader::init (this=0x8d328,
_value=0x2bb288, _length=3958, copyData=true)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/util/Reader.cpp:48
tmp = (TCHAR *) 0x2051e0
length = 3958
#3 0xfe8837f0 in
lucene::index::DocumentsWriter::ThreadState::FieldData::invertField
(this=0x92df8, field=0x8d238, analyzer=0x2cb60, maxFieldLength=1)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriterThreadState.cpp:887
stringValue = (const TCHAR *) 0x2bb288
stringValueLength = 3958
stream = (class lucene::analysis::TokenStream *) 0x0
streamValue = (class lucene::analysis::TokenStream *) 0x8d328
#4 0xfe883a58 in
lucene::index::DocumentsWriter::ThreadState::FieldData::processField
(this=0x92df8, analyzer=0x2cb60)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriterThreadState.cpp:798
field = (class lucene::document::Field *) 0x8d238
#5 0xfe883d6c in lucene::index::DocumentsWriter::ThreadState::processDocument
(this=0x8c0a8, analyzer=0x2cb60)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/util/Array.h:90
i = 6
#6 0xfe87ea00 in lucene::index::DocumentsWriter::updateDocument (this=0x84790,
doc=0x8b838, analyzer=0x2cb60, delTerm=0x0)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:946
state = (class lucene::index::DocumentsWriter::ThreadState *) 0x8c0a8
#7 0xfe87eb5c in lucene::index::DocumentsWriter::addDocument (this=0x84790,
doc=0x8b838, analyzer=0x2cb60)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/index/DocumentsWriter.cpp:930
No locals.
#8 0xfe8b1ea0 in lucene::index::IndexWriter::addDocument (this=0x8cc18,
doc=0x8b838, analyzer=0x2cb60)
at
/extpool/PACKAGES/DOVECOT/clucene-core-2.3.3.4/src/core/CLucene/index/IndexWriter.cpp:682
No locals.
#9 0xfea18e04 in lucene_index_build_flush (index=0x37ba8) at
lucene-wrapper.cc:484
ret = 0
#10 0xfea18f88 in lucene_index_build_more (index=0x37ba8, uid=2995,
data=0x185d18 "", size=23, hdr_name=0x8ba60 "Return-Path")
at lucene-wrapper.cc:507
id = {4, 0, 0, 0, -4196824, -15011172, 534768, -4196336, 516256, 0,
11560, -20155708, 9, 10, 36,
Re: [Dovecot] dovecot 2 variables
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, 7 Apr 2013, Pavel Dimow wrote: Just to be clear my goal is to have multiple domains on multiple addresses but use one dovecot instance and to let users logging without @domain part. If %l is expanded in the LDAP query, you could add an attribute with the local IP address, e.g.: pass_filter = (&(objectClass=mailUser)(|(mail=%Lu)(&(localPart=%Lu)(localIP=%l (check if parenthesis are balanced) Same with user_filter The idea: mailUserLDAPItem && ( nameWithDomain || ( nameWithoutDomain && localIP ) ) So the user could login with domain on any local port and without domain on one or more local interfaces. - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBUWK5PV3r2wJMiz2NAQKNMAgAovEJQY0nhBwT0E/d9tEd6wokF5XlVjpc 15vvc3zbJ9AaQVMz4LHAx1N4Secx+BP+UrJLiPPIegGAPbExA4gjI2oC31sEUcJB 8iBtRlbVHLn+pV0DnlG5FBn0KQgIyX1ml+AafcFrVOq/PcCTQzdWqO0oyLR6PN8a SGpArMTweVjMpbhiYdR1fqtc5iYEI2MefO4OjEG0Xxc2KfqzDZqUSZ1H4+ftPT9a oA59e2sc3PRjCrjpeA4UbWNC/ZEDSJFtt01mX3WZ00HVI/+gMiPZVY7NTeaCC9W4 rq0dI2J1O/zP2+yY40KtuZrSt7Kw4b61LXO8Yp/fJVpGoZzf6HvuWA== =/+hu -END PGP SIGNATURE-
[Dovecot] "No such file or directory" error setting up dict quota for mdbox
Hi, I've some troubles setting up the dict based quota plugin for mdbox, it
always returns a error when lmtp tries to deliver an e-mail:
Apr 8 12:40:16 mb07 dovecot: lmtp(3001, asma...@ull.es): Error:
fstat(/sharedfs/10.4.1.107/ull.es/54/asmarre/dovecot-quota.lock) failed: No
such file or directory
Apr 8 12:40:16 mb07 dovecot: lmtp(3001, asma...@ull.es): Error: file dict
commit: file_dotlock_open(/sharedfs/
10.4.1.107/ull.es/54/asmarre/dovecot-quota) failed: No such file or
directory
Apr 8 12:40:16 mb07 dovecot: lmtp(3001, asma...@ull.es): Error: dict
quota: Quota update failed, it's now desynced
In fact lmtp does create the dovecot-quota.lock file but it does not update
the dovecot-quota file.
If I run "doveadm quota recalc -u asmarre" it works and updates the
dovecot-quota file.
The version of dovecot that I'm using is 2.0.19 and the configuration is:
# 2.0.19: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-32-generic x86_64 Ubuntu 12.04.2 LTS
mail_debug = yes
mail_location = mdbox:%h
mail_plugins = quota
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
quota = dict:User quota::file:%h/dovecot-quota
quota_rule = *:storage=1G
quota_warning = storage=95%% quota-warning 95
quota_warning2 = storage=85%% quota-warning 85
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
postmaster_address = postmas...@ull.es
protocols = " imap lmtp sieve pop3"
service lmtp {
inet_listener lmtp {
port = 24
}
}
service managesieve-login {
inet_listener sieve_deprecated {
port = 2000
}
}
service quota-warning {
executable = script /opt/ccti/correo/buzones/quota_warning.sh
unix_listener quota-warning {
user = dovecot
}
user = dovecot
}
ssl_cert =
Re: [Dovecot] dsync replication: quota counted twice
On 8.4.2013, at 14.56, Oli Schacher wrote: > On Mon, 8 Apr 2013 13:15:54 +0300 > Timo Sirainen wrote: > > >> There's no way to make quota work reliably that way. You need to use >> different quota backends. Either two mysql databases (or tables), or >> the other Dovecot could simply be using a dict file quota. >> > > ok, thanks for the clarification. Using two different tables does the > trick. I'll probably also have to disable the quota-warning > scripts on one of the servers. Currently, both servers run the script > when a account goes overquota -> two hate-mails in the user's Inbox. Ah. Hmm. This should solve it: http://hg.dovecot.org/dovecot-2.2/rev/54cdc5b6727c
Re: [Dovecot] dsync replication: quota counted twice
On Mon, 8 Apr 2013 13:15:54 +0300 Timo Sirainen wrote: > There's no way to make quota work reliably that way. You need to use > different quota backends. Either two mysql databases (or tables), or > the other Dovecot could simply be using a dict file quota. > ok, thanks for the clarification. Using two different tables does the trick. I'll probably also have to disable the quota-warning scripts on one of the servers. Currently, both servers run the script when a account goes overquota -> two hate-mails in the user's Inbox.
Re: [Dovecot] dovecot 2 variables
Am 08.04.2013 13:18, schrieb Benny Pedersen: > On 2013-04-07 22:51, Pavel Dimow wrote: > >> Why does dovecot does not expand %l to local ip address? > > what it it does not contain an ip address ?, like "rm -f /" what is your f**ing problem? signature.asc Description: OpenPGP digital signature
Re: [Dovecot] dovecot 2 variables
On 2013-04-07 22:51, Pavel Dimow wrote: Why does dovecot does not expand %l to local ip address? what it it does not contain an ip address ?, like "rm -f /" -- senders that put my email into body content will deliver it to my own trashcan, so if you like to get reply, dont do it
Re: [Dovecot] repeating dsync - questions
On 8.4.2013, at 1.32, Bob Gustafson wrote: > Ok, how is this for a scheme: > > 1) Initially rsync Maildir to destination system > 2) Do initial 'dsync backup' from Maildir to sdbox on destination system > 3) Do a 2nd rsync of new Maildir data from live system to destination > system, but don't copy the old maildir-uidlist still in the Maildir of > the live system. > rsync -ar --times --exclude=dovecot-uidlist system Maildir> > > I could also use the --ignore-existing option > > In theory, rsync would not touch the dovecot-uidlist file, would not > touch the existing message files, but would copy over the new messages > received during the time 'dsync backup' was doing its previous run. > > 4) Do a 2nd 'dsync backup' from the Maildir to the sdbox (which hasn't > changed since the 1st 'dsync backup') Without rsync --delete you may end up with duplicates if message flags had changed during it. There's also another potential problem. Since you don't now update the dovecot-uidlist, the message UIDs may change. Some clients cache messages by their UID. These clients may lose messages or show wrong messages to users. So for example if: 0. rsync + dsync is run to sdbox. dovecot-uidlist now says that next_uid=123 1. Maildir receives mail A that gets assigned UID 123 2. Maildir receives mail B that gets assigned UID 124 3. User deletes mail A 4. rsync is run, which copies the new mail B 5. dsync is run, which notices a new mail B, and assigns it the a new UID 123 6. You switch user to new Dovecot 7. dbox receives a new mail C, and gets assigned UID 124 8. User's client is now pretty much completely confused about what UIDs 123 and 124 contain. User may see different mails as subject and body. User may not even see the mail B anymore without a client cache rebuild. Yet another possibility would be to use dsync to migrate the mails using IMAP protocol rather than from Maildir directly: http://wiki2.dovecot.org/Migration/Dsync
Re: [Dovecot] checkpassword protocol
On 8.4.2013, at 13.42, Heiko Schlichting wrote: > The checkpassword interface is used in many different ways: > > I found checkpassword the easiest way to implement authentication against > FreeRADIUS. Using a perl script with > >$input = IO::Handle->new_from_fd(3, "r"); > > and > >my $output = IO::Handle->new_from_fd(4, "w"); > > and skipping the checkpassword-reply binary. This works fine on our server > for many years now but every time when I read some related questions on the > list, it seems that a more generic, not so crude interface would be a good > idea for a future version of dovecot. For executing scripts/binaries I think checkpassword will stay the only way. Other than that, you could talk to Dovecot-auth via UNIX socket by using dict-proxy protocol with passdb/userdb dict. Although that way you'd need some way to keep your server running and it's not that easy yet to create a new Dovecot service without using libdovecot.
Re: [Dovecot] ssl_require_crl does not work as expected
On 7.4.2013, at 14.19, dovecot.pk...@dfgh.net wrote:
> My impression is that openssl will always try to use CRLs. If
> "ssl_require_crl = no"
> dovecot will use CRLs but tries to ignore openssl error codes
> X509_V_ERR_UNABLE_TO_GET_CRL and X509_V_ERR_CRL_HAS_EXPIRED.
>
> This is done in ssl_verify_client_cert() in ssl-proxy-openssl.c line 871,
> namely
>
> if (proxy->client_proxy && !proxy->set->ssl_require_crl &&
> (ctx->error == X509_V_ERR_UNABLE_TO_GET_CRL ||
>ctx->error == X509_V_ERR_CRL_HAS_EXPIRED)) {
> /* no CRL given with the CA list. don't worry about it. */
> preverify_ok = 1;
> }
After thinking enough about this, I kind of see what's happening:
The ssl_require_crl setting wasn't intended to do what you want it to do. It
was meant for Dovecot proxying code where it needs to act as an SSL client to
remote server. So the setting should instead have been named
ssl_require_client_crl.
For server's client cert checks you should be able to simply generate an empty
CRL I think?
> I dont know what the proxy-stuff is about so instead of ignoring CRL-related
> errors I tried to disable CRL-checking. I therefore commented out two lines
> in
> ssl_proxy_ctx_verify_client() in ssl-proxy-openssl.c line 1004, namely:
>
> // X509_STORE_set_flags(store, X509_V_FLAG_CRL_CHECK |
> // X509_V_FLAG_CRL_CHECK_ALL);
If you remove the "proxy->client_proxy &&" check it should work just as well, I
think?
> Similar code is contained in iostream-openssl-context.c,
> namely in routine ssl_iostream_ctx_verify_remote_cert()
Now I'm wondering why this code works as SSL client even though I'm not reading
CRLs anywhere.. I guess I should spend some time looking into how exactly CRLs
should work.
Re: [Dovecot] checkpassword protocol
Timo wrote: > All the OSes made the environment private 10-15 years ago. I think it's > pretty safe to assume that older multiuser systems won't be running > Dovecot with checkpassword backend. The checkpassword interface is used in many different ways: I found checkpassword the easiest way to implement authentication against FreeRADIUS. Using a perl script with $input = IO::Handle->new_from_fd(3, "r"); and my $output = IO::Handle->new_from_fd(4, "w"); and skipping the checkpassword-reply binary. This works fine on our server for many years now but every time when I read some related questions on the list, it seems that a more generic, not so crude interface would be a good idea for a future version of dovecot. > But .. eh. I guess: http://hg.dovecot.org/dovecot-2.2/rev/9feb2986945c Great. Thank you. Heiko Heiko SchlichtingFreie Universität Berlin heiko.schlicht...@fu-berlin.de Zentraleinrichtung für Datenverarbeitung Telefon +49 30 838-54327 Fabeckstraße 32 Telefax +49 30 838454327 14195 Berlin
Re: [Dovecot] dovecot 2 variables
Thank you Timo, then I guess I will runing two instances of dovecot one for
each domain until
local 1.2.3.4 {
userdb {
..
}
}
becomes ready :)
On Mon, Apr 8, 2013 at 12:14 PM, Timo Sirainen wrote:
> You could put (%l=1.2.3.4) in the ldap filter. But I guess it still gets
> sent to the LDAP server. The best solution would be:
>
> local 1.2.3.4 {
> userdb {
> ..
> }
> }
>
> But that doesn't work yet.
>
> With v2.2 you can do multiple queries. So you could have one query to
> translate IP address to the domain, and then a second lookup to lookup the
> user@domain. http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb ->
> subqueries.
>
> On 8.4.2013, at 13.07, Pavel Dimow wrote:
>
> > Well that is not very scalable as we always ask first database no matter
> > what. What if I have the same user in both databases (ldap)?
> >
> >
> > On Mon, Apr 8, 2013 at 12:36 AM, Daniel Parthey <
> > daniel.part...@informatik.tu-chemnitz.de> wrote:
> >
> >> Pavel Dimow wrote:
> >>> Just to be clear my goal is to have multiple domains on multiple
> >> addresses
> >>> but use one dovecot instance and to let users logging without @domain
> >> part.
> >>
> >> You might try to use several passdb/userdb sections with driver ldap,
> >> one for each domain:
> >>
> >> http://wiki2.dovecot.org/Authentication/MultipleDatabases
> >>
> >> Regards
> >> Daniel
> >> --
> >> https://plus.google.com/103021802792276734820
> >>
>
>
Re: [Dovecot] checkpassword protocol
On 8.4.2013, at 10.00, Heiko Schlichting wrote: >> Hmm. The AUTH_PASSWORD wasn't really an intentional addition .. but >> I guess it can stay there. Some 10 years ago that might not have been >> such a good idea since there were still some systems where process >> environment variables were readable to all users in the system, but >> I doubt there exist such systems anymore (at least where people would >> want to run Dovecot). > > Very optimistic assumption. Wouldn't it be safer to remove the password > from the environment? Anyone using checkpassword should use FD 3 and 4 for > this purpose. Environment variables and command line arguments are not safe > to transport passwords. All the OSes made the environment private 10-15 years ago. I think it's pretty safe to assume that older multiuser systems won't be running Dovecot with checkpassword backend. But .. eh. I guess: http://hg.dovecot.org/dovecot-2.2/rev/9feb2986945c
Re: [Dovecot] dsync replication: quota counted twice
On 8.4.2013, at 13.05, Oli Schacher wrote: > I enabled quota + warnings in my replication test setup and now > whenever a mail gets delivered the quota is counted twice (probably due > to both servers using the same mysql quota backend) Yes. Dovecot doesn't know that they are the same quota backend. > and quota warnings > are fired too early. After a doveadm quota recalc -u the > quota is correct again. How do I make dovecot only count the quota on > the server where the mail was added originally? There's no way to make quota work reliably that way. You need to use different quota backends. Either two mysql databases (or tables), or the other Dovecot could simply be using a dict file quota.
Re: [Dovecot] dovecot 2 variables
You could put (%l=1.2.3.4) in the ldap filter. But I guess it still gets sent
to the LDAP server. The best solution would be:
local 1.2.3.4 {
userdb {
..
}
}
But that doesn't work yet.
With v2.2 you can do multiple queries. So you could have one query to translate
IP address to the domain, and then a second lookup to lookup the user@domain.
http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb -> subqueries.
On 8.4.2013, at 13.07, Pavel Dimow wrote:
> Well that is not very scalable as we always ask first database no matter
> what. What if I have the same user in both databases (ldap)?
>
>
> On Mon, Apr 8, 2013 at 12:36 AM, Daniel Parthey <
> daniel.part...@informatik.tu-chemnitz.de> wrote:
>
>> Pavel Dimow wrote:
>>> Just to be clear my goal is to have multiple domains on multiple
>> addresses
>>> but use one dovecot instance and to let users logging without @domain
>> part.
>>
>> You might try to use several passdb/userdb sections with driver ldap,
>> one for each domain:
>>
>> http://wiki2.dovecot.org/Authentication/MultipleDatabases
>>
>> Regards
>> Daniel
>> --
>> https://plus.google.com/103021802792276734820
>>
Re: [Dovecot] dovecot 2 variables
Well that is not very scalable as we always ask first database no matter what. What if I have the same user in both databases (ldap)? On Mon, Apr 8, 2013 at 12:36 AM, Daniel Parthey < daniel.part...@informatik.tu-chemnitz.de> wrote: > Pavel Dimow wrote: > > Just to be clear my goal is to have multiple domains on multiple > addresses > > but use one dovecot instance and to let users logging without @domain > part. > > You might try to use several passdb/userdb sections with driver ldap, > one for each domain: > > http://wiki2.dovecot.org/Authentication/MultipleDatabases > > Regards > Daniel > -- > https://plus.google.com/103021802792276734820 >
[Dovecot] dsync replication: quota counted twice
Hi I enabled quota + warnings in my replication test setup and now whenever a mail gets delivered the quota is counted twice (probably due to both servers using the same mysql quota backend) and quota warnings are fired too early. After a doveadm quota recalc -u the quota is correct again. How do I make dovecot only count the quota on the server where the mail was added originally? dovecot -n is the same as in my previous message to the list today http://www.dovecot.org/list/dovecot/2013-April/089558.html (thanks Timo for the hint in this thread btw, changing the permissions made the errors go away) Thanks Oli -- message transmitted on 100% recycled electrons
Re: [Dovecot] Problem with fts lucene, on solaris 10
On 8.4.2013, at 12.19, Pigi wrote:
>> gdb /usr/local/libexec/dovecot/imap 16400
..
> I took a sligtly different approach:
> gdb /usr/local/dovecot/libexec/dovecot/indexer-worker `ps -ef | grep
> indexer-worker | grep -v grep |awk '{print $2}' `
>
> thus attaching to indexer-worker, which I think ( but not sure ) it should be
> fine.
Oh, right, I forgot which process you were gdbing :)
> Program received signal SIGSEGV, Segmentation fault.
> 0xfebd65dc in _malloc_unlocked () from /lib/libc.so.1
> (gdb) bt full
> #0 0xfebd65dc in _malloc_unlocked () from /lib/libc.so.1
> No symbol table info available.
> #1 0xfebd66e4 in realloc () from /lib/libc.so.1
> No symbol table info available.
> #2 0xff1d9a70 in pool_system_realloc (pool=0xff21576c, mem=0xff21576c,
> old_size=0, new_size=8192) at mempool-system.c:115
> No locals.
> #3 0xff1cda28 in i_realloc (mem=0x0, old_size=0, new_size=8192) at imem.c:14
> No locals.
Looks to me like memory corruption, which is just about impossible to debug if
you don't have valgrind or a similar tool (and I don't think Solaris has any
such free tools).
Do you happen to get any other crashes where it doesn't crash in malloc code?
You showed previously that it was crashing with SIGBUS somewhere.
Re: [Dovecot] Problem with fts lucene, on solaris 10
On Monday 08 April 2013, Timo Sirainen wrote:
> On 8.4.2013, at 11.54, Pigi wrote:
>
>
> gdb is just confused. Try another way if it works:
>
> gdb /usr/local/libexec/dovecot/imap 16400
> cont
> bt full
>
>
Timo,
I took a sligtly different approach:
gdb /usr/local/dovecot/libexec/dovecot/indexer-worker `ps -ef | grep
indexer-worker | grep -v grep |awk '{print $2}' `
thus attaching to indexer-worker, which I think ( but not sure ) it should be
fine.
Here the result:
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "sparc-sun-solaris2.10"...
Attaching to program `/usr/local/dovecot/libexec/dovecot/indexer-worker',
process 21672
Reading symbols from
/usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0...done.
Loaded symbols for /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /usr/local/dovecot/lib/dovecot/libdovecot.so.0...done.
Loaded symbols for /usr/local/dovecot/lib/dovecot/libdovecot.so.0
Reading symbols from /usr/local/openssl-1.0.1e/lib/libssl.so.1.0.0...done.
Loaded symbols for /usr/local/openssl-1.0.1e/lib//libssl.so.1.0.0
Reading symbols from /usr/local/openssl-1.0.1e/lib/libcrypto.so.1.0.0...done.
Loaded symbols for /usr/local/openssl-1.0.1e/lib//libcrypto.so.1.0.0
Reading symbols from /lib/libdl.so.1...
warning: Lowest section in /lib/libdl.so.1 is .hash at 00b4
done.
Loaded symbols for /lib/libdl.so.1
Reading symbols from /usr/local/lib/libiconv.so.2...done.
Loaded symbols for /usr/local/lib/libiconv.so.2
Reading symbols from /lib/libnsl.so.1...done.
Loaded symbols for /lib/libnsl.so.1
Reading symbols from /lib/libsocket.so.1...done.
Loaded symbols for /lib/libsocket.so.1
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libsendfile.so.1...done.
Loaded symbols for /lib/libsendfile.so.1
Reading symbols from /lib/libc.so.1...done.
Loaded symbols for /lib/libc.so.1
Reading symbols from /usr/sfw/lib/libgcc_s.so.1...done.
Loaded symbols for /usr/sfw/lib/libgcc_s.so.1
Reading symbols from /lib/libaio.so.1...done.
Loaded symbols for /lib/libaio.so.1
Reading symbols from /lib/libmd.so.1...done.
Loaded symbols for /lib/libmd.so.1
Reading symbols from /platform/SUNW,Sun-Fire-V240/lib/libc_psr.so.1...done.
Loaded symbols for /platform/SUNW,Sun-Fire-V240/lib/libc_psr.so.1
Reading symbols from /usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so...done.
Loaded symbols for /usr/local/dovecot/lib/dovecot/lib20_fts_plugin.so
Reading symbols from /lib/libscf.so.1...done.
Loaded symbols for /lib/libscf.so.1
Reading symbols from /lib/libdoor.so.1...done.
Loaded symbols for /lib/libdoor.so.1
Reading symbols from /lib/libuutil.so.1...done.
Loaded symbols for /lib/libuutil.so.1
Reading symbols from /lib/libgen.so.1...done.
Loaded symbols for /lib/libgen.so.1
Reading symbols from /platform/SUNW,Sun-Fire-V240/lib/libmd_psr.so.1...done.
Loaded symbols for /platform/SUNW,Sun-Fire-V240/lib/libmd_psr.so.1
Reading symbols from /lib/libmp.so.2...done.
Loaded symbols for /lib/libmp.so.2
Reading symbols from
/usr/local/dovecot/lib/dovecot/lib21_fts_lucene_plugin.so...done.
Loaded symbols for /usr/local/dovecot/lib/dovecot/lib21_fts_lucene_plugin.so
Reading symbols from /usr/local/clucene/lib/libclucene-shared.so.1...done.
Loaded symbols for /usr/local/clucene/lib//libclucene-shared.so.1
Reading symbols from /usr/local/clucene/lib/libclucene-core.so.1...done.
Loaded symbols for /usr/local/clucene/lib//libclucene-core.so.1
Reading symbols from /usr/sfw/lib/libstdc++.so.6...done.
Loaded symbols for /usr/sfw/lib/libstdc++.so.6
Reading symbols from /lib/libm.so.2...done.
Loaded symbols for /lib/libm.so.2
Reading symbols from /usr/lib/libz.so...done.
Loaded symbols for /usr/lib/libz.so
Reading symbols from /lib/ld.so.1...done.
Loaded symbols for /lib/ld.so.1
sol-thread active.
Retry #1:
Retry #2:
Retry #3:
Retry #4:
[New LWP1]
[New Thread 1 (LWP 1)]
Symbols already loaded for
/usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0
Symbols already loaded for /usr/local/dovecot/lib/dovecot/libdovecot.so.0
Symbols already loaded for /usr/local/openssl-1.0.1e/lib//libssl.so.1.0.0
Symbols already loaded for /usr/local/openssl-1.0.1e/lib//libcrypto.so.1.0.0
Symbols already loaded for /lib/libdl.so.1
Symbols already loaded for /usr/local/lib/libiconv.so.2
Symbols already loaded for /lib/libnsl.so.1
Symbols already loaded for /lib/libsocket.so.1
Symbols already loaded for /lib/librt.so.1
Symbols already loaded for /lib/libsendfile.so.1
Symbols already loaded for /lib/libc.so.1
Symbols already loaded for /usr/sfw/lib/libgcc_s.so.1
Symbols already loaded for /lib/libaio.so.1
Symbols already loaded for /lib/libmd.so.1
S
Re: [Dovecot] Problem with fts lucene, on solaris 10
On 8.4.2013, at 11.54, Pigi wrote: > I haven't had too much luck with gdb: > > 0xfec4c554 in ?? () > (gdb) cont > Continuing. > > Program received signal SIGSEGV, Segmentation fault. > 0xfebd710c in ?? () > (gdb) bt full > #0 0xfebd710c in ?? () > No symbol table info available. > (gdb) quit > The program is running. Quit anyway (and detach it)? (y or n) y > Detaching from program: , process 16400 gdb is just confused. Try another way if it works: gdb /usr/local/libexec/dovecot/imap 16400 cont bt full
Re: [Dovecot] Problem with fts lucene, on solaris 10
I haven't had too much luck with gdb:
0xfec4c554 in ?? ()
(gdb) cont
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0xfebd710c in ?? ()
(gdb) bt full
#0 0xfebd710c in ?? ()
No symbol table info available.
(gdb) quit
The program is running. Quit anyway (and detach it)? (y or n) y
Detaching from program: , process 16400
That's rather strange, as :
root# file /usr/local/dovecot/sbin/dovecot
dovecot:ELF 32-bit MSB executable SPARC Version 1, dynamically linked,
not stripped
root# file /usr/local/clucene/lib/lib*
/usr/local/clucene/lib/libclucene-core.so: ELF 32-bit MSB dynamic lib
SPARC Version 1, dynamically linked, not stripped
/usr/local/clucene/lib/libclucene-core.so.1:ELF 32-bit MSB dynamic lib
SPARC Version 1, dynamically linked, not stripped
/usr/local/clucene/lib/libclucene-core.so.2.3.3.4: ELF 32-bit MSB dynamic
lib SPARC Version 1, dynamically linked, not stripped
/usr/local/clucene/lib/libclucene-shared.so:ELF 32-bit MSB dynamic lib
SPARC Version 1, dynamically linked, not stripped
/usr/local/clucene/lib/libclucene-shared.so.1: ELF 32-bit MSB dynamic lib
SPARC Version 1, dynamically linked, not stripped
/usr/local/clucene/lib/libclucene-shared.so.2.3.3.4:ELF 32-bit MSB dynamic
lib SPARC Version 1, dynamically linked, not stripped
root #
Thus, every binary, or lib, is not stripped.
If I invoke gdb on those binary I can get every simbol:
root # gdb /usr/local/dovecot/sbin/dovecot
... SNIP ...
(gdb) list
736 "\n", IO_BLOCK_SIZE);
737 }
738
739 int main(int argc, char *argv[])
740 {
741 struct master_settings *set;
742 const char *error, *doveconf_arg = NULL;
743 failure_callback_t *orig_info_callback, *orig_debug_callback;
744 bool foreground = FALSE, ask_key_pass = FALSE;
745 bool doubleopts[argc];
or even
root # gdb /usr/local/clucene/lib/libclucene-core.so
... SNIP ...
(gdb) list
1
/*--
2 * Copyright (C) 2003-2006 Ben van Klinken and the CLucene Team
3 *
4 * Distributable under the terms of either the Apache License (Version
2.0) or
5 * the GNU Lesser General Public License, as specified in the COPYING
file.
6
--*/
7 #include "CLucene/_ApiHeader.h"
8 #include "CLucene/util/Misc.h"
9
10 #include "CLucene/search/Sort.h"
(gdb)
Should I compile with -g (and in case, how do I pass this flag to cmake ) ?
Thanks.
Pigi
On Sunday 07 April 2013, Timo Sirainen wrote:
> On 5.4.2013, at 22.59, Pigi wrote:
>
> > I'm planning to migrate my courier-imap imap server to dovecot, but I'm
> > experiencing a strange issue
> > with fts-lucene plugin.
> >
> > Basically, every time I start a search, the log starts to write:
> >
> > Apr 05 19:30:53 indexer: Error: Indexer worker disconnected, discarding 1
> > requests for XX
> > Apr 05 19:30:53 indexer-worker(X): Fatal: master:
> > service(indexer-worker): child 809 killed with signal 11 (core not dumped)
> > Apr 05 19:30:53 imap(XX): Error: indexer failed to index mailbox INBOX
> > Apr 05 19:30:56 indexer-worker(X): Error: lucene index
> > /home/X/Maildir/lucene-indexes: IndexWriter() failed (#1): Lock obtain
> > timed out
> >
> > When trussing the indexer-worker pid I can see it ends with:
>
> If you can truss it, you can also gdb it, which is much more helpful:
>
> gdb -p
> cont
>
> bt full
>
> > 527: siginfo: SIGBUS BUS_ADRALN addr=0x005B50BA
> > 527:Received signal #10, SIGBUS [default]
> > 527: siginfo: SIGBUS BUS_ADRALN addr=0x005B50BA
>
> That says it's a SIGBUS, but the Dovecot log said it's a SIGSEGV. Wonder
> what's the difference, or maybe you're getting both. Anyway I'd guess this is
> a clucene bug related to CPU alignment with Sparc CPU. gdb would confirm and
> could help with fixing it.
>
>
Re: [Dovecot] MOSTLY SOLVED: Re: client limit and STARTTLS
On Sat, 2013-04-06 at 01:09 -0700, David Benfell wrote: > > Are your certificate chains valid? Simply saying "isn't working" is > > almost a requirement for a *sigh* > > log output please, or a better description than "isn't working" ;) > Yes, the certificate chains were screwed up. And you hit the nail on the > head when you pointed to chains. It's a StartSSL cert and I've had > trouble getting this straight from the beginning. > no problems, it's the main culprit for most SSL issues > Thanks! > > Now if we can sort out how to set the client and process limits. I > changed the settings like thus: > > default_process_limit = 1024 > default_client_limit = 4096 > I see Harald has explained this already (I dont use high performance mode, I prefer security, but you wont notice any performance hit anyway, we never have) But if you can't see yourself with that many concurrent users, remove them and stick with defaults, unless you do need to increase them, wich you'll soon learn from your users, worry about it then. > The truth is, I can't imagine having 4096 clients. I mean, it's true, I > like my toys, but. > When you see this, just set a ulimit in dovecots startup script #!/bin/sh # rc.dovecot # Start/stop/restart dovecot. ulimit -n 8192 ...dovecot stuff... Cheers signature.asc Description: This is a digitally signed message part
Re: [Dovecot] v2.2.rc6 released
On 8.4.2013, at 11.22, Oli Schacher wrote: > In my dsync replication test setup I now get Permission denied errors: > > Apr 8 10:04:33 fumailsynctest2 dovecot: dsync-remote(): Error: > net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission > denied > > I haven't noticed them before rc6. Bug or something wrong with my config? dsync is trying to notify the replicator that a user was synced. I could maybe just hide that error message, since it's not a real problem. But keeping the two replicators somewhat updated about what users are synced is better, so you could also change the socket's permissions so that dsync has access to it (http://wiki2.dovecot.org/Replication).
Re: [Dovecot] v2.2.rc6 released
Hi Timo
In my dsync replication test setup I now get Permission denied errors:
Apr 8 10:04:33 fumailsynctest2 dovecot: dsync-remote(): Error:
net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: Permission denied
I haven't noticed them before rc6. Bug or something wrong with my config?
Thanks
Oli
# 2.2.rc6: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-358.2.1.el6.x86_64 x86_64 CentOS release 6.4 (Final)
auth_master_user_separator = *
auth_mechanisms = plain login
dict {
acl = mysql:/etc/dovecot/dovecot-dict-shares.conf
quotadict = mysql:/etc/dovecot/dovecot-dict-quota.conf
}
disable_plaintext_auth = no
doveadm_password =
listen = *
login_greeting = Fumail Dovecot ready.
mail_max_userip_connections = 50
mail_plugins = " quota notify replication"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date
namespace {
list = children
location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u
prefix = shared/%%u/
separator = /
subscriptions = no
type = shared
}
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Spam {
auto = subscribe
special_use = \Junk
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
acl = vfile:/etc/dovecot/acls
acl_shared_dict = proxy::acl
mail_replica = tcp:1..ch:1337
quota = dict:::proxy::quotadict
quota_rule = *:storage=10M:messages=1000
quota_rule2 = Spam:ignore
quota_rule3 = Trash:storage=+100M
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=75%% quota-warning 75 %u
sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve
}
protocols = pop3 imap lmtp sieve
service aggregator {
fifo_listener replication-notify-fifo {
user = fumail
}
unix_listener replication-notify {
user = fumail
}
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
}
unix_listener auth-master {
group = fumail
mode = 0660
user = fumail
}
user = root
}
service dict {
unix_listener dict {
mode = 0600
user = fumail
}
}
service doveadm {
inet_listener {
port = 1337
}
}
service imap {
vsz_limit = 2 G
}
service lmtp {
inet_listener lmtp {
address = 127.0.0.1
port = 24
}
process_min_avail = 5
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
}
}
service managesieve {
vsz_limit = 2 G
}
service pop3 {
vsz_limit = 2 G
}
service quota-warning {
executable = script /usr/local/bin/quotawarning.py
unix_listener quota-warning {
mode = 0666
user = fumail
}
user = fumail
}
service replicator {
process_min_avail = 1
}
ssl_cert = .crt
ssl_key = .key
userdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
protocol imap {
imap_client_workarounds =
mail_plugins = " quota notify replication acl imap_acl quota imap_quota"
}
protocol pop3 {
mail_plugins = " quota notify replication quota"
pop3_uidl_format = %08Xu%08Xv
}
protocol sieve {
managesieve_logout_format = bytes ( in=%i : out=%o )
}
protocol lmtp {
deliver_log_format = from=<%f> msgid=%m action=<%$> subject=%s
mail_plugins = " quota notify replication sieve quota"
postmaster_address = postmaster@
quota_full_tempfail = yes
}
Re: [Dovecot] mdbox and folders containing both messages and sub-folders
On 8.4.2013, at 1.51, Rich Wales wrote: > When I use the mdbox format, there are a bunch of LIST and LSUB commands > whose responses mention INBOX.Trash with the \Noselect and \HasChildren > flags (the Trash folder for this particular account happens to contain > subfolders). Before trying to save a message in the Trash folder, the > client did a "SUBSCRIBE INBOX.Trash" (which succeeded) followed by a > "STATUS INBOX.Trash" (which failed with "NO Mailbox doesn't exist"). > > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > When I use the Maildir format, the raw log never mentions the \Noselect > flag in connection with the INBOX.Trash folder. Yeah, this is the difference between Dovecot's LAYOUT=fs and LAYOUT=maildir++ backends, which is also exposed to IMAP clients. With maildir++ there are no \Noselect folders possible. Whether that's a good or a bad thing I'm not entirely sure. I've been wondering earlier also if I should change the behavior somewhat, or make it configurable. > So, with the mdbox format, Larch did not CREATE the folder before trying > to SELECT it. And Dovecot was flagging INBOX.Trash with a \Noselect flag > (which was not the case with the Maildir format). I'd say this is a Larch bug. It's relying on some specific way that some IMAP servers work.
Re: [Dovecot] checkpassword protocol
> Hmm. The AUTH_PASSWORD wasn't really an intentional addition .. but > I guess it can stay there. Some 10 years ago that might not have been > such a good idea since there were still some systems where process > environment variables were readable to all users in the system, but > I doubt there exist such systems anymore (at least where people would > want to run Dovecot). Very optimistic assumption. Wouldn't it be safer to remove the password from the environment? Anyone using checkpassword should use FD 3 and 4 for this purpose. Environment variables and command line arguments are not safe to transport passwords. Heiko Heiko SchlichtingFreie Universität Berlin heiko.schlicht...@fu-berlin.de Zentraleinrichtung für Datenverarbeitung Telefon +49 30 838-54327 Fabeckstraße 32 Telefax +49 30 838454327 14195 Berlin
