Re: [Dovecot] [PATCH] Bad boundary check in client_find_namespace
Hi,
On 26.9.2011, at 19.56, Christoph Bussenius wrote:
>/* make sure two hierarchy separators aren't next to each others */
>for (p = storage_name+1; *p != '\0'; p++) {
> if (p[0] == ns->real_sep && p[-1] == ns->real_sep) {
> client_send_tagline(cmd, "NO Invalid mailbox name.");
> return NULL;
> }
>}
>
> The loop iterates over the string storage_name starting from its second
> byte. However, the string may be "". This is the case if you select
> the root of a namespace, like in "SELECT shared" or
> "SELECT shared/user1". In that case, the code will read past the end of
> the buffer from random memory until it finds a zero byte or a duplicate
> separator.
I finally managed to get this far in my mail backlog. :) Yes, that's a bug.
Fixed now slightly differently than you:
http://hg.dovecot.org/dovecot-2.0/rev/d406e376f8ee
Re: [Dovecot] 2.0.14 doveadm proxy list segfault
Hi, On 29.9.2011, at 20.21, Tom Pawlowski wrote: > Periodic segfaults with doveadm proxy list: > > root@director1:~) doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault (core dumped) Do you still get these? gdb backtrace would be helpful. It already says "core dumped", so all that should be needed is: gdb doveadm core bt full Also it shows connection to "ipc" server dying .. Does it also crash? Anything in Dovecot's error log?
Re: [Dovecot] Syncing shared mailboxes
Hi, On 12.12.2011, at 18.25, Knud Dombrowsky wrote: > while trying to sync the mailboxes of several users who use and share > their mailboxes dsync prints this message: > > dsync-local(): Warning: Subscriptions file > /home//Maildir/subscriptions: Removing > invalid entry: shared// > > The problem is: Every user has to subscribe the shared folder again > after each run of dsync... > Is dsync capable of syncing shared mailboxes? Are there any > preliminaries to do so? Sorry for late reply. This should be fixed in v2.1 nowadays.
Re: [Dovecot] doveadm index
On 21.12.2011, at 19.35, e-frog wrote: >> But some day doveadm index (and several other commands) should allow >> wildcards in mailbox names, so you could just do doveadm index -A '*' > > Actually I'm using doveadm index -A '*' in a daily cron job and it seems to > work for me. Yes, looks like it does. I changed the doveadm help output to make it clearer. > At least the timestamps of the dovecot.index.search and > dovecot.index.search.uids files equal or close to the cron job start time in > every folder. > > Is there a way to examine the search index? Doveadm dump doesn't seem to like > it. > > $: doveadm dump dovecot.index.search > $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though.
Re: [Dovecot] crash on shared folders if remote user has empty $home
On 22.12.2011, at 23.01, Peer Heinlein wrote: > d) If foo logs in, dovecot's exploring the shared folders given from the > non-existing user bar. But bar's entry in LDAP doesn't have $home any > more. imap crashes and closes the TCP-connection immediately. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/09f964850245
Re: [Dovecot] LSUB and subscribed folders
Hi, I've finally gotten to last year's mail backlog. :) On 22.12.2011, at 14.34, Peer Heinlein wrote: > User "bar" has some folders and one shared folder from "foo" if I use the > "LIST"-command, which should list AFAIK *all* folders: Yes. > Using the LSUB-command, which should only list subscribed folders I can see > some more shared folders. AFAIK the output of LSUB must be always less then > LIST: As Michael mentioned, LSUB can show some nonexistent mailboxes also that haven't been unsubscribed. > And why is /shared/foo/INBOX listed with "HasNoChildren"? It has no visible existing children (the LSUB entries didn't exist or no longer had necessary ACLs). > And, btw: Looks like's not possible to use the "doveadm"-command to get a > list of a user's complete folder list. Maybe "doveadm mailbox" should get > some extra parameters to geht all folders / only subscribe folders / only > shared folders: Yes, that would be nice .. but maybe this is enough for now: http://hg.dovecot.org/dovecot-2.1/rev/b600af017598
Re: [Dovecot] doveadm altmove fails to pack messages?
On 1.2.2012, at 19.59, Lauri Alanko wrote: > When I use doveadm altmove, I notice that it may result in some very small > storage files in either the primary or the alternate storage location. I'm > not quite sure, but it almost seems like every storage file that contains > messages to be moved is simply split into two parts, even if a part might > only contain a single message. > > This seems quite curious, since the idea of mdbox is after all to store as > many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda > can quite successfully append messages to existing storage files. So why > can't doveadm altmove? Is this the expected behavior, or is there something > wrong with my setup? Fixed: http://hg.dovecot.org/dovecot-2.0/rev/d83356f1fce6
Re: [Dovecot] force-resync fails to recover all messages in mdbox
On 31.1.2012, at 18.34, Lauri Alanko wrote: > Well, well, well. This is interesting. Back with the indices > created by dsync: > > $ doveadm fetch guid all | grep guid: | sort | uniq -c | sort -n | tail > 17 guid: 1b28b22d4b2ee2885b5b81221c41201d > 17 guid: 730c692395661dd62f82088804b85652 > 17 guid: 865e1537fddba6698e010d0b9dbddd02 .. http://hg.dovecot.org/dovecot-2.0/rev/4a0b7dec3a22 avoids force-resync deleting these duplicates. It also logs a warning about the duplicates. http://hg.dovecot.org/dovecot-2.1/rev/2500de8f1f51 implements mbox_md5=all setting which avoids creation of these duplicates in the first place. I thought about adding some duplicate detection also to dsync (or anywhere in its path), but I couldn't do it without impacting performance in normal operation. > The complexity and opaqueness of the mdbox format is a worrisome. > It would ease my mind quite a bit if there were a simple tool > that would just dump out the plain message contents that are > stored inside the storage files, without involving any of > dovecot's index machinery. Then I would at least know that > whatever happens, as long as the storage files stay intact, I can > always migrate my mails into some other format. By using Dovecot indexes you could use e.g. "doveadm fetch" to dump them. Also "doveadm dump" can dump the dbox files' metadata, but not the message contents themselves. It probably wouldn't be difficult to implement that though. Also alternatively you could build something based on http://dovecot.org/tools/mdbox-obfuscate.pl
Re: [Dovecot] Synchronization error in NFS
On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director
Re: [Dovecot] Performance of Maildir vs sdbox/mdbox
On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file > lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: > (proxy->data_input->eof) .. > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have > any > other ideas for what might be causing it ? The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS (30 secs). It still shouldn't have crashed of course, and that crash is already fixed in v2.1 (in the LMTP simplification change). Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout.
Re: [Dovecot] Fedora 16 configuration
On 7.2.2012, at 22.44, Cliff Hayes wrote: > Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth > attempts): rip=x.x.x.x, lip=x.x.x.x From http://wiki2.dovecot.org/WhyDoesItNotWork: Aborted login (no auth attempts) means that the client isn't even attempting to log in. Most likely you have disable_plaintext_auth=yes (default) and the client isn't configured to use SSL/TLS (or you've also set ssl=no).
Re: [Dovecot] openbsd and dovecot
On 8.2.2012, at 20.56, Maxim wrote:
>>> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1):
>>> getpwnam() failed: Operation not permitted
>> service auth {
>> user = root
>> }
>>
>> Does this make any difference? If it does help, there's a bug somewhere. If
>> it doesn't help, then there's something else weird going on.
>>
> No, it didn't help. Might it be that I need to edit any system files in /etc
> directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot
> file accordingly to enable kerberos authentication for mail users in Active
> Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to
> enable kerberos for dovecot. But I've no idea of how these edits should look
> like.
The problem is that auth worker process can't do getpwnam() call. Typically
that would read from /etc/passwd, but I don't know if in OpenBSD if you have
configured it to read from elsewhere (AD?). Anyway, I don't know why it would
give EPERM error since the process is running as root. You could try ktraceing
it to see what actually fails.
Re: [Dovecot] Possible broken indexer(lucene/solr)?
On 6.2.2012, at 23.26, Ingo Thierack wrote: > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. Lucene should work.. > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. You mean you deleted Solr index, so that it's empty? That should work too. Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user).
Re: [Dovecot] vsz_limit
On 8.2.2012, at 10.58, Héctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors?
Re: [Dovecot] Slightly more intelligent way of handling issues in sdbox?
On 7.2.2012, at 14.08, Mark Zealey wrote: >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > I've not actually tried this patch yet, but looking at it, it is perhaps > useful for the situation I described below when the index is corrupt. In this > case I am describing however, the not is NOT corrupt - it is simply an older > version (ie it only thinks there are the first 2 mails in the directory, not > the 3rd). This could happen for example when mails are being stored on > different storage than indexes; say for example you have 2 servers with > remote NFS stored mails but local indexes that rsync between the servers > every hour. You manually fail over one server to the other and you then have > a copy of the correct indexes but only from an hour ago. The mails are all > there on the shared storage but because the indexes are out of date, when a > new message comes in it will be automatically overwritten. I don't recommend using local indexes with dbox, since there is actual data loss if they're not up to date (flags, and with mdbox the user may have copied/moved the mail elsewhere). Still, better to catch this situation than not: http://hg.dovecot.org/dovecot-2.1/rev/09db0f7aa6ce >>> (speaking of which, it would be great if force-resync also rebuilt the >>> cache files if there are valid cache files around, rather than just doing >>> away with them) >> Well, ideally there shouldn't be so much corruption that this matters.. > > That's true, but in our experience we usually get corruption in batches > rather than a one-off occurrence. Our most common case is something like > this: Say for example there's an issue with the NFS server (assuming we are > storing indexes on there as well now) and so we have to killall -9 dovecot > processes or similar. In that case you get a number of corrupted indexes on > the server. Rebuilding the indexes generates an IO storm (say via lmtp or a > pop3 access); then the clients log in via imap and we have to re-read all the > messages to generate the cache files which is a second IO storm. If the > caches were rebuilt at least semi-intelligently (ie you could extract from > the cache files a list of things that had previously been cached) that would > reduce the effects of rare storage level issues such as this. Well, the decisions are now remembered: http://hg.dovecot.org/dovecot-2.1/rev/d8d214cc1936 That can't really be improved .. If nothing is deleted from cache, it might contain invalid data and doveadm force-resync wouldn't be doing its job right. If anything is added to cache, it would require reading and parsing the mail contents during rebuild, and that's not in any way better than letting the imap processes do it later when the mailbox isn't locked.
Re: [Dovecot] openbsd and dovecot
07.02.2012 0:27, Timo Sirainen пишет:
On 3.2.2012, at 15.15, User Max wrote:
Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1):
getpwnam() failed: Operation not permitted
service auth {
user = root
}
Does this make any difference? If it does help, there's a bug somewhere. If it
doesn't help, then there's something else weird going on.
No, it didn't help. Might it be that I need to edit any system files in
/etc directory? When I set up dovecot on debian I had to edit
/etc/pam.d/dovecot file accordingly to enable kerberos authentication
for mail users in Active Directory. Someone suggested that I edit
/etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've
no idea of how these edits should look like.
Re: [Dovecot] vsz_limit
On Feb 8, 2012, at 3:58 AM, Héctor Moreno Blanco wrote:
-snip-
> service imap-login {
> executable = /usr/local/dovecot/libexec/dovecot/imap-login
> inet_listener imap {
>address = *
>port = 143
> }
> inet_listener imaps {
>address = *
>port = 993
> }
> process_limit = 2048
> process_min_avail = 20
> user = vmail
> }
> service imap {
> executable = /usr/local/dovecot/libexec/dovecot/imap
> process_limit = 2048
> }
- snip, again -
>
> The problem is at the moment of maximum load of the system. We can see many
> imap sessions but there are also many imap-login processes, which is weird.
> Has it something to do with any parameter of our configuration? We think is
> something with the vsz_limit parameter. How can we tune it?
I'm not 100% sure but if this is the case but it looks like you setting
process_limit and process_min_avail in the imap-login service block is putting
dovecot in "High Security" mode. http://wiki2.dovecot.org/LoginProcess has good
details on that. In short, I suspect your settings are creating a new
imap-login process for each new IMAP connection. As I understand it, vsz_limit
is more important when running in "High Performance" mode where you have just 1
imap-login process that handles all new IMAP connections. That process can grow
quite large if you have a lot of simultaneous IMAP clients.
>
> Thank you very much in advanced.
>
> Kind regards.
>
>
> [cid:image001.gif@01CCE647.65B8FB30]
>
> Héctor Moreno Blanco
> División de Seguridad e Infraestructuras /
> Security and Infrastructures Division
>
> GMV
> Isaac Newton, 11
> P.T.M. Tres Cantos
> E-28760 Madrid
> Tel. +34 91 807 21 00
> Fax +34 91 807 21 99
> www.gmv.com
>
>
>
> [cid:image003.gif@01CCE647.65B8FB30]
>
>
>
> [cid:image004.gif@01CCE647.65B8FB30]
>
>
>
> [cid:image005.gif@01CCE647.65B8FB30]
>
>
>
>
>
>
> __
> This message including any attachments may contain confidential
> information, according to our Information Security Management System,
> and intended solely for a specific individual to whom they are addressed.
> Any unauthorised copy, disclosure or distribution of this message
> is strictly forbidden. If you have received this transmission in error,
> please notify the sender immediately and delete it.
>
> __
> Este mensaje, y en su caso, cualquier fichero anexo al mismo,
> puede contener informacion clasificada por su emisor como confidencial
> en el marco de su Sistema de Gestion de Seguridad de la
> Informacion siendo para uso exclusivo del destinatario, quedando
> prohibida su divulgacion copia o distribucion a terceros sin la
> autorizacion expresa del remitente. Si Vd. ha recibido este mensaje
> erroneamente, se ruega lo notifique al remitente y proceda a su borrado.
> Gracias por su colaboracion.
>
> __
>
Hope this helps,
David Warden
[Dovecot] vsz_limit
Hello Timo and dovecot teamwork,
We have dovecot 2.0.10 installed with this configuration:
# dovecot -n
# 2.0.10: /usr/local/dovecot/etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-92.1.18.el5 x86_64 Scientific Linux SL release 5.2 (Boron)
auth_cache_size = 500 k
auth_cache_ttl = 1000 secs
disable_plaintext_auth = no
first_valid_uid = 501
lock_method = dotlock
mail_fsync = always
mail_gid = vmail
mail_location = maildir:%h/Maildir
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = quota autocreate
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date
mmap_disable = yes
namespace {
inbox = yes
location =
prefix =
separator = .
type = private
}
namespace {
hidden = yes
inbox = no
list = no
location =
prefix = INBOX.
separator = .
type = private
}
namespace {
location = maildir:%h/Maildir/_backup
prefix = _backup.
separator = .
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf
driver = ldap
}
plugin/autocreate = Spam
plugin/autosubscribe = Spam
plugin/quota = maildir:_backup:ns=
plugin/quota_warning = storage=75%% quota-warning 75 %u
plugin/quota_warning2 = storage=90%% quota-warning 90 %u
plugin/sieve = %h/.dovecot.sieve
plugin/sieve_dir = %h/sieve
protocols = imap pop3 sieve
service anvil {
client_limit = 5000
}
service auth {
client_limit = 9000
unix_listener auth-master {
group = vmail
mode = 0700
user = vmail
}
user = root
vsz_limit = 2 G
}
service imap-login {
executable = /usr/local/dovecot/libexec/dovecot/imap-login
inet_listener imap {
address = *
port = 143
}
inet_listener imaps {
address = *
port = 993
}
process_limit = 2048
process_min_avail = 20
user = vmail
}
service imap {
executable = /usr/local/dovecot/libexec/dovecot/imap
process_limit = 2048
}
service managesieve-login {
executable = /usr/local/dovecot/libexec/dovecot/managesieve-login
inet_listener sieve {
address = *
port = 4190
}
inet_listener sieve_deprecated {
address = *
port = 12000
}
user = vmail
}
service managesieve {
executable = /usr/local/dovecot/libexec/dovecot/managesieve
}
service pop3-login {
executable = /usr/local/dovecot/libexec/dovecot/pop3-login
inet_listener pop3 {
address = *
port = 110
}
inet_listener pop3s {
address = *
port = 995
}
process_limit = 2048
process_min_avail = 20
user = vmail
}
service pop3 {
executable = /usr/local/dovecot/libexec/dovecot/pop3
process_limit = 2048
}
service quota-warning {
executable = script /usr/local/dovecot/bin/quota-warning.sh
unix_listener quota-warning {
group = vmail
mode = 0700
user = vmail
}
}
ssl_cert = fue rechazado automaticamente por
nuestro sistema :%n%r
rejection_subject = Rechazado: %s
sendmail_path = /usr/sbin/exim
}
protocol sieve {
managesieve_implementation_string = dovecot Pigeonhole
managesieve_logout_format = bytes ( in=%i : out=%o )
managesieve_max_line_length = 65536
}
The problem is at the moment of maximum load of the system. We can see many
imap sessions but there are also many imap-login processes, which is weird. Has
it something to do with any parameter of our configuration? We think is
something with the vsz_limit parameter. How can we tune it?
Thank you very much in advanced.
Kind regards.
[cid:image001.gif@01CCE647.65B8FB30]
Héctor Moreno Blanco
División de Seguridad e Infraestructuras /
Security and Infrastructures Division
GMV
Isaac Newton, 11
P.T.M. Tres Cantos
E-28760 Madrid
Tel. +34 91 807 21 00
Fax +34 91 807 21 99
www.gmv.com
[cid:image003.gif@01CCE647.65B8FB30]
[cid:image004.gif@01CCE647.65B8FB30]
[cid:image005.gif@01CCE647.65B8FB30]
__
This message including any attachments may contain confidential
information, according to our Information Security Management System,
and intended solely for a specific individual to whom they are addressed.
Any unauthorised copy, disclosure or distribution of this message
is strictly forbidden. If you have received this transmission in error,
please notify the sender immediately and delete it.
__
Este mensaje, y en su caso, cualquier fichero anexo al mismo,
puede contener informacion clasificada por su emisor como confidencial
en el marco de su Sistema de Gestion de Seguridad de la
Informacion siendo para uso exclusivo del destinatario, quedando
prohibida su divulgacion copia o distribucion a terceros sin la
autorizacion expresa del remitente. Si Vd. ha recibido este mensaje
erroneamente, se ruega lo
