Re: [Dovecot] BUG report: doveadm HEADER when concatenating with another Search key

[Timo Sirainen]

On 7.1.2014, at 20.49, mego...@inboxalias.com wrote:

> I'm pretty sure I've found a bug with doveadm when concatenating with 
> another search key.
> 
> While doveadm HEADER  works fine when not using additional 
> search keys, it doesn't work properly when concatenating with other 
> serarch keys - it then simply seems to ignore the additional HEADER  pattern> search key.
> 
> It's reproduceable, also with different user accounts on 2 different 
> productive systems. Testing it on latest dovecot-ee 2.2.9.2 and 2.2.5.5.

Finally fixed, happened only with fts-lucene: 
http://hg.dovecot.org/dovecot-2.2/rev/4b67da0b86a3

Re: [Dovecot] Bug Report

[Nick Edwards]

huh? You are logging a bug for antique version, if you grab latest
source, build and install it, and problem remains with current
version, THEN it is a bug, most devs would ignore you for such
sillyness otherwise

On 2/20/14, Jon Thompson  wrote:
> Since I'm obviously using version 2.2.5, which is apparently a supported
> version by dovecot, I'd still maintain that it should be submitted as a bug.
> I've also submitted the bug to Apple as well.
>
> Regardless, has anyone compiled a drop-in update that supports Apple Push
> Notifications, Sieve, and Open Directory? I'm trying to, but the first two
> are causing my configuration files to fail.
> --
> - Jon
>
>
>
>
> On Feb 17, 2014, at 5:21 PM, Reindl Harald  wrote:
>
>> since you are obviously using the binary shipped with
>> Mac OSX and packaged by Apple Inc. which is heavily
>> outdated while current dovecot is version 2.2.12
>> you may complain to Apple Inc.
>>
>> Am 18.02.2014 00:19, schrieb Jon Thompson:
>>> So when I try and create a sub mailbox in shared-folders, I get this
>>> error...
>>>
>>> imap(pid 56143 user -): Fatal: master: service(imap): child 56143
>>> killed with signal 11 (core dumps disabled)
>>>
>>> I've not been able to figure out how to get dovecot in OS X to core dump,
>>> so I haven't included that. The documentation on the web doesn't seem to
>>> apply to the version Apple ships.
>>>
>>> Here's dovecotd -n, with the ssl certs removed and the domain replaced
>>> with the word domain.
>>> # 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf
>>
>>
>

Re: [Dovecot] Bug Report

[Jon Thompson]

Since I’m obviously using version 2.2.5, which is apparently a supported 
version by dovecot, I’d still maintain that it should be submitted as a bug. 
I’ve also submitted the bug to Apple as well.

Regardless, has anyone compiled a drop-in update that supports Apple Push 
Notifications, Sieve, and Open Directory? I’m trying to, but the first two are 
causing my configuration files to fail.
-- 
- Jon




On Feb 17, 2014, at 5:21 PM, Reindl Harald  wrote:

> since you are obviously using the binary shipped with
> Mac OSX and packaged by Apple Inc. which is heavily
> outdated while current dovecot is version 2.2.12
> you may complain to Apple Inc.
> 
> Am 18.02.2014 00:19, schrieb Jon Thompson:
>> So when I try and create a sub mailbox in shared-folders, I get this error…
>> 
>> imap(pid 56143 user -): Fatal: master: service(imap): child 56143 killed 
>> with signal 11 (core dumps disabled)
>> 
>> I’ve not been able to figure out how to get dovecot in OS X to core dump, so 
>> I haven’t included that. The documentation on the web doesn’t seem to apply 
>> to the version Apple ships.
>> 
>> Here’s dovecotd -n, with the ssl certs removed and the domain replaced with 
>> the word domain.
>> # 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf
> 
> 

[Dovecot] Bug report

[Ramon]

I'm using release 2.2.12, during a copy operation i get segfault on imap 
process. I have full bt of the issue:


(gdb) bt
#0  0x7fcc521029cd in mail_cache_header_fields_read 
(cache=0x15c7210) at mail-cache-fields.c:369
#1  0x7fcc52100664 in mail_cache_open_and_verify (cache=0x15c7210) 
at mail-cache.c:497
#2  0x7fcc5210244d in mail_cache_register_get_list (cache=0x15c7210, 
pool=0x1573608, count_r=0x7fff7b3b2418) at mail-cache-fields.c:194
#3  0x7fcc520ef845 in index_mail_parse_header_register_all_wanted 
(mail=0x15d4b60, headers=) at index-mail-headers.c:170
#4  index_mail_parse_header_init (mail=0x15d4b60, headers=optimized out>) at index-mail-headers.c:230
#5  0x7fcc520efdb8 in index_mail_cache_parse_init (_mail=0x15d4b60, 
input=0x15ddb38) at index-mail-headers.c:376
#6  0x7fcc520bb163 in maildir_save_add (_ctx=0x15d3b20, 
tmp_fname=0x15735e8 "1392731650.M182940P5176.pecpop5", src_mail=0x0) at 
maildir-save.c:225
#7  0x7fcc520bb4b8 in maildir_save_begin (_ctx=0x15d3b20, 
input=0x15debb8) at maildir-save.c:415
#8  0x7fcc511e7360 in quota_save_begin (ctx=0x15d3b20, 
input=0x15debb8) at quota-storage.c:274
#9  0x7fcc520cc182 in mailbox_save_begin (ctx=0x7fff7b3b25a8, 
input=0x15debb8) at mail-storage.c:2025
#10 0x7fcc520c6242 in mail_storage_try_copy (ctx=0x15d3b20, 
mail=) at mail-copy.c:76
#11 mail_storage_copy (ctx=0x15d3b20, mail=) at 
mail-copy.c:101
#12 0x7fcc520b6916 in maildir_copy (ctx=0x15d3b20, mail=0x15cf300) 
at maildir-copy.c:148
#13 0x7fcc50dd7461 in notify_copy (ctx=0x15d3b20, mail=0x15cf300) at 
notify-storage.c:107
#14 0x7fcc511e7183 in quota_copy (ctx=0x15d3b20, mail=0x15cf300) at 
quota-storage.c:220
#15 0x7fcc520cc01f in mailbox_copy (_ctx=, 
mail=0x15cf300) at mail-storage.c:2133
#16 0x0040c63b in fetch_and_copy (cmd=0x1593810, move=false) at 
cmd-copy.c:70

#17 cmd_copy_full (cmd=0x1593810, move=false) at cmd-copy.c:123
#18 0x00416fbd in command_exec (cmd=0x1593810) at 
imap-commands.c:158
#19 0x004160e0 in client_command_input (cmd=0x1593810) at 
imap-client.c:780
#20 0x004161cd in client_command_input (cmd=0x1593810) at 
imap-client.c:841
#21 0x0041644d in client_handle_next_command (client=0x1592fb0) 
at imap-client.c:879

#22 client_handle_input (client=0x1592fb0) at imap-client.c:891
#23 0x004167bf in client_input (client=0x1592fb0) at 
imap-client.c:933

#24 0x7fcc51e1f336 in io_loop_call_io (io=0x158c1d0) at ioloop.c:388
#25 0x7fcc51e203a7 in io_loop_handler_run (ioloop=out>) at ioloop-epoll.c:220

#26 0x7fcc51e1f2a8 in io_loop_run (ioloop=0x157b720) at ioloop.c:412
#27 0x7fcc51dcdf73 in master_service_run (service=0x157b5b0, 
callback=) at master-service.c:566

#28 0x0041fd07 in main (argc=1, argv=0x157b390) at main.c:400

Here's config:

auth_debug = yes
auth_master_user_separator = *
auth_username_format =
default_vsz_limit = 512 M
disable_plaintext_auth = no
dotlock_use_excl = no
first_valid_gid = 101
first_valid_uid = 102
imap_id_send =
import_environment = TZ
info_log_path = /var/log/dovecot.log
log_path = /var/log/dovecot.log
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_fsync = always
mail_gid = 101
mail_location = maildir:%h/Maildir
mail_max_userip_connections = 50
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = quota expire fts fts_lucene mail_log notify
mail_uid = 102
maildir_broken_filename_sizes = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
mbox_write_locks = fcntl
mmap_disable = yes
namespace inbox {
   inbox = yes
   location =
   mailbox Drafts {
 special_use = \Drafts
   }
   mailbox Junk {
 special_use = \Junk
   }
   mailbox Sent {
 special_use = \Sent
   }
   mailbox "Sent Messages" {
 special_use = \Sent
   }
   mailbox Trash {
 special_use = \Trash
   }
   prefix = INBOX.
   separator = .
   subscriptions = yes
   type = private
}
passdb {
   args = /opt/dovecot-2.2.10/etc/dovecot/passwd.masterusers
   driver = passwd-file
   master = yes
}
passdb {
   args = /opt/dovecot-2.2.10/etc/dovecot/dovecot-ldap.conf
   driver = ldap
}
passdb {
   args = /opt/dovecot-2.2.10/etc/dovecot/dovecot-ldap-multiuser.conf
   driver = ldap
   master = yes
}
plugin {
   fts = lucene
   fts_lucene =
   mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename
   mail_log_fields = uid box msgid size
   quota = maildir
   quota_rule2 = INBOX.Trash:storage=+100M
}
protocols = imap pop3
service anvil {
   client_limit = 3000
}
service auth {
   client_limit = 4096
   unix_listener auth-userdb {
 mode = 0600
   }
}
service imap-login {
   inet_listener imap {
 address = 10.11.47.49
 port = 143
   }
   inet_listener imaps {
 address = 10.11.47.49
 port = 993
 ssl = yes
   }

Re: [Dovecot] Bug Report

[Reindl Harald]

since you are obviously using the binary shipped with
Mac OSX and packaged by Apple Inc. which is heavily
outdated while current dovecot is version 2.2.12
you may complain to Apple Inc.

Am 18.02.2014 00:19, schrieb Jon Thompson:
> So when I try and create a sub mailbox in shared-folders, I get this error…
> 
> imap(pid 56143 user -): Fatal: master: service(imap): child 56143 killed 
> with signal 11 (core dumps disabled)
> 
> I’ve not been able to figure out how to get dovecot in OS X to core dump, so 
> I haven’t included that. The documentation on the web doesn’t seem to apply 
> to the version Apple ships.
> 
> Here’s dovecotd -n, with the ssl certs removed and the domain replaced with 
> the word domain.
> # 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf




signature.asc
Description: OpenPGP digital signature

[Dovecot] Bug Report

[Jon Thompson]

So when I try and create a sub mailbox in shared-folders, I get this error…

imap(pid 56143 user -): Fatal: master: service(imap): child 56143 killed 
with signal 11 (core dumps disabled)

I’ve not been able to figure out how to get dovecot in OS X to core dump, so I 
haven’t included that. The documentation on the web doesn’t seem to apply to 
the version Apple ships.

Here’s dovecotd -n, with the ssl certs removed and the domain replaced with the 
word domain.
# 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf
# OS: Darwin 13.0.0 x86_64  hfs
aps_topic = com.apple.mail.XServer.b137086a-440f-45fe-89c4-8ddf798318a3
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = cram-md5 plain login apop digest-md5
auth_realms = office.domain.com
auth_socket_path = /var/run/dovecot/auth-userdb
auth_username_format = %n
auth_verbose = yes
debug_log_path = /Library/Logs/Mail/mail-debug.log
default_internal_user = _dovecot
default_login_user = _dovenull
disable_plaintext_auth = no
first_valid_gid = 6
first_valid_uid = 6
imap_id_log = *
imap_id_send = "name" * "version" *
imap_urlauth_submit_user = submit
info_log_path = /Library/Logs/Mail/mail-info.log
log_path = /Library/Logs/Mail/mail-err.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_access_groups = mail
mail_attribute_dict = file:/Library/Server/Mail/Data/attributes/attributes.dict
mail_debug = yes
mail_location = maildir:/Volumes/PromiseRAID/Library/Server/Mail/Data/mail/%u
mail_log_prefix = "%s(pid %p user %u): "
mail_plugins = quota zlib acl fts fts_sk
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
mdbox_rotate_size = 200 M
namespace acl-mailboxes {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/mail/users/%%u:INDEX=/Library/Server/Mail/Data/mail/shared/%%u
  prefix = shared.%%u.
  separator = .
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}
namespace list-archives {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/listserver/messages/archive/lists/%%u:INDEX=/Library/Server/Mail/Data/listserver/messages/archive/shared/%%u
  prefix = archives.%%u.
  separator = .
  subscriptions = no
  type = shared
}
namespace shared-folders {
  list = yes
  location = maildir:/Volumes/PromiseRAID/Library/Server/Mail/Data/Shared
  prefix = shared-folders.
  separator = .
  subscriptions = no
  type = shared
}
passdb {
  driver = od
}
passdb {
  args = /Library/Server/Mail/Config/dovecot/submit.passdb
  driver = passwd-file
}
plugin {
  acl = vfile:/Library/Server/Mail/Config/dovecot/global-acls:cache_secs=300
  acl_anyone = allow
  acl_shared_dict = file:/Library/Server/Mail/Data/shared/shared-mailboxes
  fts = sk
  quota = maildir:User quota
  quota_warning = storage=100%% quota-exceeded %u
  sieve = /Library/Server/Mail/Data/rules/%u/dovecot.sieve
  sieve_dir = /Library/Server/Mail/Data/rules/%u
  stats_refresh = 30 secs
  stats_track_cmds = yes
}
postmaster_address = postmas...@office.domain.com
protocols = imap pop3 lmtp sieve
quota_full_tempfail = yes
service auth {
  extra_groups = _keytabusers
  idle_kill = 15 mins
  unix_listener auth-userdb {
user = _dovecot
  }
}
service dns_client {
  unix_listener dns-client {
mode = 0600
  }
}
service imap-login {
  inet_listener imap {
port = 143
  }
  inet_listener imaps {
port = 993
ssl = yes
  }
  service_count = 0
}
service imap {
  client_limit = 5
  process_limit = 200
  service_count = 0
}
service indexer-worker {
  user = _dovecot
}
service lmtp {
  unix_listener lmtp {
mode = 0600
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
port = 995
ssl = yes
  }
}
service pop3 {
  client_limit = 5
  process_limit = 200
  service_count = 0
}
service quota-exceeded {
  executable = script 
/Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-exceeded.sh
  unix_listener quota-exceeded {
group = mail
mode = 0660
user = _dovecot
  }
  user = _dovecot
}
service quota-warning {
  executable = script 
/Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-warning.sh
  unix_listener quota-warning {
group = mail
mode = 0660
user = _dovecot
  }
  user = _dovecot
}
service stats {
  fifo_listener stats-mail {
mode = 0600
user = _dovecot
  }
}
ssl = required
ssl_ca =
ssl_cert = 
ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL:!ADH:!eNULL
ssl_key = 
ssl_k

Re: [Dovecot] BUG report: doveadm HEADER when concatenating with another Search key

[Joseph Tam]

So E.g. When doing
doveadm fetch -u exampleuser hdr SUBJECT Silvester HEADER
Return-Path someu...@example.com

it initially outputs 13 headers (same as when using with SUBJECT Silvester
only) , when piping it to "grep -i 'Return-Path'" it even found only 3 out
of 13 headers containing a Return-Path at all, 2 of them different than
someu...@example.com - so SAME behaviour as with search.


I tried using logical conjections of header searches and it works with
my installation of dovecot, so either it's a bug introduced later than
2.1.16, or perhaps it from the different search engine you use (I use
the default that comes with dovecot -- squat?).

Sorry, can't help you out further.

Joseph Tam 

Re: [Dovecot] BUG report: doveadm HEADER when concatenating with another Search key

[megodin]

Hey Joseph!

Thanks for testing.

> I couldn't recreate this with 2.1.16.  Also, my version of "doveadm
> search" does not accept missing header patterns.  Is this corroborated
> by dumping out the header with this same search query?
>
>   doveadm fetch -u exampleuser hdr SUBJECT Silvester \
>   HEADER Return-Path someuser at example.com

Yes, it's same phenomenon when dumping with fetch.
Of course you can only test this properly when you really have some mails 
with corresponding existing Header fields. (You can also try out other 
HEADER fields, like Reply-To, References, X-Spam-Score or whatever - just 
look at some mail headers of your mails and try it with existing fields.)

So E.g. When doing
doveadm fetch -u exampleuser hdr SUBJECT Silvester HEADER 
Return-Path someu...@example.com

it initially outputs 13 headers (same as when using with SUBJECT Silvester 
only) , when piping it to "grep -i 'Return-Path'" it even found only 3 out 
of 13 headers containing a Return-Path at all, 2 of them different than 
someu...@example.com - so SAME behaviour as with search.

Maybe also relevant, as mail format I'm using sdbox (on a larger user base 
of 40.000 people).

--
(relevant part) of dovecot -n

# 2.2.5.5 (e5350245ad9b): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-358.23.2.el6.x86_64 x86_64 Red Hat Enterprise Linux 
Server release 6.4 (Santiago) nfs
auth_cache_size = 100 M
auth_cache_ttl = 10 mins
auth_master_user_separator = *
auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = yes
disable_plaintext_auth = no
import_environment = TZ DEBUG_OUTOFMEM DOVECOT_HOSTDOMAIN
listen = *
lock_method = dotlock
mail_fsync = always
mail_home = /mnt/nfs/vmails/%u
mail_location = sdbox:/mnt/nfs/vmails/%u/mail:ALT=/mnt/archivnfs/vmails/%u/mail
mail_plugins = " quota fts fts_lucene"
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
copy include variables body enotify environment mailbox date ihave
mmap_disable = yes
service imap-login {
   process_limit = 200
   process_min_avail = 8
   service_count = 0
   vsz_limit = 512 M
}
service imap {
   client_limit = 1
   process_limit = 1
}
service pop3-login {
   process_limit = 200
   process_min_avail = 8
}
service pop3 {
   process_limit = 1
}
verbose_proctitle = yes
protocol lda {
   mail_plugins = " quota fts fts_lucene sieve"
}
protocol imap {
   imap_client_workarounds = tb-extra-mailbox-sep
   mail_max_userip_connections = 50
   mail_plugins = " quota fts fts_lucene imap_quota"
}
_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] BUG report: doveadm HEADER when concatenating with another Search key

[Joseph Tam]

On Wed, 8 Jan 2014, dovecot-requ...@dovecot.org wrote:


doveadm search -u exampleuser SUBJECT Silvester
(outputs correctly 10 matches)

...

doveadm search -u exampleuser SUBJECT Silvester HEADER Return-Path
someu...@example.com
(gives exactly the same output - but this is WRONG, as the given Subject
AND Return-Path is really only found in 1 Message - uid 15697)


I couldn't recreate this with 2.1.16.  Also, my version of "doveadm
search" does not accept missing header patterns.  Is this corroborated
by dumping out the header with this same search query?

doveadm fetch -u exampleuser hdr SUBJECT Silvester \
HEADER Return-Path someu...@example.com

Joseph Tam 

[Dovecot] BUG report: doveadm HEADER when concatenating with another Search key

[megodin]

Hey there,

I'm pretty sure I've found a bug with doveadm when concatenating with 
another search key.

While doveadm HEADER  works fine when not using additional 
search keys, it doesn't work properly when concatenating with other 
serarch keys - it then simply seems to ignore the additional HEADER  search key.

It's reproduceable, also with different user accounts on 2 different 
productive systems. Testing it on latest dovecot-ee 2.2.9.2 and 2.2.5.5.

I think it's best explained with showing some examples:


doveadm search -u exampleuser SUBJECT Silvester
(outputs correctly 10 matches)

5a09f50926daef51b50e25ab05f8 1106
5a09f50926daef51b50e25ab05f8 1107
5a09f50926daef51b50e25ab05f8 1579
5a09f50926daef51b50e25ab05f8 4092
5a09f50926daef51b50e25ab05f8 5330
5a09f50926daef51b50e25ab05f8 9773
5a09f50926daef51b50e25ab05f8 11460
5a09f50926daef51b50e25ab05f8 12718
5a09f50926daef51b50e25ab05f8 15653
5a09f50926daef51b50e25ab05f8 15697


doveadm search -u exampleuser SUBJECT Silvester HEADER Return-Path 
someu...@example.com
(gives exactly the same output - but this is WRONG, as the given Subject 
AND Return-Path is really only found in 1 Message - uid 15697)

5a09f50926daef51b50e25ab05f8 1106
5a09f50926daef51b50e25ab05f8 1107
5a09f50926daef51b50e25ab05f8 1579
5a09f50926daef51b50e25ab05f8 4092
5a09f50926daef51b50e25ab05f8 5330
5a09f50926daef51b50e25ab05f8 9773
5a09f50926daef51b50e25ab05f8 11460
5a09f50926daef51b50e25ab05f8 12718
5a09f50926daef51b50e25ab05f8 15653
5a09f50926daef51b50e25ab05f8 15697


doveadm search -u exampleuser HEADER Return-Path someu...@example.com
(showing that Return-Path as standalone search key is working fine - note 
that message uid 15697 with subject "Silvester" is also found of course.
I manually checked all Mails showing up here, they all contained the 
Return-Path "someu...@example.com" as they should):

5a09f50926daef51b50e25ab05f8 14080
5a09f50926daef51b50e25ab05f8 14814
5a09f50926daef51b50e25ab05f8 14815
5a09f50926daef51b50e25ab05f8 14820
5a09f50926daef51b50e25ab05f8 14825
5a09f50926daef51b50e25ab05f8 14830
5a09f50926daef51b50e25ab05f8 14832
5a09f50926daef51b50e25ab05f8 14833
5a09f50926daef51b50e25ab05f8 15427
5a09f50926daef51b50e25ab05f8 15428
5a09f50926daef51b50e25ab05f8 15430
5a09f50926daef51b50e25ab05f8 15431
5a09f50926daef51b50e25ab05f8 15697


Further testing with:

doveadm search -u exampleuser HEADER Return-Path 
completegarbageforsurenotinheaders
(Correctly no output was given.)

doveadm search -u exampleuser SUBJECT Silvester HEADER Return-Path 
completegarbageforsurenotinheaders
(again ignoring the additional HEADER and showing all uids which just 
match the SUBJECT)

5a09f50926daef51b50e25ab05f8 1106
5a09f50926daef51b50e25ab05f8 1107
5a09f50926daef51b50e25ab05f8 1579
5a09f50926daef51b50e25ab05f8 4092
5a09f50926daef51b50e25ab05f8 5330
5a09f50926daef51b50e25ab05f8 9773
5a09f50926daef51b50e25ab05f8 11460
5a09f50926daef51b50e25ab05f8 12718
5a09f50926daef51b50e25ab05f8 15653
5a09f50926daef51b50e25ab05f8 15697


Moreove, when playing with HEADER Reply-To it shows some strange 
behaviour, note the differences with the ":" :

doveadm search -u exampleuser SUBJECT blubbs HEADER Reply-to 
ka...@example.com
f64cf50926daef51b50e25ab05f8 2231
f64cf50926daef51b50e25ab05f8 2781
5a09f50926daef51b50e25ab05f8 13620
5a09f50926daef51b50e25ab05f8 15763

doveadm search -u exampleuser SUBJECT blubbs HEADER Reply-to: 
ka...@example.com
f64cf50926daef51b50e25ab05f8 2231
5a09f50926daef51b50e25ab05f8 13620


Can you please have a look at this and fix it?

Thanks in advance,
Megodin

--
mailstorage @ NFS with nfsvers=3 (NetApp)

# 2.2.5.5 (e5350245ad9b): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-358.23.2.el6.x86_64 x86_64 Red Hat Enterprise Linux 
Server release 6.4 (Santiago) nfs
import_environment = TZ DEBUG_OUTOFMEM DOVECOT_HOSTDOMAIN

# 2.2.9.2 (8d71ef248a6e): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-358.23.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) 
nfs
_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] bug report: doveadm assertion failure "mail_index_map_lookup_seq_range"

[Timo Sirainen]

On 6.12.2013, at 15.33, email-from-bobby  wrote:

> A negated subject search on an empty folder causes an assertion failure in 
> doveadm. To reproduce:
> 
> bash-3.2# doveadm mailbox create -u ephraim "Freshly Empty"
> bash-3.2# doveadm search -u ephraim mailbox "Freshly Empty" NOT subject 
> '***JUNK MAIL***'
> doveadm(ephraim): Panic: file mail-index-map.c: line 548 
> (mail_index_map_lookup_seq_range): assertion failed: (first_uid <= last_uid\
> )

Fixed: http://hg.dovecot.org/dovecot-2.2/rev/d837bd49ddee

> bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd -n
> # 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf

But note that if you build Dovecot yourself, you lose several Apple-patched 
functionality, at least unless you add the patches yourself (AFAIK not yet 
publicly available).

Re: [Dovecot] bug report: doveadm assertion failure "mail_index_map_lookup_seq_range"

[Ephraim Vishniac]

Since reporting this (from the wrong account, sorry), I've tried to 
investigate whether the problem exists in 2.2.9. While building 2.2.9 
was no problem, I balked at actually installing it for fear of causing 
trouble with future system updates. (Or current operation, for that matter.)


Could somebody please check whether this is problem is already fixed?

On 12/6/13, 8:33 AM, email-from-bobby wrote:
A negated subject search on an empty folder causes an assertion 
failure in doveadm. To reproduce:


bash-3.2# doveadm mailbox create -u ephraim "Freshly Empty"
bash-3.2# doveadm search -u ephraim mailbox "Freshly Empty" NOT 
subject '***JUNK MAIL***'
doveadm(ephraim): Panic: file mail-index-map.c: line 548 
(mail_index_map_lookup_seq_range): assertion failed: (first_uid <= 
last_uid\

)
doveadm(ephraim): Error: Raw backtrace: 2 
libdovecot.0.dylib  0x000100347463 
default_fatal_handler + 486 -> 3 \
  libdovecot.0.dylib  0x0001003472c0 
default_fatal_handler + 67 -> 4 libdovecot.0.dylib  
0x000\
100347705 i_fatal + 0 -> 5 libdovecot-storage.0.dylib  
0x0001002a799f mail_index_map_lookup_seq_range + 309 -> 6  \
 libdovecot-storage.0.dylib  0x0001002b6d4f 
mail_index_lookup_seq_range + 12 -> 7 
lib20_fts_plugin.so \
0x0001003fe325 fts_search_lookup + 3307 -> 8 
lib20_fts_plugin.so 0x0001003fddfb 
fts_search_lookup + 1985 -\
> 9   lib20_fts_plugin.so 0x0001003fd6e2 
fts_search_lookup + 168 -> 10 lib20_fts_plugin.so 0x00\
01003ff148 fts_mailbox_allocated + 1139 -> 11 
doveadm 0x0001001dd3d2 
doveadm_mail_iter_init + 1\
99 -> 12  doveadm 0x0001001df3ce cmd_search_run + 138 -> 13 
doveadm 0x0\
001001d94f9 doveadm_mail_next_user + 196 -> 14 
doveadm 0x0001001d9ad1 
doveadm_mail_try_run + 77\
6 -> 15  doveadm 0x0001001e1a09 main + 
1150 -> 16  libdyld.dylib 0x7fff915\

ba5fd start + 1 -> 17  ??? 0x0009 0x0 + 9
Abort trap: 6
bash-3.2#

A positive search is no problem:

bash-3.2# doveadm search -u ephraim mailbox "Freshly Empty" subject 
'***JUNK MAIL***'

bash-3.2#

Version and configuration info:

bash-3.2# 
/Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd --version

2.2.5
bash-3.2# 
/Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd -n

# 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf
# OS: Darwin 13.0.0 x86_64  hfs
auth_mechanisms = cram-md5 login apop plain digest-md5
auth_realms = mail.vishniac.com
auth_socket_path = /var/run/dovecot/auth-userdb
auth_username_format = %n
debug_log_path = /Library/Logs/Mail/mail-debug.log
default_internal_user = _dovecot
default_login_user = _dovenull
disable_plaintext_auth = no
first_valid_gid = 6
first_valid_uid = 6
hostname = mail.vishniac.com
imap_id_log = *
imap_id_send = "name" * "version" *
imap_urlauth_submit_user = submit
info_log_path = /Library/Logs/Mail/mail-info.log
log_path = /Library/Logs/Mail/mail-err.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_access_groups = mail
mail_attribute_dict = 
file:/Library/Server/Mail/Data/attributes/attributes.dict

mail_location = maildir:/Library/Server/Mail/Data/mail/users/%u
mail_log_prefix = "%s(pid %p user %u): "
mail_plugins = quota zlib acl fts fts_sk
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational \
regex imap4flags copy include variables body enotify environment 
mailbox date ihave

mdbox_rotate_size = 200 M
namespace acl-mailboxes {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/mail/users/%%u:INDEX=/Library/Server/Mail/Data/mail/shared/%%u

  prefix = shared.%%u.
  separator = .
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
namespace list-archives {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/listserver/messages/archive/lists/%%u:INDEX=/Library/Server/Mail/Data/listserver/mess\

ages/archive/shared/%%u
  prefix = archives.%%u.
  separator = .
  subscriptions = no
  type = shared
}
passdb {
  driver = od
}
passdb {
  args = /Library/Server/Mail/Config/dovecot/submit.passdb
  driver = passwd-file
}
plugin {
  acl = 
vfile:/Library/Server/Mail/Config/dovecot/global-acls:cache_secs=300
  acl_shared_dict = 
file:/Library/Server/Mail/Data/shared/shared-mailboxes

  fts = sk
  quota = maildir:User quota
  quota_warning = storage=100%% quota-exceeded %u
  sieve = /Library/Server/Mail/Data/rules/%u/dove

[Dovecot] bug report: doveadm assertion failure "mail_index_map_lookup_seq_range"

[email-from-bobby]

A negated subject search on an empty folder causes an assertion failure 
in doveadm. To reproduce:


bash-3.2# doveadm mailbox create -u ephraim "Freshly Empty"
bash-3.2# doveadm search -u ephraim mailbox "Freshly Empty" NOT subject 
'***JUNK MAIL***'
doveadm(ephraim): Panic: file mail-index-map.c: line 548 
(mail_index_map_lookup_seq_range): assertion failed: (first_uid <= last_uid\

)
doveadm(ephraim): Error: Raw backtrace: 2   
libdovecot.0.dylib  0x000100347463 
default_fatal_handler + 486 -> 3 \
  libdovecot.0.dylib  0x0001003472c0 
default_fatal_handler + 67 -> 4   libdovecot.0.dylib  0x000\
100347705 i_fatal + 0 -> 5   libdovecot-storage.0.dylib  
0x0001002a799f mail_index_map_lookup_seq_range + 309 -> 6  \
 libdovecot-storage.0.dylib  0x0001002b6d4f 
mail_index_lookup_seq_range + 12 -> 7   
lib20_fts_plugin.so \
0x0001003fe325 fts_search_lookup + 3307 -> 8   
lib20_fts_plugin.so 0x0001003fddfb fts_search_lookup 
+ 1985 -\
> 9   lib20_fts_plugin.so 0x0001003fd6e2 
fts_search_lookup + 168 -> 10  lib20_fts_plugin.so 0x00\
01003ff148 fts_mailbox_allocated + 1139 -> 11  
doveadm 0x0001001dd3d2 
doveadm_mail_iter_init + 1\
99 -> 12  doveadm 0x0001001df3ce 
cmd_search_run + 138 -> 13  doveadm 0x0\
001001d94f9 doveadm_mail_next_user + 196 -> 14  
doveadm 0x0001001d9ad1 
doveadm_mail_try_run + 77\
6 -> 15  doveadm 0x0001001e1a09 main + 
1150 -> 16  libdyld.dylib   0x7fff915\
ba5fd start + 1 -> 17  ??? 
0x0009 0x0 + 9

Abort trap: 6
bash-3.2#

A positive search is no problem:

bash-3.2# doveadm search -u ephraim mailbox "Freshly Empty" subject 
'***JUNK MAIL***'

bash-3.2#

Version and configuration info:

bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd 
--version

2.2.5
bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/sbin/dovecotd -n
# 2.2.5: /Library/Server/Mail/Config/dovecot/dovecot.conf
# OS: Darwin 13.0.0 x86_64  hfs
auth_mechanisms = cram-md5 login apop plain digest-md5
auth_realms = mail.vishniac.com
auth_socket_path = /var/run/dovecot/auth-userdb
auth_username_format = %n
debug_log_path = /Library/Logs/Mail/mail-debug.log
default_internal_user = _dovecot
default_login_user = _dovenull
disable_plaintext_auth = no
first_valid_gid = 6
first_valid_uid = 6
hostname = mail.vishniac.com
imap_id_log = *
imap_id_send = "name" * "version" *
imap_urlauth_submit_user = submit
info_log_path = /Library/Logs/Mail/mail-info.log
log_path = /Library/Logs/Mail/mail-err.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_access_groups = mail
mail_attribute_dict = 
file:/Library/Server/Mail/Data/attributes/attributes.dict

mail_location = maildir:/Library/Server/Mail/Data/mail/users/%u
mail_log_prefix = "%s(pid %p user %u): "
mail_plugins = quota zlib acl fts fts_sk
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational \
regex imap4flags copy include variables body enotify environment mailbox 
date ihave

mdbox_rotate_size = 200 M
namespace acl-mailboxes {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/mail/users/%%u:INDEX=/Library/Server/Mail/Data/mail/shared/%%u

  prefix = shared.%%u.
  separator = .
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
namespace list-archives {
  list = children
  location = 
maildir:/Library/Server/Mail/Data/listserver/messages/archive/lists/%%u:INDEX=/Library/Server/Mail/Data/listserver/mess\

ages/archive/shared/%%u
  prefix = archives.%%u.
  separator = .
  subscriptions = no
  type = shared
}
passdb {
  driver = od
}
passdb {
  args = /Library/Server/Mail/Config/dovecot/submit.passdb
  driver = passwd-file
}
plugin {
  acl = 
vfile:/Library/Server/Mail/Config/dovecot/global-acls:cache_secs=300

  acl_shared_dict = file:/Library/Server/Mail/Data/shared/shared-mailboxes
  fts = sk
  quota = maildir:User quota
  quota_warning = storage=100%% quota-exceeded %u
  sieve = /Library/Server/Mail/Data/rules/%u/dovecot.sieve
  sieve_dir = /Library/Server/Mail/Data/rules/%u
  stats_refresh = 30 secs
  stats_track_cmds = yes
}
postmaster_address = postmas...@vishniac.com
protocols = imap pop3 lmtp sieve
quota_full_tempfail = yes
service auth {
  extra_groups = _keytabusers
  idle_kill = 15 mins
  unix_listener auth-userdb {
  

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[megodin]

> If doveadm says it's invalid UTF-8, it's invalid UTF-8. I guess your 
> terminal isn't actually using UTF-8 then, but something else. ("locale" 
> output should say something about UTF-8.) I guess doveadm could also 
> automatically translate parameters to UTF-8, but that's a bit annoying 
> to implement.

You were absolutely right. After thorough testing I could make up the 
problem to the SSH-Client PuTTY I was using on a Windows machine while 
testing. The default "remote character setting" is 
"ISO-8859-1:1998 (Latin-1, West Europe)", when re-setting to "UTF-8", 
opening a new shell and testing the "doveadm mailbox rename ..." with 
german umlauts just works fine then.
(Just for the sake of completeness, the "locale" settings were set to 
(LANG=de_DE.utf-8) globally in /etc/sysconfig/i18n per default.)


> The problem here is that * is expanded by your shell, not doveadm. 
> And it expands into Tr&-AOQ-sh as it's in the filesystem, but that's 
> only the mUTF-7 encoding of it. The UTF-8 version of the name is 
> Tr&AOQ-sh. So doveadm only sees that you attempted to resync a 
> nonexistent mailbox. Using '*' with quotes would work, since doveadm 
> would do the expansion then.

Thanks for pointing that * / '*' issue out.

I now unterstand that the "doevadm mailbox rename" converts the input 
to UTF _before_ applying it in the filesystem.

Now it makes sense that 
doveadm mailbox rename -u user 'Trash' 'Tr&AOQ-sh'
must be expanded to Tr&-AOQ-sh. The "-" character directly after the 
"&"in Tr&-AOQ-sh comes from a special mUTF-Specification (as stated in 
RFC 3501, section 5.1.3):

"In modified UTF-7, printable US-ASCII characters, except for "&",
represent themselves; that is, characters with octet values 0x20-0x25
and 0x27-0x7e.  The character "&" (0x26) is represented by the
two-octet sequence "&-"."

So e.g. if I wanted a german umlaut to be encoded in the filesystem, 
I must enter it directly into dovedm instead of the UTF encoded value.

One small point left...

> The UTF-8 version of the name is Tr&AOQ-sh
Just for understanding - "Tr&AOQ-sh"  is IMHO UTF-7, not UTF-8. 
Accordingly to what stated before, "Tr&-AOQ-sh" and "Tr&AOQ-sh" are 
encoded both the same (UTF-7), the first seen in clients as 
"Tr&AOQ-sh" and the second as "Träsh".

Thanks for all your help!
Megodin



---
Alle Postfächer an einem Ort. Jetzt wechseln und E-Mail-Adresse mitnehmen! 
http://email.freenet.de/basic/Informationen

_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[Timo Sirainen]

On 23.9.2013, at 16.52, mego...@inboxalias.com wrote:

>> I suspect your shell is breaking the 8bit chars into invalid UTF-8. I
>> just tried and it works fine with me.
> 
> Sorry, we got confusion here... apparently my (lousy) freemail provider 
> which I use to post on this mailinglist doesn't send some special 
> characters etc. unaltered exactly as I typed  - so I made a console 
> screenshot of the doveadm rename problem to make things clear.
> Please have a look here, it will reveal the doveadm rename problem 
> (and that my shell is working fine): 
> http://i.imgur.com/08wKji4.png

If doveadm says it's invalid UTF-8, it's invalid UTF-8. I guess your terminal 
isn't actually using UTF-8 then, but something else. ("locale" output should 
say something about UTF-8.) I guess doveadm could also automatically translate 
parameters to UTF-8, but that's a bit annoying to implement.

> 
>>> To get things worse, when renaming the mailfolder in Thunderbird to 
>>> "geschäftliches" it gets correctly written in the filesystem as 
>>> "gesch&AOQ-ftliches"- but afterwards it's NOT found anymore in the 
>>> dovecot index, e.g. when trying doveadm force-resync -u user it is 
>>> not listed anymore (untilback-renaming in fs to the name it was 
>>> indexed before).
> 
>> Folders aren't indexed by default (mailbox_list_index=no) and
>> force-resync doesn't affect them. Are you saying
>> "doveadm mailbox list -u user" doesn't show some folder? I'm not
>> aware of any bugs related to this.
> 
> No, the "doveadm mailbox list -u user" works just fine.
> 
> BUT - I'm still wondering about the resync issue - As I understand the 
> "doveadm force-resync -u user *" command rebuilds the index for the 
> MAILS within the mailfolders? If so, how can the resync be done if the 
> mailfolder itself is not found by doveadm anymore?

With mailbox_list_index=no there is no way a folder isn't found by doveadm.

>> I'd need to get some more specifics. Show show exists in filesystems
>> and what commands don't produce expected results (and what they 
>> produce instead).
> 
> For clearing up my point, I made another console screenshot with 
> red markings to point out the problem (bug). 
> Please have another look here:
> http://i.imgur.com/icPfys9.png

The problem here is that * is expanded by your shell, not doveadm. And it 
expands into Tr&-AOQ-sh as it's in the filesystem, but that's only the mUTF-7 
encoding of it. The UTF-8 version of the name is Tr&AOQ-sh. So doveadm only 
sees that you attempted to resync a nonexistent mailbox. Using '*' with quotes 
would work, since doveadm would do the expansion then.

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[megodin]

Hi there Timo,
first of all thank you for taking your time for support!

> Forget about mUTF-7 with Dovecot. It's an IMAP-protocol relic, and
> Dovecot is trying to get rid of it in as many places as possible. 
> Although by default the filesystem users mUTF-7, but that can also 
> be changed with a setting.

Thanks for clarifying this.


> I suspect your shell is breaking the 8bit chars into invalid UTF-8. I
> just tried and it works fine with me.

Sorry, we got confusion here... apparently my (lousy) freemail provider 
which I use to post on this mailinglist doesn't send some special 
characters etc. unaltered exactly as I typed  - so I made a console 
screenshot of the doveadm rename problem to make things clear.
Please have a look here, it will reveal the doveadm rename problem 
(and that my shell is working fine): 
http://i.imgur.com/08wKji4.png



> > To get things worse, when renaming the mailfolder in Thunderbird to 
> > "geschäftliches" it gets correctly written in the filesystem as 
> > "gesch&AOQ-ftliches"- but afterwards it's NOT found anymore in the 
> > dovecot index, e.g. when trying doveadm force-resync -u user it is 
> > not listed anymore (untilback-renaming in fs to the name it was 
> > indexed before).

> Folders aren't indexed by default (mailbox_list_index=no) and
> force-resync doesn't affect them. Are you saying
> "doveadm mailbox list -u user" doesn't show some folder? I'm not
> aware of any bugs related to this.

No, the "doveadm mailbox list -u user" works just fine.

BUT - I'm still wondering about the resync issue - As I understand the 
"doveadm force-resync -u user *" command rebuilds the index for the 
MAILS within the mailfolders? If so, how can the resync be done if the 
mailfolder itself is not found by doveadm anymore?

> I'd need to get some more specifics. Show show exists in filesystems
> and what commands don't produce expected results (and what they 
> produce instead).

For clearing up my point, I made another console screenshot with 
red markings to point out the problem (bug). 
Please have another look here:
http://i.imgur.com/icPfys9.png


Thanks for your support,
Megodin





---
Alle Postfächer an einem Ort. Jetzt wechseln und E-Mail-Adresse mitnehmen! 
http://email.freenet.de/basic/Informationen

_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[Timo Sirainen]

On 20.9.2013, at 17.42, mego...@inboxalias.com wrote:

> So when using "doveadm rename" trying to rename a folder with special 
> characters e.g. german umlauts like "ä" it seems to create the folder 
> wrongly in the filesystem.
> 
> Example: When I want to rename a users folder 'geschaftliches' to 
> 'gesch&AOQ-ftliches' ("&AOQ-" is the correct mUTF-7 encoding for "ä") using 
> the command

Forget about mUTF-7 with Dovecot. It's an IMAP-protocol relic, and Dovecot is 
trying to get rid of it in as many places as possible. Although by default the 
filesystem users mUTF-7, but that can also be changed with a setting.

> (Apparently "doveadm rename" supports only UTF-8 which shows up when trying 
> to rename the folder with 
> "doveadm mailbox rename -u user 'geschäftliches'" I get the error message 
> "Fatal: Mailbox name not valid UTF-8: geschäftliches")

I suspect your shell is breaking the 8bit chars into invalid UTF-8. I just 
tried and it works fine with me.

> To get things worse, when renaming the mailfolder in Thunderbird to 
> "geschäftliches" it gets correctly written in the filesystem as 
> "gesch&AOQ-ftliches" - but afterwards it's NOT found anymore in the dovecot 
> index, e.g. when trying 
> doveadm force-resync -u user
> it is not listed anymore (until back-renaming in fs to the name it was 
> indexed before).

Folders aren't indexed by default (mailbox_list_index=no) and force-resync 
doesn't affect them. Are you saying "doveadm mailbox list -u user" doesn't show 
some folder? I'm not aware of any bugs related to this.

> This problem can be reproduced. Can you please fix this? 

I'd need to get some more specifics. Show show exists in filesystems and what 
commands don't produce expected results (and what they produce instead).

> (Not sure if this might change - maybe add an option to support UTF-7 similar 
> to doveadm mailbox list "-7" option)

Nah, mUTF-7 hopefully will die some day.

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[megodin]

What the heck, I don't get it... seems like it's bug's day for me today... the 
d*mn freenet web-mailclient omitted half of the sentence on exact the same part 
*again* - so I split it to several lines and omit some quotes...

Hope this is the last try to get the _whole_ message out:


Naturally renaming a users mail folder should be done via 
doveadm rename
- and not via filesystems "mv" command - to keep dovecot indexes consistent.



So sorry again for all that additional reading garbage (forum admins, please 
feel free to fill in the correct sentence to my original post and delete the 2 
correction posts).

Seems like a have to write another bug report to freenet admins... (sigh)

Sorry again,
Megodin




---
Alle Postfächer an einem Ort. Jetzt wechseln und E-Mail-Adresse mitnehmen! 
http://email.freenet.de/basic/Informationen

_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[megodin]

Sorry, it seems like some part of the sh*tty freenet webmailer omitted half of 
my second sentence of the message I just posted before...


. ..so please replace the weird sentence:
"Naturally renaming a users mail folder should be done via "doveadm es 
consistent."

with:
"Naturally renaming a users mail folder should be done via "doveadm es 
consistent."


Sorry for confusion,
Megodin




---
Alle Postfächer an einem Ort. Jetzt wechseln und E-Mail-Adresse mitnehmen! 
http://email.freenet.de/basic/Informationen

_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

[Dovecot] Bug report: "doveadm rename" encodes special characters wrongly in mUTF-7 (in fs)

[megodin]

Hello there,

I'm pretty sure I found a bug in latest Dovecot 2.2.5.5 (EE) when playing with 
the doveadm command.

Naturally renaming a users mail folder should be done via "doveadm es 
consistent.

So when using "doveadm rename" trying to rename a folder with special 
characters e.g. german umlauts like "ä" it seems to create the folder wrongly 
in the filesystem.

Example: When I want to rename a users folder 'geschaftliches' to 
'gesch&AOQ-ftliches' ("&AOQ-" is the correct mUTF-7 encoding for "ä") using 
the command

doveadm mailbox rename -u user 'geschaftliches' 'gesch&AOQ-ftliches'

it ends up in the filesystem renamed as "gesch&-AOQ-ftliches" which has an 
falsely added "-" set after the "&" and therefore the folder is also wrongly 
shown in Mailclients like Thunderbird or web-mail systems like Horde.
(Apparently "doveadm rename" supports only UTF-8 which shows up when trying to 
rename the folder with 
"doveadm mailbox rename -u user 'geschäftliches'" I get the error message 
"Fatal: Mailbox name not valid UTF-8: geschäftliches")

To get things worse, when renaming the mailfolder in Thunderbird to 
"geschäftliches" it gets correctly written in the filesystem as 
"gesch&AOQ-ftliches" - but afterwards it's NOT found anymore in the dovecot 
index, e.g. when trying 
doveadm force-resync -u user
it is not listed anymore (until back-renaming in fs to the name it was indexed 
before).

This problem can be reproduced. Can you please fix this? 
(Not sure if this might change - maybe add an option to support UTF-7 similar 
to doveadm mailbox list "-7" option)

Thanks in advance,
Megodin



My (relevant part) dovecot configuration (dovecot -n): 

mailstorage @ NFS with nfsvers=3 (NetApp)

# 2.2.5.5 (e5350245ad9b): /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-279.22.1.el6.x86_64 x86_64 CentOS release 6.4 (Final) nfs
auth_cache_size = 10 M
auth_cache_ttl = 10 mins
auth_debug = yes
auth_master_user_separator = *
auth_username_format = %n
auth_verbose = yes
disable_plaintext_auth = no
listen = *
lock_method = dotlock
mail_debug = yes
mail_fsync = always
mail_gid = vmail
mail_home = /mnt/placeholder/%u
mail_location = sdbox:/mnt/placeholder/%u/mailstore
mail_plugins = " quota fts fts_lucene"
mail_privileged_group = mail
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
mmap_disable = yes
passdb {
  args = /etc/dovecot/users.blocked
  deny = yes
  driver = passwd-file
}
passdb {
  args = /etc/dovecot/passwd.masterusers
  driver = passwd-file
  master = yes
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_lucene = whitespace_chars=@.
  quota = dict:User quota::file:%h/dovecot-quota
  quota_rule = *:storage=100M
  quota_rule2 = Trash:storage=+50M
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=98%% quota-warning 98 %u
  sieve = /mnt/placeholder/%u/.dovecot.sieve
  sieve_dir = /mnt/placeholder/%u/sieve
  sieve_global_dir = /var/lib/dovecot/sieve/global/
  sieve_global_path = /var/lib/dovecot/sieve/default.sieve
}
postmaster_address = postmas...@example.com
protocols = imap pop3 sieve
service auth {
  unix_listener auth-userdb {
group = vmail
user = vmail
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
user = vmail
  }
  user = vmail
}
ssl_cert = http://email.freenet.de/basic/Informationen

_
Send and receive anonymous emails to your inbox with InboxAlias.
http://www.inboxalias.com

Re: [Dovecot] bug report: auth via dict

[Timo Sirainen]

On Fri, 2013-01-11 at 00:08 -0800, ForestYang wrote:

> I was testing dovecot server , auth via redis.
> first I follow http://master.wiki2.dovecot.org/AuthDatabase/Dict 
> but auth always fail, last , I found
> value { "uid": 123, "gid": 123, "home": "/home/username" }  was wrong,
> the uid and gid must as string , not number.
> { "uid": "123", "gid": "123", "home": "/home/username" } was worked.
> it's hard to find the problem :(

Did you get some error message with the first way? Both work fine with
my tests.

[Dovecot] bug report: auth via dict

[ForestYang]

hi all,

I was testing dovecot server , auth via redis.
first I follow http://master.wiki2.dovecot.org/AuthDatabase/Dict 
but auth always fail, last , I found
value { "uid": 123, "gid": 123, "home": "/home/username" }  was wrong,
the uid and gid must as string , not number.
{ "uid": "123", "gid": "123", "home": "/home/username" } was worked.
it's hard to find the problem :(

pls notice that.




--
View this message in context: 
http://dovecot.2317879.n4.nabble.com/bug-report-auth-via-dict-tp39685.html
Sent from the Dovecot mailing list archive at Nabble.com.

Re: [Dovecot] Bug report - crash on group lookup

[Timo Sirainen]

On Fri, 2012-05-25 at 18:52 +0200, Peter Meier wrote:

> I was doing some migration from a 1.2 installation to a 2.1. While
> testing my new installation dovecot crashed at two test-cases constantly
> with with a "Panic: Trying to allocate 0 bytes" message.
..
> I see two problems:
> 
> 1. Don't panic while "looking" up a group
> 2. Give a meaningful error messages.

Asserts/Panics can't give very meaningful error messages to users,
because they only happen when there's a bug in the code. The resulting
raw backtrace is helpful to coders though :) Fixed:
http://hg.dovecot.org/dovecot-2.1/rev/fe688ecd7564

[Dovecot] Bug report - crash on group lookup

[Peter Meier]

Hi

I was doing some migration from a 1.2 installation to a 2.1. While
testing my new installation dovecot crashed at two test-cases constantly
with with a "Panic: Trying to allocate 0 bytes" message.

Both times it was because I was using a wrong group (or none) in my config.

The output I got by dovecot was not that helpful. I only figured it out,
that it has something to do with the group by looking at the code, to
figure out where the stacktrace I got could come from.

Unfortunately, I don't remember one of the examples anymore that
correctly (except that it was also a group configuration related issue).
The other example, was that I was testing the quota warning scripts:

Normal delivery worked fine, but as soon as dovecot should send a quota
warning, after the mail got already delivered, I received said panic.
For the MTA the mail delivery failed (-> bounce), although the LDA
already delivered the mail, but failed in post processing (sending the
quota warning).

My fix was to add:

  unix_listener quota-warning {
group = mail
mode = 0660
  }

to the quota warning service. The LDA process has an own user per
mailbox and the mail group.
It looks to me as it would panic by looking up the user of the lda
process, but that's pure guess.

Attached is a debug output and the configuration that triggered the problem.

I see two problems:

1. Don't panic while "looking" up a group
2. Give a meaningful error messages.

If you have any further questions, please let me know.

~pete


==> /var/log/dovecot/debug.log <==
May 25 12:19:50 lda: Debug: Loading modules from directory: /usr/lib/dovecot
May 25 12:19:50 lda: Debug: Module loaded:
/usr/lib/dovecot/lib10_quota_plugin.so
May 25 12:19:50 lda: Debug: Module loaded:
/usr/lib/dovecot/lib90_sieve_plugin.so
May 25 12:19:50 lda: Debug: auth input: dove...@example.com
home=/var/mail/dove...@example.com uid=41407 gid=12 quota_rule=*:storage=1M
May 25 12:19:50 lda: Debug: Added userdb setting:
plugin/quota_rule=*:storage=1M
May 25 12:19:50 lda(dove...@example.com): Debug: Effective uid=41407,
gid=12, home=/var/mail/dove...@example.com
May 25 12:19:50 lda(dove...@example.com): Debug: Quota root: name=User
quota backend=maildir args=
May 25 12:19:50 lda(dove...@example.com): Debug: Quota rule: root=User
quota mailbox=* bytes=1048576 messages=0
May 25 12:19:50 lda(dove...@example.com): Debug: Quota rule: root=User
quota mailbox=INBOX.Trash bytes=+104857 (10%) messages=0
May 25 12:19:50 lda(dove...@example.com): Debug: Quota warning:
bytes=996147 (95%) messages=0 reverse=no command=quota-warning 95
dove...@example.co,
May 25 12:19:50 lda(dove...@example.com): Debug: Quota warning:
bytes=838860 (80%) messages=0 reverse=no command=quota-warning 80
dove...@example.com
May 25 12:19:50 lda(dove...@example.com): Debug: Namespace inbox:
type=private, prefix=INBOX., sep=, inbox=yes, hidden=no, list=yes,
subscriptions=yes location=maildir:/var/mail/dove...@example.com
May 25 12:19:50 lda(dove...@example.com): Debug: maildir++:
root=/var/mail/dove...@example.com/mail, index=, control=,
inbox=/var/mail/dove...@example.com/mail, alt=
May 25 12:19:50 lda(dove...@example.com): Debug: Quota root: name=User
quota backend=maildir args=
May 25 12:19:50 lda(dove...@example.com): Debug: Quota rule: root=User
quota mailbox=* bytes=0 messages=0
May 25 12:19:50 lda(dove...@example.com): Debug: Quota rule: root=User
quota mailbox=INBOX.Trash bytes=0 (10%) messages=0
May 25 12:19:50 lda(dove...@example.com): Debug: Quota warning: bytes=0
(95%) messages=0 reverse=no command=quota-warning 95 raw mail ur
May 25 12:19:50 lda(dove...@example.com): Debug: Quota warning: bytes=0
(80%) messages=0 reverse=no command=quota-warning 80 raw mail ur
May 25 12:19:50 lda(dove...@example.com): Debug: none: root=, index=,
control=, inbox=, alt=
May 25 12:19:50 lda(dove...@example.com): Debug: Destination address:
dove...@example.com (source: -a parameter)
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: user's script
path /var/mail/dove...@example.com/dovecot.sieve doesn't exist
(usinglobal script path in stead)
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: using sieve path
for user's script: /var/lib/dovecot-sieve/default.sieve
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: opening script
/var/lib/dovecot-sieve/default.sieve
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: script binary
/var/lib/dovecot-sieve/default.svbin successfully loaded
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: binary save: not
saving binary /var/lib/dovecot-sieve/default.svbin, because its already
stored
May 25 12:19:50 lda(dove...@example.com): Debug: sieve: executing script
from /var/lib/dovecot-sieve/default.svbin
May 25 12:19:51 lda(dove...@example.com): Debug: quota: Executing
warning: quota-warning 80 dove...@example.com

==> /var/log/dovecot/deliver-errors.log <==
May 25 12:19:51 lda(dove...@example.com): Panic: Trying to allocate 0 bytes
May 25 12:19:51 lda(dove...@example.com): 

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox - Problem reproduced

[Timo Sirainen]

On Fri, 2011-08-19 at 17:52 +0300, Kostas Zorbadelos wrote:
> The problem is more easily introduced than I imagined.

Well, I still couldn't reproduce it. But I can kind of see the problem.
http://hg.dovecot.org/dovecot-2.0/rev/030394c74f54 should help.

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox - Problem reproduced

[Kostas Zorbadelos]

The problem is more easily introduced than I imagined.

Included attached is an example mailbox containing just 3 mails.
Use it to make a couple of POP sessions. Make sure indexes are 
generated for it by setting appropriately  mbox_min_index_size

e.g. mbox_min_index_size = 2k

Now, in the first session just generate the indexes and quit

kzorba@enigma(1)[05:23 PM]~->telnet dovecot-dev 110
Trying ...
Connected to dovecot-dev.
Escape character is '^]'.
+OK OTENET ready
user kzorba1
+OK
pass XXX
+OK Logged in.
LIST
+OK 3 messages:
1 812
2 821
3 816
.
quit
+OK Logging out.
Connection closed by foreign host.

In the second session we LIST and try to DELE a message

kzorba@enigma(1)[05:25 PM]~->telnet dovecot-dev 110
Trying ...
Connected to dovecot-dev.
Escape character is '^]'.
+OK OTENET ready
user kzorba1
+OK
pass XXX
+OK Logged in.
LIST
+OK 3 messages:
1 812
2 821
3 816
.
dele 2
+OK Marked to be deleted.
quit
Connection closed by foreign host.

In the server logs we get
Aug 19 17:25:24 dovecot dovecot: master: Error: service(pop3): child 3489 
killed with signal 11 (core dumped)

With the backtrace:

#0  0x2b9ca3e93dce in istream_raw_mbox_get_start_offset (stream=0x0) at 
istream-raw-mbox.c:498
#1  0x2b9ca3e97956 in mbox_mail_get_special (_mail=0x101cf7d0, 
field=MAIL_FETCH_GUID, value_r=0x7fff1fd69798) at mbox-mail.c:198
#2  0x2b9ca3e47729 in mail_get_special (mail=0x101cf7d0, 
field=MAIL_FETCH_GUID, value_r=0x7fff1fd69798) at mail.c:188
#3  0x2b9ca3e6d133 in index_mail_expunge (mail=0x101cf7d0) at 
index-mail.c:1503
#4  0x2b9ca3e4784d in mail_expunge (mail=0x101cf7d0) at mail.c:233
#5  0x00406272 in client_update_mails (client=0x101baa00) at 
pop3-commands.c:255
#6  0x0040635a in cmd_quit (client=0x101baa00, args=0x407f62 "") at 
pop3-commands.c:274
#7  0x0040774a in client_command_execute (client=0x101baa00, name=0x101960c0 
"QUIT", args=0x407f62 "") at pop3-commands.c:773
#8  0x0040567c in client_handle_input (client=0x101baa00) at 
pop3-client.c:628
#9  0x00405870 in client_input (client=0x101baa00) at pop3-client.c:681
#10 0x2b9ca418af83 in io_loop_call_io (io=0x101b61d0) at ioloop.c:384
#11 0x2b9ca418c685 in io_loop_handler_run (ioloop=0x1019e610) at 
ioloop-epoll.c:213
#12 0x2b9ca418b024 in io_loop_run (ioloop=0x1019e610) at ioloop.c:405
#13 0x2b9ca4171ca2 in master_service_run (service=0x1019e4e0, callback=0x403bab 
) at master-service.c:478
#14 0x00403e14 in main (argc=1, argv=0x1019e370) at main.c:252

I guess something is wrong in the indexing code because if I disable the index 
creation
using an appropriate value for mbox_min_index_size everything works as expected 
and
mails are deleted.

In our case, all clients that leave messages on server cannot actually delete 
messages
from their inboxes. Clients with Thunderbird, keep getting the mails after they
delete them, customers with Outlook don't see them again, but the deleted 
messages
remain in the server. Needless to say that I will try to provide a patch but
it will take me far more time than any developer familiar with the code already.

Regards,

Kostas

From kzo...@otenet.gr Fri Jun 24 15:40:59 2011
X-UIDL: ~&c!!/(1"!JG4!!T&0!!
Return-Path: 
Received: from noc.otenet.gr (noc.otenet.gr [195.170.0.29])
by speedy.otenet.gr (8.13.8/8.13.8/Debian-3) with ESMTP id 
p5OCexJ7025830;
Fri, 24 Jun 2011 15:40:59 +0300
Received: from [212.205.221.137] (enigma.otenet.gr [212.205.221.137])
by noc.otenet.gr (Postfix) with ESMTP id 8D4F68B81C7;
Fri, 24 Jun 2011 15:40:59 +0300 (EEST)
Message-ID: <4e04867b.3000...@otenet.gr>
Date: Fri, 24 Jun 2011 15:43:39 +0300
From: Kostas Zorbadelos 
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) 
Gecko/20110424 Lightning/1.0b2 Thunderbird/3.1.10
MIME-Version: 1.0
To: kzor...@otenet.gr
Subject: Test receive
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-IMAPbase: 1140602935 000477 Old
X-UID: 475
Status: RO
X-Keywords: Old  

Test it.

From kzo...@otenet.gr Fri Jun 24 16:16:44 2011
X-UIDL: D[H"!e^Q!!0=:"!=G,"!
Return-Path: 
Received: from noc.otenet.gr (noc.otenet.gr [195.170.0.29])
by speedy.otenet.gr (8.13.8/8.13.8/Debian-3) with ESMTP id 
p5ODGhv9024786;
Fri, 24 Jun 2011 16:16:43 +0300
Received: from [212.205.221.137] (enigma.otenet.gr [212.205.221.137])
by noc.otenet.gr (Postfix) with ESMTP id C57AB8B81F2;
Fri, 24 Jun 2011 16:16:43 +0300 (EEST)
Message-ID: <4e048edc.1090...@otenet.gr>
Date: Fri, 24 Jun 2011 16:19:24 +0300
From: Kostas Zorbadelos 
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) 
Gecko/20110424 Lightning/1.0b2 Thunderbird/3.1.10
MIME-Version: 1.0
To: kzor...@otenet.gr
Subject: Test receive 2
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Status: RO
X-UID: 476  

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox - Problem reproduced

[Kostas Zorbadelos]

On 08/16/2011 04:42 PM, Kostas Zorbadelos wrote:

On 08/15/2011 11:17 AM, kzo...@otenet.gr wrote:

Quoting Timo Sirainen:



OK,

we managed to reproduce the problem with the following scenario.

Using Thunderbird (3.1.11 if that matters) we set up a mail account 
using POP served by dovecot. In the account settings we have checked the 
boxes "Leave messages on server" and "Until I delete them" which is the 
default setting for new accounts.


We make the first POP session and get the mails. We then delete a 
message (moving it to trash) and try to Get Mails. Dovecot crashes with 
the aforementioned stack trace (included below). If you again try Get 
Mail in the client, you get the deleted mail again.


It seems to be related to the indexing code since if we change the setting

mbox_min_index_size

to something bigger than the mailbox (so no cache index files are 
created) the problem does not appear and the mails get deleted from the 
server normally. We didn't manage to reproduce the problem with 
Microsoft Outlook.


We 'll keep investigating until we have a fix.

Regards,

Kostas


With a quick test I can't reproduce pop3_lock_session=yes causing a
crash. I guess it needs something else besides what I tested. It would
be helpful if your Dovecot binaries weren't stripped of debug symbols. I
could then ask for some more information from the core dumps with gdb.





Timo, all

here is a backtrace with debuging symbols in the executables:

[root@pop08 ]# gdb /opt/dovecot-debug/libexec/dovecot/pop3 core.2929
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /opt/dovecot-debug/libexec/dovecot/pop3...done.
BFD: Warning: /var/mail7/folders/P/K/U/amihal/core.2929 is truncated: expected 
core file size>= 569344, found: 565248.
Reading symbols from 
/opt/dovecot-debug/lib/dovecot/libdovecot-storage.so.0...done.
Loaded symbols for /opt/dovecot-debug/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /opt/dovecot-debug/lib/dovecot/libdovecot.so.0...done.
Loaded symbols for /opt/dovecot-debug/lib/dovecot/libdovecot.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/libpthread.so.0
Core was generated by `dovecot/pop3'.
Program terminated with signal 11, Segmentation fault.
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
498 istream-raw-mbox.c: No such file or directory.
 in istream-raw-mbox.c
(gdb) bt
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
#1  0x2b4114afc769 in mbox_mail_get_special (_mail=0x1bad8fa0, field=, value_r=0x7fff9600fa88) at mbox-mail.c:198
#2  0x2b4114adb2d0 in index_mail_expunge (mail=0x0) at index-mail.c:1503
#3  0x00405eac in client_update_mails (client=0x1bac4a00) at 
pop3-commands.c:255
#4  0x004061d1 in cmd_quit (client=0x1bac4a00, name=0x1baccbd4 "", args=0x406a0c 
"") at pop3-commands.c:274
#5  client_command_execute (client=0x1bac4a00, name=0x1baccbd4 "", args=0x406a0c 
"") at pop3-commands.c:773
#6  0x004045c9 in client_handle_input (client=0x1bac4a00) at 
pop3-client.c:628
#7  0x2b4114db0698 in io_loop_call_io (io=0x1bac01d0) at ioloop.c:384
#8  0x2b4114db19d5 in io_loop_handler_run (ioloop=) at 
ioloop-epoll.c:213
#9  0x2b4114db062d in io_loop_run (ioloop=0x1baa8610) at ioloop.c:405
#10 0x2b4114d9ef13 in master_service_run (service=0x1baa84e0, 
callback=0x1baccbe4) at master-service.c:478
#11 0x004039a4 in main (argc=1, argv=0x1baa8370) at main.c:252
(gdb) bt full
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
 rstream =
 __FUNCTION__ = "istream_raw_mbox_get_start_offset"
#1  0x2b4114afc769 in mbox_mail_get_special (_mail=0x1bad8fa0, field=, value_r=0x7fff9600fa88) at mbox-mail.c:198
 mail =
 mbox = 0x1bac97d0
 offset = 0
#2  0x2b4114adb2d0 in index_mail_expunge (mail=0x0) at index-mail.c:1503
 value =
 guid_128 = 
"\240\267\252\033\000\000\000\000\001\000\000\000\0

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox

[Kostas Zorbadelos]

On 08/15/2011 11:17 AM, kzo...@otenet.gr wrote:
> Quoting Timo Sirainen :
> 
>> With a quick test I can't reproduce pop3_lock_session=yes causing a
>> crash. I guess it needs something else besides what I tested. It would
>> be helpful if your Dovecot binaries weren't stripped of debug symbols. I
>> could then ask for some more information from the core dumps with gdb.
>>
> 

Timo, all

here is a backtrace with debuging symbols in the executables:

[root@pop08 ]# gdb /opt/dovecot-debug/libexec/dovecot/pop3 core.2929
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /opt/dovecot-debug/libexec/dovecot/pop3...done.
BFD: Warning: /var/mail7/folders/P/K/U/amihal/core.2929 is truncated: expected 
core file size >= 569344, found: 565248.
Reading symbols from 
/opt/dovecot-debug/lib/dovecot/libdovecot-storage.so.0...done.
Loaded symbols for /opt/dovecot-debug/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /opt/dovecot-debug/lib/dovecot/libdovecot.so.0...done.
Loaded symbols for /opt/dovecot-debug/lib/dovecot/libdovecot.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/libpthread.so.0
Core was generated by `dovecot/pop3'.
Program terminated with signal 11, Segmentation fault.
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
498 istream-raw-mbox.c: No such file or directory.
in istream-raw-mbox.c
(gdb) bt
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
#1  0x2b4114afc769 in mbox_mail_get_special (_mail=0x1bad8fa0, field=, value_r=0x7fff9600fa88) at mbox-mail.c:198
#2  0x2b4114adb2d0 in index_mail_expunge (mail=0x0) at index-mail.c:1503
#3  0x00405eac in client_update_mails (client=0x1bac4a00) at 
pop3-commands.c:255
#4  0x004061d1 in cmd_quit (client=0x1bac4a00, name=0x1baccbd4 "", 
args=0x406a0c "") at pop3-commands.c:274
#5  client_command_execute (client=0x1bac4a00, name=0x1baccbd4 "", 
args=0x406a0c "") at pop3-commands.c:773
#6  0x004045c9 in client_handle_input (client=0x1bac4a00) at 
pop3-client.c:628
#7  0x2b4114db0698 in io_loop_call_io (io=0x1bac01d0) at ioloop.c:384
#8  0x2b4114db19d5 in io_loop_handler_run (ioloop=) at 
ioloop-epoll.c:213
#9  0x2b4114db062d in io_loop_run (ioloop=0x1baa8610) at ioloop.c:405
#10 0x2b4114d9ef13 in master_service_run (service=0x1baa84e0, 
callback=0x1baccbe4) at master-service.c:478
#11 0x004039a4 in main (argc=1, argv=0x1baa8370) at main.c:252
(gdb) bt full
#0  istream_raw_mbox_get_start_offset (stream=0x0) at istream-raw-mbox.c:498
rstream = 
__FUNCTION__ = "istream_raw_mbox_get_start_offset"
#1  0x2b4114afc769 in mbox_mail_get_special (_mail=0x1bad8fa0, field=, value_r=0x7fff9600fa88) at mbox-mail.c:198
mail = 
mbox = 0x1bac97d0
offset = 0
#2  0x2b4114adb2d0 in index_mail_expunge (mail=0x0) at index-mail.c:1503
value = 
guid_128 = 
"\240\267\252\033\000\000\000\000\001\000\000\000\000\000\000"
#3  0x00405eac in client_update_mails (client=0x1bac4a00) at 
pop3-commands.c:255
search_args = 0x0
ctx = 0x1bace150
mail = 0x1bad8fa0
msgnum = 0
bit = 464309220
ret = 
#4  0x004061d1 in cmd_quit (client=0x1bac4a00, name=0x1baccbd4 "", 
args=0x406a0c "") at pop3-commands.c:274
No locals.
#5  client_command_execute (client=0x1bac4a00, name=0x1baccbd4 "", 
args=0x406a0c "") at pop3-commands.c:773
No locals.
#6  0x004045c9 in client_handle_input (client=0x1bac4a00) at 
pop3-client.c:628
_data_stack_cur_id = 3
line = 0x1baab872 "QUIT"
args = 0x406a0c ""
ret = 
#7  0x2b4114db0698 in io_loop_call_io (io=0x1bac01d0) at ioloop.c:384
ioloop = 0x1baa8610
t_id = 2
#8  0x2b4114db19d5 in io_loop_handler_run (ioloop=) at 
ioloop-epoll.c:213
ctx = 
event = 0x1baa88b0
list = 0x1bac0220
io = 0x0
tv = {tv_sec = 9, tv_usec = 871818}
ev

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox

[kzorba]

Quoting Timo Sirainen :


With a quick test I can't reproduce pop3_lock_session=yes causing a
crash. I guess it needs something else besides what I tested. It would
be helpful if your Dovecot binaries weren't stripped of debug symbols. I
could then ask for some more information from the core dumps with gdb.



Hi Timo,

indeed it is a bug that I could not reproduce myself.
Having debug symbols and producing the stack trace is the next
logical step and I will work on this tomorrow.
Since --enable-debug does not work in your configure script, can you
direct me as to what is needed? Is there an option in configure or
do I need to mess with the makefiles?

On the other hand, I have found two different bugs.
Having pop3_lock_session=yes we have the situation described here and also
of course delays in local deliveries in case a client has an active pop
session. And I can tell you we have a lot of abusing clients that keep
hitting our pop servers continuously, or keep connections open for a VERY
long time.

To address that, we put pop3_lock_session=no. In this case, there is an fcntl
lock leak somewhere. The good news is that we have reproduced that and I will
send relevant information in a different mail.
I also read the following thread, from a while back:

http://www.dovecot.org/list/dovecot/2009-February/037098.html

Regards,

Kostas


On Wed, 2011-08-10 at 13:07 +0300, Kostas Zorbadelos wrote:

On 07/22/2011 01:02 PM, Kostas Zorbadelos wrote:

Hello,

since I saw no action on this, here is a newer update we discovered today.

After setting pop3_lock_session = no the core dumps went away.
We will leave it like that and watch it for the next few days. If we set
pop3_lock_session = yes, the problem is reproduced.

If I can do anything else to help debug the problem, please let me know.

Regards,

Kostas

> Greetings to all.
>
> It's my first post to the list. We just completed a migration  
from qpopper to dovecot

> for our IMAP and POP3 services. We have a rather large mail environment
> (we are the biggest provider in Greece).
>
> So, here are the details:
>
> - Keep getting errors like these in our production environment
>
> Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3):  
child 4078 killed with signal 11 (core dumps disabled)
> Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3):  
child 18849 killed with signal 11 (core dumps disabled)

>
> -
> dovecot -n output
> -
> /opt/dovecot/sbin/dovecot -n
> # 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf
> # OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final)
> auth_cache_negative_ttl = 10 mins
> auth_cache_size = 5 M
> auth_cache_ttl = 10 mins
> auth_verbose = yes
> default_client_limit = 5000
> default_process_limit = 500
> disable_plaintext_auth = no
> first_valid_uid = 200
> listen = *
> log_timestamp = "%Y-%m-%d %H:%M:%S "
> login_greeting =  ready
> mail_access_groups = mail otemail disk root
> mail_fsync = always
> mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u
> mail_nfs_storage = yes
> mbox_lock_timeout = 2 mins
> mbox_min_index_size = 200 k
> mbox_read_locks = dotlock_try fcntl
> mbox_write_locks = dotlock_try fcntl
> passdb {
>args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
>driver = ldap
> }
> protocols = imap pop3
> service auth-worker {
>user = dovenull
> }
> service imap-login {
>inet_listener imap {
>  port = 143
>}
>inet_listener imaps {
>  port = 993
>  ssl = yes
>}
> }
> service pop3-login {
>inet_listener pop3 {
>  port = 110
>}
>inet_listener pop3s {
>  port = 995
>  ssl = yes
>}
> }
> ssl = no
> userdb {
>args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
>driver = ldap
> }
> verbose_proctitle = yes
> protocol imap {
>imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
>mail_max_userip_connections = 100
> }
> protocol pop3 {
>mail_max_userip_connections = 100
>pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
>pop3_fast_size_lookups = yes
>pop3_lock_session = yes
>pop3_reuse_xuidl = yes
>pop3_uidl_format = %08Xu%08Xv
> }
>
> I enabled core dumps in one of our backend servers and here is  
the relevant gdb trace:

>
> [root@pop08 ~]# gdb  
/opt/dovecot/libexec/dovecot/pop3/core.9273

> GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
> Copyright (C) 2009 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or  
later

> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> ...
> Reading symbols from /opt/dovecot/

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox

[Timo Sirainen]

With a quick test I can't reproduce pop3_lock_session=yes causing a
crash. I guess it needs something else besides what I tested. It would
be helpful if your Dovecot binaries weren't stripped of debug symbols. I
could then ask for some more information from the core dumps with gdb.

On Wed, 2011-08-10 at 13:07 +0300, Kostas Zorbadelos wrote:
> On 07/22/2011 01:02 PM, Kostas Zorbadelos wrote:
> 
> Hello,
> 
> since I saw no action on this, here is a newer update we discovered today.
> 
> After setting pop3_lock_session = no the core dumps went away.
> We will leave it like that and watch it for the next few days. If we set 
> pop3_lock_session = yes, the problem is reproduced.
> 
> If I can do anything else to help debug the problem, please let me know.
> 
> Regards,
> 
> Kostas
> 
> > Greetings to all.
> >
> > It's my first post to the list. We just completed a migration from qpopper 
> > to dovecot
> > for our IMAP and POP3 services. We have a rather large mail environment
> > (we are the biggest provider in Greece).
> >
> > So, here are the details:
> >
> > - Keep getting errors like these in our production environment
> >
> > Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3): child 4078 
> > killed with signal 11 (core dumps disabled)
> > Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3): child 18849 
> > killed with signal 11 (core dumps disabled)
> >
> > -
> > dovecot -n output
> > -
> > /opt/dovecot/sbin/dovecot -n
> > # 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf
> > # OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final)
> > auth_cache_negative_ttl = 10 mins
> > auth_cache_size = 5 M
> > auth_cache_ttl = 10 mins
> > auth_verbose = yes
> > default_client_limit = 5000
> > default_process_limit = 500
> > disable_plaintext_auth = no
> > first_valid_uid = 200
> > listen = *
> > log_timestamp = "%Y-%m-%d %H:%M:%S "
> > login_greeting =  ready
> > mail_access_groups = mail otemail disk root
> > mail_fsync = always
> > mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u
> > mail_nfs_storage = yes
> > mbox_lock_timeout = 2 mins
> > mbox_min_index_size = 200 k
> > mbox_read_locks = dotlock_try fcntl
> > mbox_write_locks = dotlock_try fcntl
> > passdb {
> >args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
> >driver = ldap
> > }
> > protocols = imap pop3
> > service auth-worker {
> >user = dovenull
> > }
> > service imap-login {
> >inet_listener imap {
> >  port = 143
> >}
> >inet_listener imaps {
> >  port = 993
> >  ssl = yes
> >}
> > }
> > service pop3-login {
> >inet_listener pop3 {
> >  port = 110
> >}
> >inet_listener pop3s {
> >  port = 995
> >  ssl = yes
> >}
> > }
> > ssl = no
> > userdb {
> >args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
> >driver = ldap
> > }
> > verbose_proctitle = yes
> > protocol imap {
> >imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
> >mail_max_userip_connections = 100
> > }
> > protocol pop3 {
> >mail_max_userip_connections = 100
> >pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
> >pop3_fast_size_lookups = yes
> >pop3_lock_session = yes
> >pop3_reuse_xuidl = yes
> >pop3_uidl_format = %08Xu%08Xv
> > }
> >
> > I enabled core dumps in one of our backend servers and here is the relevant 
> > gdb trace:
> >
> > [root@pop08 ~]# gdb 
> > /opt/dovecot/libexec/dovecot/pop3/core.9273
> > GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
> > Copyright (C) 2009 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-redhat-linux-gnu".
> > For bug reporting instructions, please see:
> > ...
> > Reading symbols from /opt/dovecot/libexec/dovecot/pop3...(no debugging 
> > symbols found)...done.
> > Reading symbols from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0...(no 
> > debugging symbols found)...done.
> > Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot-storage.so.0
> > Reading symbols from /opt/dovecot/lib/dovecot/libdovecot.so.0...(no 
> > debugging symbols found)...done.
> > Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot.so.0
> > Reading symbols from /lib64/libdl.so.2...(no debugging symbols 
> > found)...done.
> > Loaded symbols for /lib64/libdl.so.2
> > Reading symbols from /lib64/librt.so.1...(no debugging symbols 
> > found)...done.
> > Loaded symbols for /lib64/librt.so.1
> > Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
> > Loaded symbols for /lib64/libc.so.6
> > Reading symbols from /lib64/ld-

Re: [Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox

[Kostas Zorbadelos]

On 07/22/2011 01:02 PM, Kostas Zorbadelos wrote:

Hello,

since I saw no action on this, here is a newer update we discovered today.

After setting pop3_lock_session = no the core dumps went away.
We will leave it like that and watch it for the next few days. If we set 
pop3_lock_session = yes, the problem is reproduced.


If I can do anything else to help debug the problem, please let me know.

Regards,

Kostas


Greetings to all.

It's my first post to the list. We just completed a migration from qpopper to 
dovecot
for our IMAP and POP3 services. We have a rather large mail environment
(we are the biggest provider in Greece).

So, here are the details:

- Keep getting errors like these in our production environment

Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3): child 4078 killed 
with signal 11 (core dumps disabled)
Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3): child 18849 killed 
with signal 11 (core dumps disabled)

-
dovecot -n output
-
/opt/dovecot/sbin/dovecot -n
# 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final)
auth_cache_negative_ttl = 10 mins
auth_cache_size = 5 M
auth_cache_ttl = 10 mins
auth_verbose = yes
default_client_limit = 5000
default_process_limit = 500
disable_plaintext_auth = no
first_valid_uid = 200
listen = *
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_greeting =  ready
mail_access_groups = mail otemail disk root
mail_fsync = always
mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u
mail_nfs_storage = yes
mbox_lock_timeout = 2 mins
mbox_min_index_size = 200 k
mbox_read_locks = dotlock_try fcntl
mbox_write_locks = dotlock_try fcntl
passdb {
   args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
protocols = imap pop3
service auth-worker {
   user = dovenull
}
service imap-login {
   inet_listener imap {
 port = 143
   }
   inet_listener imaps {
 port = 993
 ssl = yes
   }
}
service pop3-login {
   inet_listener pop3 {
 port = 110
   }
   inet_listener pop3s {
 port = 995
 ssl = yes
   }
}
ssl = no
userdb {
   args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
verbose_proctitle = yes
protocol imap {
   imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
   mail_max_userip_connections = 100
}
protocol pop3 {
   mail_max_userip_connections = 100
   pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
   pop3_fast_size_lookups = yes
   pop3_lock_session = yes
   pop3_reuse_xuidl = yes
   pop3_uidl_format = %08Xu%08Xv
}

I enabled core dumps in one of our backend servers and here is the relevant gdb 
trace:

[root@pop08 ~]# gdb 
/opt/dovecot/libexec/dovecot/pop3/core.9273
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /opt/dovecot/libexec/dovecot/pop3...(no debugging symbols 
found)...done.
Reading symbols from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0...(no 
debugging symbols found)...done.
Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /opt/dovecot/lib/dovecot/libdovecot.so.0...(no debugging 
symbols found)...done.
Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/libpthread.so.0
Core was generated by `dovecot/pop3'.
Program terminated with signal 11, Segmentation fault.
#0  0x2b52e1027e54 in istream_raw_mbox_get_start_offset () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
(gdb) bt full
#0  0x2b52e1027e54 in istream_raw_mbox_get_start_offset () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbol table info available.
#1  0x2b52e102b759 in ?? () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbol table info available.
#2  0x2b52e100a2c0 in index_mail_expunge () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbo

[Dovecot] Bug report v2.0.13 - CentOS x86_64 - NFS - mbox

[Kostas Zorbadelos]

Greetings to all.

It's my first post to the list. We just completed a migration from qpopper to 
dovecot 
for our IMAP and POP3 services. We have a rather large mail environment 
(we are the biggest provider in Greece).

So, here are the details:

- Keep getting errors like these in our production environment

Jul 22 00:18:21 pop01 dovecot: master: Error: service(pop3): child 4078 killed 
with signal 11 (core dumps disabled)
Jul 22 00:19:31 pop03 dovecot: master: Error: service(pop3): child 18849 killed 
with signal 11 (core dumps disabled)

-
dovecot -n output
-
/opt/dovecot/sbin/dovecot -n
# 2.0.13: /opt/dovecot/etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.5 (Final) 
auth_cache_negative_ttl = 10 mins
auth_cache_size = 5 M
auth_cache_ttl = 10 mins
auth_verbose = yes
default_client_limit = 5000
default_process_limit = 500
disable_plaintext_auth = no
first_valid_uid = 200
listen = *
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_greeting =  ready
mail_access_groups = mail otemail disk root
mail_fsync = always
mail_location = mbox:INDEX=/var/index/dovecot/%2.16Hn/%2.254Hn/%u
mail_nfs_storage = yes
mbox_lock_timeout = 2 mins
mbox_min_index_size = 200 k
mbox_read_locks = dotlock_try fcntl
mbox_write_locks = dotlock_try fcntl
passdb {
  args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
protocols = imap pop3
service auth-worker {
  user = dovenull
}
service imap-login {
  inet_listener imap {
port = 143
  }
  inet_listener imaps {
port = 993
ssl = yes
  }
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
port = 995
ssl = yes
  }
}
ssl = no
userdb {
  args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
verbose_proctitle = yes
protocol imap {
  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
  mail_max_userip_connections = 100
}
protocol pop3 {
  mail_max_userip_connections = 100
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_fast_size_lookups = yes
  pop3_lock_session = yes
  pop3_reuse_xuidl = yes
  pop3_uidl_format = %08Xu%08Xv
}

I enabled core dumps in one of our backend servers and here is the relevant gdb 
trace:

[root@pop08 ~]# gdb /opt/dovecot/libexec/dovecot/pop3 
/core.9273 
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-32.el5_6.2)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /opt/dovecot/libexec/dovecot/pop3...(no debugging symbols 
found)...done.
Reading symbols from /opt/dovecot/lib/dovecot/libdovecot-storage.so.0...(no 
debugging symbols found)...done.
Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot-storage.so.0
Reading symbols from /opt/dovecot/lib/dovecot/libdovecot.so.0...(no debugging 
symbols found)...done.
Loaded symbols for /opt/dovecot/lib/dovecot/libdovecot.so.0
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols 
found)...done.
Loaded symbols for /lib64/libpthread.so.0
Core was generated by `dovecot/pop3'.
Program terminated with signal 11, Segmentation fault.
#0  0x2b52e1027e54 in istream_raw_mbox_get_start_offset () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
(gdb) bt full
#0  0x2b52e1027e54 in istream_raw_mbox_get_start_offset () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbol table info available.
#1  0x2b52e102b759 in ?? () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbol table info available.
#2  0x2b52e100a2c0 in index_mail_expunge () from 
/opt/dovecot/lib/dovecot/libdovecot-storage.so.0
No symbol table info available.
#3  0x00405e9c in client_update_mails ()
No symbol table info available.
#4  0x004061c1 in client_command_execute ()
No symbol table info available.
#5  0x004045b9 in client_handle_input ()
No symbol table info available.
#6  0x2b52e12df698 in io_loop_call_io () from 
/opt/dovecot/lib/dovecot/libdovecot.so.0
No symbol table info available.
#7  0x2b52e12e09d5 in io_loop_handler_run 

[Dovecot] Bug report: Panic: Trying to sync mailbox archive with open transactions

[Erik Hetzner]

Hi,

I am getting the following error trying to search a virtual folder.

May 26 11:35:29 xam dovecot: imap(egh): Panic: Trying to sync mailbox archive 
with open transactions
May 26 11:35:29 xam dovecot: imap(egh): Error: Raw backtrace: 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(+0x39ce1) [0xd2dce1] 
-> /usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(+0x39d4f) 
[0xd2dd4f] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0xd0485e] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot-storage.so.0(+0x24dd6) 
[0xf5add6] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/lib20_virtual_plugin.so(virtual_storage_sync_init+0x80f)
 [0x4a1a7f] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x39)
 [0xf5ada9] -> dovecot/imap(imap_sync_init+0x54) [0x805df74] -> 
dovecot/imap(cmd_sync_delayed+0x1bc) [0x805ecdc] -> 
dovecot/imap(client_handle_input+0x21a) [0x805658a] -> 
dovecot/imap(client_input+0x5f) [0x8056daf] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x51)
 [0xd3b681] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(io_loop_han
 dler_run+0xd3) [0xd3c8b3] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(io_loop_run+0x40) 
[0xd3b600] -> 
/usr/local/stow/dovecot-2.0.13/lib/dovecot/libdovecot.so.0(master_service_run+0x2a)
 [0xd26e1a] -> dovecot/imap(main+0x32d) [0x805f99d] -> 
/lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x808e37] -> 
dovecot/imap() [0x804e131]
May 26 11:35:29 xam dovecot: master: Error: service(imap): child 18403 killed 
with signal 6 (core dumps disabled)

As you can see, I am using fts_solr plugin.

the virtual mailbox is pretty simple:

  INBOX
  *
  -spam
  -Trash
  -trash
  -drafts

My client is Wanderlust.

Let me know any other information I can provide.

Thank you!

best, Erik Hetzner
# 2.0.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.38-8-generic i686 Ubuntu 11.04 
log_timestamp = "%Y-%m-%d %H:%M:%S "
mail_location = maildir:~/maildir
mail_plugins = " virtual fts fts_solr"
mail_privileged_group = mail
namespace {
  inbox = yes
  location = maildir:~/maildir
  prefix = 
  separator = /
  type = private
}
namespace {
  location = virtual:~/maildir/virtual
  prefix = virtual/
  separator = /
}
passdb {
  args = failure_show_msg=yes dovecot
  driver = pam
}
plugin {
  fts = solr
  fts_solr = url=http://localhost:8080/solr/mail/ break-imap-search
}
protocols = imap
service auth {
  user = root
}
ssl = required
ssl_cert = Sent from my free software system .

[Dovecot] Bug Report: Failure delay config is ignored

[Kevin Goodsell]

I have a set of automated tests for a system that can authenticate a
user by forwarding authentication details to a server. That is, in order
to authenticate a user, this system logs on to, e.g., an IMAP server
using the credentials supplied by the user to verify the correctness of
those credentials. I've been using dovecot as the authentication server
to test IMAP, IMAPs, POP3, and APOP. To run the tests (most of which are
authentication failures), I configure dovecot without authentication
failure delays, since obviously I don't want to wait several minutes for
the tests to finish.

Setting the configuration variable auth_failure_delay to 0 worked fine
in Dovecot 1.2, but since Debian updated to Dovecot 2.0 it no longer
works. The tests take a very long time to run.

I tried changing my passwd file to include nodelay=y, but this only
works in a very limited way. For the set of tests which use the correct
username, but an incorrect password, this successfully disables the
failure delay. However, for the set of tests which use incorrect
usernames (names not found in the user database or the password
database, and which therefore don't have the extra nodelay field), the
failure delay still occurs.

There seems to be at least two problems here. The first is that it isn't
clear what effect auth_failure_delay has. The comment in the sample
config file just says "Time to delay before replying to failed
authentications", which is certainly an incomplete description if not
totally inaccurate. This setting seems to be completely unmentioned in
the 2.0 wiki.

The second problem is that the nodelay extra field can only be applied
on a per-user basis. There doesn't seem to be any way to apply it to all
users, or to apply it to authentication attempts for non-existent users.

It seems to me that the failure delay that is actually used should
either be 1) the actual value of auth_failure_delay (ideally with some
random variance for extra security, but recognizing 0 as meaning
disabled) or 2) some multiple of auth_failure_delay, where the
multiplier increases with subsequent failures up to some max delay, with
0 still giving no delay.

Config information follows. Thanks.

-Kevin

$ dovecot -n
# 2.0.12: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-5-686 i686 Debian wheezy/sid
auth_debug = yes
auth_failure_delay = 0
auth_mechanisms = plain apop
auth_verbose = yes
listen = 127.0.0.1
log_path = /var/log/dovecot
mail_location = maildir:~/Maildir
passdb {
  args = /etc/dovecot/passwd
  driver = passwd-file
}
protocols = " imap pop3"
ssl_cert = 

Re: [Dovecot] (BUG REPORT) sieve 0.1.14: vacation encodes Subject twice

[Stephan Bosch]

Timo Sirainen wrote:

On 7.1.2010, at 11.40, Stephan Bosch wrote:


Timo, are you going to change this in Dovecot or should I implement the 
non-ASCII-only check in Sieve?


I don't think my function should do such checks. It might be wanted in some 
situations that ASCII-only text gets encoded.


Ok, I'll give this one a look later today.

Regards,

Stephan.

Re: [Dovecot] (BUG REPORT) sieve 0.1.14: vacation encodes Subject twice

[Timo Sirainen]

On 7.1.2010, at 11.40, Stephan Bosch wrote:

> Timo, are you going to change this in Dovecot or should I implement the 
> non-ASCII-only check in Sieve?

I don't think my function should do such checks. It might be wanted in some 
situations that ASCII-only text gets encoded.

Re: [Dovecot] (BUG REPORT) sieve 0.1.14: vacation encodes Subject twice

[Stephan Bosch]

Christian Rohmann wrote:

Hey folks,

allow me to bump this notice about a possible plain bug in the sieve
filter conversion.


Oh, sorry, this one got a bit backlogged.


According to RFC5230 section 4.3 this shouldn't happen:

   The ":subject" parameter specifies a subject line to attach to any
   vacation response that is generated.  UTF-8 characters can be used in
   the string argument; implementations MUST convert the string to
   [RFC2047] encoded words if and only if non-ASCII characters are
   present.  Implementations MUST generate an appropriate default
   subject line as specified below if no :subject parameter is
   specified.

If I understand this right, sieve should do the RFC2047 encoding only
if the string is non-ASCII.  But the RFC2047 encoded string is already
ASCII, so it should not be encoded a second time.


Yes, that seems to be the intended behavior.


I just compared this with sieve 0.1.13, which behaves different: It
does not encode the Subject header at all.  As long as the managesieve
client (in our situation Horde/IMP) encodes the subject, everything
works correct, while we run into trouble with 0.1.14.

Timo and I first introduced header encoding in that last version. Before 
that, no attempt to encode the header was made at all.


Timo, are you going to change this in Dovecot or should I implement the 
non-ASCII-only check in Sieve?


Regards,

Stephan

Re: [Dovecot] (BUG REPORT) sieve 0.1.14: vacation encodes Subject twice

[Christian Rohmann]

Hey folks,

allow me to bump this notice about a possible plain bug in the sieve
filter conversion.


Regards


Christian

On 12/30/2009 01:37 PM, Roland Rosenfeld wrote:
> Hi!
> 
> I noticed the following problem in dovecot-1.2-sieve-0.1.14:
> The following vacation causes trouble with the Subject line:
> 
> if not exists ["List-Id", "List-Post"]  { 
> vacation :days 7 :addresses ["t...@netcologne.de"] :subject 
> "=?utf-8?b?w4TDlsOc?= sadasd" "Bin gerade nicht hier.";
> } 
> 
> As you can see, the Subject is already MIME encoded according to
> RFC2047.
> 
> Starting with sieve 0.1.14 the subject is now encoded by sieve again,
> so I get the following in my vacation mail:
> 
> Subject: =?utf-8?q?=3D=3Futf-8=3Fb=3Fw4TDlsOc=3F=3D?= sadasd
> 
> As you can see, the RFC2047 encoding is RFC2047 encoded, which results
> in an unreadable subject.
> 
> According to RFC5230 section 4.3 this shouldn't happen:
> 
>The ":subject" parameter specifies a subject line to attach to any
>vacation response that is generated.  UTF-8 characters can be used in
>the string argument; implementations MUST convert the string to
>[RFC2047] encoded words if and only if non-ASCII characters are
>present.  Implementations MUST generate an appropriate default
>subject line as specified below if no :subject parameter is
>specified.
> 
> If I understand this right, sieve should do the RFC2047 encoding only
> if the string is non-ASCII.  But the RFC2047 encoded string is already
> ASCII, so it should not be encoded a second time.
> 
> 
> I just compared this with sieve 0.1.13, which behaves different: It
> does not encode the Subject header at all.  As long as the managesieve
> client (in our situation Horde/IMP) encodes the subject, everything
> works correct, while we run into trouble with 0.1.14.
> 
> I think that there needs to some mechanism in sieve that checks
> whether the subject contains non-ASCII and does the encoding only in
> this case.
> 
> I just checked out, that 0.1.14 does not encode a non RFC2047 encoded
> 7bit subject, while a 7bit subject starting with =?utf-8?... is
> encoded twice.
> 
> Tscho
> 
> Roland
> 

Re: [Dovecot] Bug report

[Timo Sirainen]

On Wed, 2009-11-11 at 20:11 -0800, Bryan Seitz wrote:

> Nov 11 23:11:20 triton dovecot: IMAP(sam): Panic: file
> istream-raw-mbox.c: line 578 (istream_raw_mbox_get_body_size):
> assertion failed: (rstream->body_offset != (uoff_t)-1) 

Are you still seeing these? Wonder if these patches would at least make
it log more:

http://hg.dovecot.org/dovecot-1.2/rev/8d4621094e54
http://hg.dovecot.org/dovecot-1.2/rev/fe1f31b4e21e



signature.asc
Description: This is a digitally signed message part

[Dovecot] Bug report

[Bryan Seitz]

One of my users is causing dovecot to exit with signal 6 and generate a core 
file.  Seems to be in the mbox code.

Dovecot configuration:

# 1.2.4: /usr/local/etc/dovecot.conf
# OS: FreeBSD 7.2-STABLE amd64  
protocols: imap imaps pop3s
listen(default): localhost:143
listen(imap): localhost:143
listen(pop3): *
ssl_listen(default): *:993
ssl_listen(imap): *:993
ssl_listen(pop3): *:995
ssl_cert_file: /etc/ssl/certs/mail.bsd-unix.net.pem
ssl_key_file: /etc/ssl/certs/mail.bsd-unix.net.pem
login_dir: /var/run/dovecot/login
login_executable(default): /usr/local/libexec/dovecot/imap-login
login_executable(imap): /usr/local/libexec/dovecot/imap-login
login_executable(pop3): /usr/local/libexec/dovecot/pop3-login
verbose_proctitle: yes
first_valid_uid: 1000
first_valid_gid: 100
mail_privileged_group: mail
mail_location: mbox:~/mail/:INBOX=/var/mail/%u
mail_executable(default): /usr/local/libexec/dovecot/imap
mail_executable(imap): /usr/local/libexec/dovecot/imap
mail_executable(pop3): /usr/local/libexec/dovecot/pop3
mail_process_size: 1024
mail_plugins(default): fts fts_squat
mail_plugins(imap): fts fts_squat
mail_plugins(pop3): 
mail_plugin_dir(default): /usr/local/lib/dovecot/imap
mail_plugin_dir(imap): /usr/local/lib/dovecot/imap
mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3
imap_client_workarounds(default): delay-newmail outlook-idle netscape-eoh 
tb-extra-mailbox-sep
imap_client_workarounds(imap): delay-newmail outlook-idle netscape-eoh 
tb-extra-mailbox-sep
imap_client_workarounds(pop3): 
pop3_client_workarounds(default): 
pop3_client_workarounds(imap): 
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
lda:
  postmaster_address: postmas...@example.com
  sendmail_path: /usr/sbin/sendmail
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd
  socket:
type: listen
client:
  path: /var/spool/postfix/private/auth
  mode: 384
  user: postfix
  group: postfix
plugin:
  fts: squat


FreeBSD triton.bsd-unix.net 7.2-STABLE FreeBSD 7.2-STABLE #17: Sat Oct  3 
15:47:44 EDT 2009 r...@triton.bsd-unix.net:/usr/obj/usr/src/sys/TRITON  
amd64

dovecot-1.2.4_1 Secure and compact IMAP and POP3 servers

CPU Arch: x86_64

Filesystem: UFS

(gdb) bt full
#0  0x40b2989c in kill () from /lib/libc.so.7
No symbol table info available.
#1  0x40b286bb in abort () from /lib/libc.so.7
No symbol table info available.
#2  0x004d9165 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) 
at failures.c:160
backtrace = 0x40d4d048 ""
#3  0x004d9f7e in i_internal_fatal_handler (type=Could not find the 
frame base for "i_internal_fatal_handler".
) at failures.c:440
No locals.
#4  0x004d944a in i_panic (format=Could not find the frame base for 
"i_panic".
) at failures.c:207
args = Could not find the frame base for "i_panic".
(gdb) bt
#0  0x40b2989c in kill () from /lib/libc.so.7
#1  0x40b286bb in abort () from /lib/libc.so.7
#2  0x004d9165 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) 
at failures.c:160
#3  0x004d9f7e in i_internal_fatal_handler (type=Could not find the 
frame base for "i_internal_fatal_handler".
) at failures.c:440
#4  0x004d944a in i_panic (format=Could not find the frame base for 
"i_panic".
) at failures.c:207
#5  0x0045aea8 in istream_raw_mbox_get_body_size (stream=0x40d76070, 
expected_body_size=18446744073709551615) at istream-raw-mbox.c:578
#6  0x004622f3 in mbox_sync_read_next_mail (sync_ctx=0x7fffe290, 
mail_ctx=0x7fffe0b0) at mbox-sync.c:151
#7  0x004649f6 in mbox_sync_loop (sync_ctx=0x7fffe290, 
mail_ctx=0x7fffe0b0, partial=false) at mbox-sync.c:1004
#8  0x00465c00 in mbox_sync_do (sync_ctx=0x7fffe290, 
flags=MBOX_SYNC_UNDIRTY) at mbox-sync.c:1535
#9  0x004667b8 in mbox_sync_int (mbox=0x40d23848, 
flags=MBOX_SYNC_UNDIRTY, lock_id=0x7fffe4b8) at mbox-sync.c:1842
#10 0x00466a26 in mbox_sync (mbox=0x40d23848, flags=MBOX_SYNC_UNDIRTY) 
at mbox-sync.c:1894
#11 0x00466b8a in mbox_storage_sync_init (box=0x40d23848, flags=65) at 
mbox-sync.c:1936
#12 0x00488d34 in mailbox_sync_init (box=0x40d23848, flags=65) at 
mail-storage.c:562
#13 0x00488dfa in mailbox_sync (box=0x40d23848, flags=65, 
status_items=239, status_r=0x7fffe580) at mail-storage.c:593
#14 0x0041f77a in select_open (ctx=0x40d4d148, mailbox=0x40d52170 
"Drafts", readonly=false) at cmd-select.c:273
#15 0x0041fba0 in cmd_select_full (cmd=0x40d4d048, readonly=false) at 
cmd-select.c:381
#16 0x0041fbda in cmd_select (cmd=0x40d4d048) at cmd-select.c:388
#17 0x00423164 in client_command_input (cmd=0x40d4d048) at client.c:611
#18 0x004233a1 in client_command_input (cmd=0x40d4d048) at client.c:660
#19 0x004234e3 in client_handle_next_command (client=0x40d45030, 
remove_io_r=0x7fffe73d) at client.c:701
#20 0x00423568 in clien

Re: [Dovecot] Bug Report in Connection with Expire Plugin

[Timo Sirainen]

On Fri, 2008-12-26 at 11:10 +0100, Jörg Leis wrote:
> I tried using a "user_field = user" additionally as is written here: 
> http://wiki.dovecot.org/Plugins/Expire. (Incidentally, there must be 
> some mismatch in the documentation because the tables for v1.1 do not 
> contain a field username). Of course I adapted the MySQL database 
> accordingly.

I updated the page to contain "username_field = not_used".

> Unfortunately I was unable to produce a backtrace.

I couldn't reproduce the crash with v1.1.8. And without backtrace or a
way to reproduce it I can't really do anything..

The backtrace would be easiest to get with:

 - Log in to IMAP
 - gdb -p `pidof imap`
cont
 - Make it crash
bt full

Although if you didn't compile Dovecot yourself the backtrace probably
won't be all that useful since there are no debug symbols.



signature.asc
Description: This is a digitally signed message part

[Dovecot] Bug Report on Expire PlugIn (update)

[Jörg Leis]

I did the same thing on the console and this is what happens:

bash-3.2$ openssl s_client -connect example.com:993
CONNECTED(0003)
[...]
Server certificate
-BEGIN CERTIFICATE-
[...]
-END CERTIFICATE-
[...]
CA
---
No client certificate CA names sent
---
SSL handshake has read 4595 bytes and written 322 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Compression: NONE
Expansion: NONE
SSL-Session:
[...]
---
* OK Dovecot ready
. login ad...@example.com secret
. OK Logged in.
. select INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft KMAILFORWARDED 
KMAILTODO KMAILWATCHED KMAILIGNORED $FORWARDED $TODO $WATCHED $IGNORED 
NonJunk)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft 
KMAILFORWARDED KMAILTODO KMAILWATCHED KMAILIGNORED $FORWARDED $TODO 
$WATCHED $IGNORED NonJunk \*)] Flags permitted.

* 4 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1230136990] UIDs valid
* OK [UIDNEXT 77] Predicted next UID
. OK [READ-WRITE] Select completed.
. copy 1 Trash
closed

[Dovecot] Bug Report in Connection with Expire Plugin

[Jörg Leis]

If the expire plugin is enabled and a new message (not in trash) is 
deleted via IMAP in Mozilla Thunderbird (only tested in this client), it 
will indefinately copy the message into the folder. Dovecot, meanwhile, 
occupies itself with this:



r...@server:~# cat /var/log/dovecot.log
[...]
dovecot: 2008-12-26 09:21% Error: child 6132 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6133 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6134 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6135 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6136 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6138 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6139 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6140 (imap) killed with signal 11
dovecot: 2008-12-26 09:21% Error: child 6142 (imap) killed with signal 11
[...]


r...@server:~# cat /var/log/dovecot.info
[...]
dovecot: 2008-12-26 09:21% Info: imap-login: Login: 
user=, method=CRAM-MD5, rip=84.75.159.19, 
lip=92.42.190.87, TLS
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Loading modules 
from directory: /usr/lib/dovecot/modules/imap
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib10_quota_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib20_expire_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Effective 
uid=5000, gid=5000, home=/home/vmail/Maildirs/bildig.ch/admin
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota root: 
name=User Quota backend=maildir args=
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota rule: 
root=User Quota mailbox=* bytes=524288000 (0%) messages=0 (0%)
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota rule: 
root=User Quota mailbox=Trash bytes=52428800 (0%) messages=0 (0%)
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota warning: 
bytes=498073600 (95%) messages=0 (0%) 
command=/usr/local/bin/mail-quota-warning.sh 95
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota warning: 
bytes=419430400 (80%) messages=0 (0%) 
command=/usr/local/bin/mail-quota-warning.sh 80
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): maildir: 
data=/home/vmail/Maildirs/bildig.ch/admin
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): maildir++: 
root=/home/vmail/Maildirs/bildig.ch/admin, index=, control=, 
inbox=/home/vmail/Maildirs/bildig.ch/admin
dovecot: 2008-12-26 09:21% Info: imap-login: Login: 
user=, method=CRAM-MD5, rip=84.75.159.19, 
lip=92.42.190.87, TLS
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Loading modules 
from directory: /usr/lib/dovecot/modules/imap
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib10_quota_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib11_imap_quota_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Module loaded: 
/usr/lib/dovecot/modules/imap/lib20_expire_plugin.so
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Effective 
uid=5000, gid=5000, home=/home/vmail/Maildirs/bildig.ch/admin
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota root: 
name=User Quota backend=maildir args=
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota rule: 
root=User Quota mailbox=* bytes=524288000 (0%) messages=0 (0%)
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota rule: 
root=User Quota mailbox=Trash bytes=52428800 (0%) messages=0 (0%)
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota warning: 
bytes=498073600 (95%) messages=0 (0%) 
command=/usr/local/bin/mail-quota-warning.sh 95
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): Quota warning: 
bytes=419430400 (80%) messages=0 (0%) 
command=/usr/local/bin/mail-quota-warning.sh 80
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): maildir: 
data=/home/vmail/Maildirs/bildig.ch/admin
dovecot: 2008-12-26 09:21% Info: IMAP(ad...@bildig.ch): maildir++: 
root=/home/vmail/Maildirs/bildig.ch/admin, index=, control=, 
inbox=/home/vmail/Maildirs/bildig.ch/admin

[...]


When the expire plugin is disabled no error occurs. I use Dovecot 1.1.4 
on Ubuntu 8.10 Intrepid with following settings:



r...@server:~# dovecot -n
# 1.1.4: /etc/dovecot/dovecot.conf
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot.log
info_log_path: /var/log/dovecot.info
log_timestamp: %Y-%m-%d %H:%M%
protocols: imaps pop3s
ssl_cert_file: /etc/ssl/certs/www_bildig_ch__chained.crt
ssl_key_file: /etc/ssl/private/www_bildig_ch.key
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib/dovecot/imap-login
login_execut

[Dovecot] Fwd: Re: Fwd: dovecot bug report (64 bit file offsets)

[markevich]

Yes, the problem is the compatibility with the mysql library but I stull can't 
figure 
out how this might affect reading/writing.

-- Forwarded message --
  No: [EMAIL PROTECTED]
 Kam: Marcus Rueckert <[EMAIL PROTECTED]>
Atbildt: [EMAIL PROTECTED]
  Datums: Tue, 27 Nov 2007 14:15:29 +0200
  Temats: Re: [Dovecot] Fwd: dovecot bug report (64 bit file offsets)

I agree that it’s kind strange behavior. 

Also I’ve googled that exactly the same errors (unexpected eof
found … on file ‘UNOPENED’) were related to mysql.
But how mysql compile options might affect configuration file parsing
and read/write operations? (indexes for example) Does any one have
any idea why this might happen? 

“any slight change 
to the main dovecot.conf file can prevent dovecot from starting
(segfault 
with "Unexpected eof found when reading file 'UNOPENED' (Errcode: 0)"
message), whereas i386 packages work somewhat better.” 

http://dovecot.org/list/dovecot/2006-April/012340.html 

Regards, Andris 
Quoting Marcus Rueckert :
which is funny actually as on opensuse (10.1 -> 10.3 -> factory) has
no
problem with dovecot running on x86_64.

darix

-- 
openSUSE - SUSE Linux is my linux
openSUSE is good for you
www.opensuse.org
-- Tavs bezmaksas pasts Inbox.lv

Links:
--
[1] mailto:[EMAIL PROTECTED]
-- End of forwarded message --

-- Tavs bezmaksas pasts Inbox.lv

Re: [Dovecot] Fwd: dovecot bug report (64 bit file offsets)

[Marcus Rueckert]

On 2007-11-27 14:15:29 +0200, [EMAIL PROTECTED] wrote:
> I agree that it’s kind strange behavior. 
> 
> Also I’ve googled that exactly the same errors (unexpected eof
> found … on file ‘UNOPENED’) were related to mysql.
> But how mysql compile options might affect configuration file parsing
> and read/write operations? (indexes for example) Does any one have
> any idea why this might happen? 
> 
> “any slight change 
> to the main dovecot.conf file can prevent dovecot from starting
> (segfault 
> with "Unexpected eof found when reading file 'UNOPENED' (Errcode: 0)"
> message), whereas i386 packages work somewhat better.” 
> 
> http://dovecot.org/list/dovecot/2006-April/012340.html 

maybe some overlapping function names? i am not sure how fedora builds
mysql and dovecot. maybe give opensuse a try and see if it fails in your
setup.

darix

-- 
   openSUSE - SUSE Linux is my linux
   openSUSE is good for you
   www.opensuse.org

Re: [Dovecot] Fwd: dovecot bug report (64 bit file offsets)

[markevich]

I agree that it’s kind strange behavior. 

Also I’ve googled that exactly the same errors (unexpected eof
found … on file ‘UNOPENED’) were related to mysql.
But how mysql compile options might affect configuration file parsing
and read/write operations? (indexes for example) Does any one have
any idea why this might happen? 

“any slight change 
to the main dovecot.conf file can prevent dovecot from starting
(segfault 
with "Unexpected eof found when reading file 'UNOPENED' (Errcode: 0)"
message), whereas i386 packages work somewhat better.” 

http://dovecot.org/list/dovecot/2006-April/012340.html 

Regards, Andris 
Quoting Marcus Rueckert :
which is funny actually as on opensuse (10.1 -> 10.3 -> factory) has
no
problem with dovecot running on x86_64.

darix

-- 
openSUSE - SUSE Linux is my linux
openSUSE is good for you
www.opensuse.org
-- Tavs bezmaksas pasts Inbox.lv

Links:
--
[1] mailto:[EMAIL PROTECTED]

Re: [Dovecot] Fwd: dovecot bug report (64 bit file offsets)

[Marcus Rueckert]

On 2007-11-27 11:21:16 +0200, [EMAIL PROTECTED] wrote:
> I have found similar problem: 
> http://www.dovecot.org/list/dovecot/2007-January/018623.html 
> "After configure remove HAVE_PREAD from config.h, recompile and see
> if it works." 
> 
> I think I figured out where might be the problem. If the binary is
> compiled in the 64bit environment and you want to use 64 bit file
> offset you must use 64 bit equivalents of the function. Of course if
> you are using open/pread (which are 32 bit compatible only) the whole
> program functionality will be broken. You must use open64/pread64 etc.
> Else macros _LARGE_FILE_API should be defined in order to use large
> files in other case nothing will work (random parsing errors and dead
> indexes will appear ). As I can see there is a big issue in the
> dovecot. 
> 
> http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/apis/pread64.htm
> 
> 
> Regards, Andris 
> -- Forwarded message -- 
> No: [EMAIL PROTECTED] 
> Kam: [EMAIL PROTECTED] 
> Reply-To: [EMAIL PROTECTED] 
> Date: Tue, 27 Nov 2007 10:45:23 +0200 
> Temats: dovecot bug report (64 bit file offsets) 
> Hello. I found 2 issues while using dovecot in the 64-bit system.
> Before that I was using it on the Solaris 10 32bit and everything
> worked fine. 
> 
> First issue is that it's impossible to specify Openssl library
> directory, there is only possiblity to specify type gnutls/openssl.
> But if there are 2 versions of openssl installed it's hard to specify
> which one we want to use, we need to play with AUXLIBS and LD flags to
> prevent looking in un-needed locations. I'm suggesting to add new
> ./configure key like --with-ssl-dir like all other programs do (why
> dovecot hasn't it?). For example in FC8 openssl 0.9.8b is available
> in the packages but latest is 0.9.8.g, and it's impossible to compile
> dovecot with 0.9.8b because of the missing function in it which
> present in 0.9.8.g. 
> 
> Second and most serious bug is that it seems that dovecot is unable
> to use 64 file offsets correctly. First problem was that I as unable
> to launch dovecot at all - it gave a lot of configuration file
> (dovecot.conf) parsing errors (which is working perfectly on the 32
> bit Solaris). It was very strange because he complained that he is
> expecting to see '=' in the line where only comments are located.
> Deleting some lines at the start of the configuration file produced
> the error on random lines (I can't describe it in other way) but with
> the comments too. For example, first error complained that line 42 is
> incorrect (only comments are there), after deletion of 2 empty lines
> at the beginning dovecot complained that line 172 has errors but any
> way there are only comments there too. I've deleted all of the
> comments from the configuration file but still dovecot randomly is
> giving such errors. Any way dovecot is UNABLE to parse all my file, I
> can see it from dovecot -n. He is starting normally (time to time) but
> only half of the configuration file is read. Similar problems goes
> with the indexes dovecot.index.log - dovecot is able to create the
> files but is unable to read them after any access to it dovecot
> generates errors like unexpected eof found and file is corrupted.
> From all this I can assume that 64bit file offsets on a 64 bit system
> aren't working at all. 
> 
> Latest dovecot 1.0.7 (also tried 1.0.5) 
> Fedora core 8 64bit (with latest patches and updates) 
> Intel core duo 6300 (x86_64) 
> XFS (only local filesystem) 

which is funny actually as on opensuse (10.1 -> 10.3 -> factory) has no
problem with dovecot running on x86_64.

darix

-- 
   openSUSE - SUSE Linux is my linux
   openSUSE is good for you
   www.opensuse.org

[Dovecot] Fwd: dovecot bug report (64 bit file offsets)

[markevich]

I have found similar problem: 
http://www.dovecot.org/list/dovecot/2007-January/018623.html 
"After configure remove HAVE_PREAD from config.h, recompile and see
if it works." 

I think I figured out where might be the problem. If the binary is
compiled in the 64bit environment and you want to use 64 bit file
offset you must use 64 bit equivalents of the function. Of course if
you are using open/pread (which are 32 bit compatible only) the whole
program functionality will be broken. You must use open64/pread64 etc.
Else macros _LARGE_FILE_API should be defined in order to use large
files in other case nothing will work (random parsing errors and dead
indexes will appear ). As I can see there is a big issue in the
dovecot. 

http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/apis/pread64.htm


Regards, Andris 
-- Forwarded message -- 
No: [EMAIL PROTECTED] 
Kam: [EMAIL PROTECTED] 
Reply-To: [EMAIL PROTECTED] 
Date: Tue, 27 Nov 2007 10:45:23 +0200 
Temats: dovecot bug report (64 bit file offsets) 
Hello. I found 2 issues while using dovecot in the 64-bit system.
Before that I was using it on the Solaris 10 32bit and everything
worked fine. 

First issue is that it's impossible to specify Openssl library
directory, there is only possiblity to specify type gnutls/openssl.
But if there are 2 versions of openssl installed it's hard to specify
which one we want to use, we need to play with AUXLIBS and LD flags to
prevent looking in un-needed locations. I'm suggesting to add new
./configure key like --with-ssl-dir like all other programs do (why
dovecot hasn't it?). For example in FC8 openssl 0.9.8b is available
in the packages but latest is 0.9.8.g, and it's impossible to compile
dovecot with 0.9.8b because of the missing function in it which
present in 0.9.8.g. 

Second and most serious bug is that it seems that dovecot is unable
to use 64 file offsets correctly. First problem was that I as unable
to launch dovecot at all - it gave a lot of configuration file
(dovecot.conf) parsing errors (which is working perfectly on the 32
bit Solaris). It was very strange because he complained that he is
expecting to see '=' in the line where only comments are located.
Deleting some lines at the start of the configuration file produced
the error on random lines (I can't describe it in other way) but with
the comments too. For example, first error complained that line 42 is
incorrect (only comments are there), after deletion of 2 empty lines
at the beginning dovecot complained that line 172 has errors but any
way there are only comments there too. I've deleted all of the
comments from the configuration file but still dovecot randomly is
giving such errors. Any way dovecot is UNABLE to parse all my file, I
can see it from dovecot -n. He is starting normally (time to time) but
only half of the configuration file is read. Similar problems goes
with the indexes dovecot.index.log - dovecot is able to create the
files but is unable to read them after any access to it dovecot
generates errors like unexpected eof found and file is corrupted.
From all this I can assume that 64bit file offsets on a 64 bit system
aren't working at all. 

Latest dovecot 1.0.7 (also tried 1.0.5) 
Fedora core 8 64bit (with latest patches and updates) 
Intel core duo 6300 (x86_64) 
XFS (only local filesystem) 

Regards, Andris 

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Timo Sirainen]

On Sat, 2007-09-08 at 12:36 -0400, Bill Cole wrote:
> The only way a program 
> could know of the failure would be to trace the syslog configuration 
> and look for whether messages are appearing, then take action 
> (probably calling closelog() and openlog()) as needed.

I already sent a patch that did closelog() when reopening logs, but that
didn't help. I forgot openlog(), but that shouldn't be needed..



signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

On Sep 8, 2007, at 12:36 PM, Bill Cole wrote:

If you run them, you would also note that Postfix, BIND, INN, and  
MIMEDefang all suffer from the same problem. In fact, I don't know  
of any non-Apple daemon that does not have this problem. I believe  
it is a bug in Apple's ground-up re-implementation of system  
logging in Tiger.


Thanks for the detailed explanation, Bill.  Guess I'll stick with my  
nightly routine.


B. Bodger

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bill Cole]

At 9:54 AM -0400 9/8/07, Bruce Bodger wrote:

Timo,

Compiled and installed v1.0.4 this morning on OS X 10.4.10...

./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl 
--with-notify=kqueue


...all is well.

I wonder if you remember the conversation that we had back on May 
11.  This was the situation where after the system ran nightly 
maintenance dovecot info stopped appearing in mail.log (I'm being 
careful how I state that: I believe you had surmised a problem with 
Apple's syslogd implementation.


If you run them, you would also note that Postfix, BIND, INN, and 
MIMEDefang all suffer from the same problem. In fact, I don't know of 
any non-Apple daemon that does not have this problem. I believe it is 
a bug in Apple's ground-up re-implementation of system logging in 
Tiger.


The condition is a silent failure for clients of the syslog subsystem 
using the BSD standard (i.e. portable) API. There's no feedback from 
the system to alert a caller to syslog() that the call didn't 
actually result in anything getting written to a log file, since the 
call returns no value and doesn't set errno. The only way a program 
could know of the failure would be to trace the syslog configuration 
and look for whether messages are appearing, then take action 
(probably calling closelog() and openlog()) as needed. Alternatively, 
a program might just do that as housekeeping every few minutes, but 
that seems like an awfully busy bit of accommodation for a broken OS 
subsystem.


In short: Apple needs to fix this. It's their bug, created by their 
decision to replace traditional BSD syslog with fresh new code 
written by people who couldn't be bothered to look at what had been 
done by others before them or at how their code interoperated with 
existing widely used software. Just like the embarrassment called 
LaunchD, ASL is the replacement of a *very mature* set of software 
with software that in the same metaphor might best be termed 
*infantile*. Hopefully both will at least reach juvenile or 
adolescent status in Leopard...




--
Bill Cole  
[EMAIL PROTECTED]

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

Timo,

Compiled and installed v1.0.4 this morning on OS X 10.4.10...

./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl  
--with-notify=kqueue


...all is well.

I wonder if you remember the conversation that we had back on May  
11.  This was the situation where after the system ran nightly  
maintenance dovecot info stopped appearing in mail.log (I'm being  
careful how I state that: I believe you had surmised a problem with  
Apple's syslogd implementation.


Ever since our conversation I've been running these commands in  
daily.local... after maintenance rolls the logs...


killall dovecot
sleep 1
/usr/sbin/postfix reload

Of course, launchd restarts dovecot.

That's been working just fine... no problems at all.  But now that  
we're up to dovecot 1.0.4 I'm wondering if you've ever looked into  
this anomaly...  Perhaps things have changed regarding logging?  I  
don't mind testing it here without the nightly kill but before I do,  
wondered if you ever made any adjustments in the code for it.


Thanks,
B. Bodger


On May 11, 2007, at 11:27 AM, Timo Sirainen wrote:


On Fri, 2007-05-11 at 10:57 -0400, Bruce Bodger wrote:


Applied the patch, recompiled v1.1.0 using same line to config as
before..

./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl

Sorry... no change: does not fix it.


In that case there's nothing I can do. You'll just have to restart
Dovecot. Or make it log directly into files instead of using syslog
(log_path setting).

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Timo Sirainen]

On Fri, 2007-05-11 at 10:57 -0400, Bruce Bodger wrote:

> Applied the patch, recompiled v1.1.0 using same line to config as  
> before..
> 
> ./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl
> 
> Sorry... no change: does not fix it.

In that case there's nothing I can do. You'll just have to restart
Dovecot. Or make it log directly into files instead of using syslog
(log_path setting).



signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

Applied the patch, recompiled v1.1.0 using same line to config as  
before..


./configure --with-ssldir=/System/Library/OpenSSL --with-ssl=openssl

Sorry... no change: does not fix it.

Regards,
Bruce

On May 11, 2007, at 10:24 AM, Timo Sirainen wrote:


On Fri, 2007-05-11 at 09:26 -0400, Bruce Bodger wrote:


That would be accurate, Bill.  I suspect the problem to be within
dovecot since, as I mentioned earlier, neither uw-imap or other
running daemons exhibit this problem.


That's probably because UW-IMAP and most other servers keep  
creating new

processes which open new connections to syslog, while Dovecot has one
long-running dovecot process.

If you apply the patch below and send SIGUSR1 signal to dovecot  
process,

does it fix it?

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Timo Sirainen]

On Fri, 2007-05-11 at 09:26 -0400, Bruce Bodger wrote:

> That would be accurate, Bill.  I suspect the problem to be within  
> dovecot since, as I mentioned earlier, neither uw-imap or other  
> running daemons exhibit this problem. 

That's probably because UW-IMAP and most other servers keep creating new
processes which open new connections to syslog, while Dovecot has one
long-running dovecot process.

If you apply the patch below and send SIGUSR1 signal to dovecot process,
does it fix it?

Index: src/master/main.c
===
RCS file: /var/lib/cvs/dovecot/src/master/main.c,v
retrieving revision 1.80.2.21
diff -u -r1.80.2.21 main.c
--- src/master/main.c   11 May 2007 13:28:52 -  1.80.2.21
+++ src/master/main.c   11 May 2007 14:23:59 -
@@ -168,6 +168,7 @@
 static void sig_reopen_logs(int signo __attr_unused__,
void *context __attr_unused__)
 {
+   closelog();
set_logfile(settings_root->defaults);
 }
 



signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

On May 11, 2007, at 8:48 AM, Bill Cole wrote:

My system.log is rotated nightly followed by a pretty standard  
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/ 
syslog.pid | head -1);


That's not a restart, it is a 'hangup' signal that will make a  
normal syslogd reread the config file. The problem with that is  
that when you do it, Apple's syslogd completely stops logging. I'm  
not sure why that is, but I've been working around it since the  
10.4 release and Apple has had a bug report for that long.


If you actually kill syslogd with a TERM or KILL signal it will be  
restarted by launchd and it should log properly.


Of course you're right regarding HANGUP vs. restart.  However, that's  
the line that appears in Apple's default daily cron job.


I've just tried modifying it to remove -HUP from the line and force  
it to truly be restarted by launchd.  Same result regarding dovecot,  
though.



Killing and restarting dovecot causes it to write to the log again.


That's a bit surprising, since if you are using syslog, the dovecot  
processes don't know anything about the specific log files that  
syslog  is using. The implication of what you are seeing is that  
the breakage is between dovecot and syslogd, and that restarting  
dovecot fixes it.


That would be accurate, Bill.  I suspect the problem to be within  
dovecot since, as I mentioned earlier, neither uw-imap or other  
running daemons exhibit this problem.


Thanks for the feedback.

Regards,
Bruce

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bill Cole]

At 7:04 AM -0400 5/11/07, Bruce Bodger  imposed structure on a stream 
of electrons, yielding:

It appears as though dovecot doesn't like syslogd being restarted.


That's probably not it.

The whole point of syslog is for individual message sources (e.g. 
dovecot) to not have to worry about where the messages go.


This is actually at least partly a bug in the MacOS X syslog 
implementation. With 10.4, Apple decided that neither traditional BSD 
syslog or syslog-ng was what they wanted, so they reimplemented from 
the ground up and did a lousy job.


My system.log is rotated nightly followed by a pretty standard 
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat 
/var/run/syslog.pid | head -1);


That's not a restart, it is a 'hangup' signal that will make a normal 
syslogd reread the config file. The problem with that is that when 
you do it, Apple's syslogd completely stops logging. I'm not sure why 
that is, but I've been working around it since the 10.4 release and 
Apple has had a bug report for that long.


If you actually kill syslogd with a TERM or KILL signal it will be 
restarted by launchd and it should log properly.




It appears from that point on dovecot no longer writes to mail.log.



Dovecot was never writing to mail.log. Syslogd was, and it was broken.


Killing and restarting dovecot causes it to write to the log again.


That's a bit surprising, since if you are using syslog, the dovecot 
processes don't know anything about the specific log files that 
syslog  is using. The implication of what you are seeing is that the 
breakage is between dovecot and syslogd, and that restarting dovecot 
fixes it.




--
Bill Cole  
[EMAIL PROTECTED]

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Charles Marcus]

Bruce Bodger wrote:
Thank you very much, Charles.  I have resorted to that workaround 
(restarting dovecot) which does, indeed, work.  I just wanted to 
document the situation since dovecot's behavior in this regard is unlike 
other imap servers.


This thread mentioned only sending an HUP - or maybe even just a USR1 - 
signal - so, maybe restarting dovecot (which interrupts all processes) 
is not necessary?


Specifically:


With 1.0-tests you could instead send HUP signal which makes Dovecot
reopen the log files as well as re-reading the configuration files.
Hmm. Looks like Apache uses USR1 signal as well for just log 
reopening, I'll add that too.


--

Best regards,

Charles

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

Thank you very much, Charles.  I have resorted to that workaround  
(restarting dovecot) which does, indeed, work.  I just wanted to  
document the situation since dovecot's behavior in this regard is  
unlike other imap servers.


Best regards,
Bruce


On May 11, 2007, at 8:27 AM, Charles Marcus wrote:


Bruce Bodger wrote:

It appears as though dovecot doesn't like syslogd being restarted.


Hmmm... could this possibly be a fix, or at least a workaround?

www.dovecot.org/list/dovecot/2004-November/005219.html

--

Best regards,

Charles

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Charles Marcus]

Bruce Bodger wrote:

It appears as though dovecot doesn't like syslogd being restarted.


Hmmm... could this possibly be a fix, or at least a workaround?

www.dovecot.org/list/dovecot/2004-November/005219.html

--

Best regards,

Charles

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

It appears as though dovecot doesn't like syslogd being restarted.

My system.log is rotated nightly followed by a pretty standard  
syslogd restart...


if [ -f /var/run/syslog.pid ]; then kill -HUP $(cat /var/run/ 
syslog.pid | head -1);


It appears from that point on dovecot no longer writes to mail.log.   
Killing and restarting dovecot causes it to write to the log again.


If you care to take a stab at a suggestion, please kind in mind that  
I'm having to run dovecot with the -F switch (see below).


Thanks in advance,

B. Bodger
Ronkonkoma, NY



On May 11, 2007, at 6:17 AM, Bruce Bodger wrote:

Please disregard the below reported bug report.  I experienced the  
same problem today and this time "sleep" was not involved.


I rotate mail.log weekly.  This morning I noticed that dovecot was  
no longer logging to mail.log although, again, it was running fine.


Is it possible that dovecot has a problem after midnight and will  
not log with the new day?  Something is stopping dovecot from  
logging and it appears that it might have to do with change of the  
day?


This is a new (two day old) dovecot installation.  Coming from uw- 
imap.  It never exhibited this problem.


Thank you,
B. Bodger
Ronkonkoma, NY


On May 10, 2007, at 7:34 AM, Bruce Bodger wrote:


Version: dovecot --version:  1.0.0
OS: OS X - 10.4.9
Architecture: G5 Dual/1.8Ghz

Had a fairly normal time correction last night:
May  9 20:56:56 G518X2 ntpd[219]: time set -1.212733 s

Which dovecot duly noted:
May  9 20:56:56 G518X2 dovecot: auth(default): Time just moved  
backwards by 1 seconds. I'll sleep now until we're back in present.


This morning all of the dovecot processes are running fine but  
dovecot has stopped logging.  (I have modified nothing in  
dovecot.conf relative to logging: dovecot logs to syslog using the  
"mail" syslog_facility.)


ps -aux | grep dovecot
root53   0.0 -0.027772544  ??  Ss4:23PM
0:04.69 /usr/local/sbin/dovecot -F
root79   0.0 -0.028176556  ??  S 4:23PM
0:03.21 dovecot-auth
dovecot   9548   0.0 -0.027780496  ??  S 7:12AM
0:00.05 imap-login
dovecot   9631   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login
dovecot   9632   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login


I am starting dovecot via OS X's launchdaemon facility thus the "- 
F" command line switch requirement.


Sending a "kill -HUP" to the process did not get it to start logging.
Sending a "Kill -9" to the process forced a restart and logging  
began again.


ps -aux | grep dovecot
root  9738   0.0 -0.027772540  ??  Ss7:26AM
0:00.03 /usr/local/sbin/dovecot -F
root  9739   0.0 -0.028176544  ??  S 7:26AM
0:00.02 dovecot-auth
dovecot   9740   0.0 -0.027780500  ??  S 7:26AM
0:00.02 imap-login
dovecot   9751   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login
dovecot   9762   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login


Before posting this bug I compared v1.0.0's /src/lib/ioloop.c to  
that in the nightly snapshot and saw no adjustments in, what I  
presumed to be, the relevant portion of the code.


Output of "dovecot -n"

protocols: imap
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
first_valid_uid: 490
mail_extra_groups: mail
mail_location: mbox:~/Library/Mail/IMAP:INBOX=/var/mail/%u:INDEX=% 
h/Library/Mail/DOVECOT.indexes

dotlock_use_excl: yes
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd

Thank you.

Re: [Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

Please disregard the below reported bug report.  I experienced the  
same problem today and this time "sleep" was not involved.


I rotate mail.log weekly.  This morning I noticed that dovecot was no  
longer logging to mail.log although, again, it was running fine.


Is it possible that dovecot has a problem after midnight and will not  
log with the new day?  Something is stopping dovecot from logging and  
it appears that it might have to do with change of the day?


This is a new (two day old) dovecot installation.  Coming from uw- 
imap.  It never exhibited this problem.


Thank you,
B. Bodger
Ronkonkoma, NY


On May 10, 2007, at 7:34 AM, Bruce Bodger wrote:


Version: dovecot --version:  1.0.0
OS: OS X - 10.4.9
Architecture: G5 Dual/1.8Ghz

Had a fairly normal time correction last night:
May  9 20:56:56 G518X2 ntpd[219]: time set -1.212733 s

Which dovecot duly noted:
May  9 20:56:56 G518X2 dovecot: auth(default): Time just moved  
backwards by 1 seconds. I'll sleep now until we're back in present.


This morning all of the dovecot processes are running fine but  
dovecot has stopped logging.  (I have modified nothing in  
dovecot.conf relative to logging: dovecot logs to syslog using the  
"mail" syslog_facility.)


ps -aux | grep dovecot
root53   0.0 -0.027772544  ??  Ss4:23PM
0:04.69 /usr/local/sbin/dovecot -F
root79   0.0 -0.028176556  ??  S 4:23PM
0:03.21 dovecot-auth
dovecot   9548   0.0 -0.027780496  ??  S 7:12AM
0:00.05 imap-login
dovecot   9631   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login
dovecot   9632   0.0 -0.027780500  ??  S 7:17AM
0:00.03 imap-login


I am starting dovecot via OS X's launchdaemon facility thus the "- 
F" command line switch requirement.


Sending a "kill -HUP" to the process did not get it to start logging.
Sending a "Kill -9" to the process forced a restart and logging  
began again.


ps -aux | grep dovecot
root  9738   0.0 -0.027772540  ??  Ss7:26AM
0:00.03 /usr/local/sbin/dovecot -F
root  9739   0.0 -0.028176544  ??  S 7:26AM
0:00.02 dovecot-auth
dovecot   9740   0.0 -0.027780500  ??  S 7:26AM
0:00.02 imap-login
dovecot   9751   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login
dovecot   9762   0.0 -0.027780500  ??  S 7:27AM
0:00.01 imap-login


Before posting this bug I compared v1.0.0's /src/lib/ioloop.c to  
that in the nightly snapshot and saw no adjustments in, what I  
presumed to be, the relevant portion of the code.


Output of "dovecot -n"

protocols: imap
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
first_valid_uid: 490
mail_extra_groups: mail
mail_location: mbox:~/Library/Mail/IMAP:INBOX=/var/mail/%u:INDEX=%h/ 
Library/Mail/DOVECOT.indexes

dotlock_use_excl: yes
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd

Thank you.

[Dovecot] BUG REPORT - Stops logging after sleep

[Bruce Bodger]

Version: dovecot --version:  1.0.0
OS: OS X - 10.4.9
Architecture: G5 Dual/1.8Ghz

Had a fairly normal time correction last night:
May  9 20:56:56 G518X2 ntpd[219]: time set -1.212733 s

Which dovecot duly noted:
May  9 20:56:56 G518X2 dovecot: auth(default): Time just moved  
backwards by 1 seconds. I'll sleep now until we're back in present.


This morning all of the dovecot processes are running fine but  
dovecot has stopped logging.  (I have modified nothing in  
dovecot.conf relative to logging: dovecot logs to syslog using the  
"mail" syslog_facility.)


ps -aux | grep dovecot
root53   0.0 -0.027772544  ??  Ss4:23PM
0:04.69 /usr/local/sbin/dovecot -F
root79   0.0 -0.028176556  ??  S 4:23PM   0:03.21  
dovecot-auth
dovecot   9548   0.0 -0.027780496  ??  S 7:12AM   0:00.05  
imap-login
dovecot   9631   0.0 -0.027780500  ??  S 7:17AM   0:00.03  
imap-login
dovecot   9632   0.0 -0.027780500  ??  S 7:17AM   0:00.03  
imap-login


I am starting dovecot via OS X's launchdaemon facility thus the "-F"  
command line switch requirement.


Sending a "kill -HUP" to the process did not get it to start logging.
Sending a "Kill -9" to the process forced a restart and logging began  
again.


ps -aux | grep dovecot
root  9738   0.0 -0.027772540  ??  Ss7:26AM
0:00.03 /usr/local/sbin/dovecot -F
root  9739   0.0 -0.028176544  ??  S 7:26AM   0:00.02  
dovecot-auth
dovecot   9740   0.0 -0.027780500  ??  S 7:26AM   0:00.02  
imap-login
dovecot   9751   0.0 -0.027780500  ??  S 7:27AM   0:00.01  
imap-login
dovecot   9762   0.0 -0.027780500  ??  S 7:27AM   0:00.01  
imap-login


Before posting this bug I compared v1.0.0's /src/lib/ioloop.c to that  
in the nightly snapshot and saw no adjustments in, what I presumed to  
be, the relevant portion of the code.


Output of "dovecot -n"

protocols: imap
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
first_valid_uid: 490
mail_extra_groups: mail
mail_location: mbox:~/Library/Mail/IMAP:INBOX=/var/mail/%u:INDEX=%h/ 
Library/Mail/DOVECOT.indexes

dotlock_use_excl: yes
auth default:
  mechanisms: plain login
  passdb:
driver: pam
  userdb:
driver: passwd

Thank you.

Re: [Dovecot] Bug-report for rc28: Mailbox isn't a valid mbox file

[Timo Sirainen]

On 28.3.2007, at 23.47, Erik van Oosten wrote:

When I try to open the attached mbox file I get the following  
warning in Thunderbird 1.5.0.10:
'The current command failed. The e-mail server responded: Mailbox  
isn't a valid mbox file'.


You have an extra empty line at the beginning of the file. Remove it  
and it should work. Dovecot 0.99.x used to have problems with mboxes  
and it left these extra linefeeds. It shoudn't be a problem anymore  
with 1.0rcs.





PGP.sig
Description: This is a digitally signed message part