Re: [Dovecot] mailbox count folders issues
On Fri, 20 Nov 2009 13:53:18 -0500 Timo Sirainen t...@iki.fi wrote: On Fri, 2009-11-20 at 10:23 +0200, Nikita Koshikov wrote: Nov 20 10:16:00 IMAP(al...@domain.com): Panic: file mail-transaction-log-append.c: line 31 (log_append_buffer): assertion failed: ((type MAIL_TRANSACTION_) Oh, missed that one: http://hg.dovecot.org/dovecot-1.2/rev/fa8a438c64ce With above patch applied, I got this one crash: Nov 23 09:42:59 IMAP(al...@domain.com): Panic: file mail-index-sync-ext.c: line 625 (mail_index_sync_ext_hdr_update): assertion failed: (ext-hdr_offset + offset + size = map-hdr.header_size) Nov 23 09:42:59 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2521] - imap [0x80d2592] - imap [0x80d1ef9] - imap [0x80c06ef] - imap(mail_index_sync_record+0x210) [0x80af010] - imap(mail_index_sync_map+0x2b9) [0x80afd39] - imap(mail_index_map+0x3c4) [0x80a4f34] - imap(mail_index_sync_commit+0xc2) [0x80ad552] - /usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7deebc6] - /usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c) [0xb7df16ac] - imap(mailbox_sync+0x35) [0x80975a5] - imap(cmd_select_full+0x3d8) [0x8062fe8] - imap(cmd_select+0x19) [0x8063729] - imap [0x80659ac] - imap [0x8065a53] - imap(client_handle_input+0x2d) [0x8065bad] - imap(client_input+0x5f) [0x80664ef] - imap(io_loop_handler_run+0xe0) [0x80da680] - imap(io_loop_run+0x20) [0x80d9b20] - imap(main+0x5dc) [0x806ee7c] - /lib/libc.so.6(__libc_start_main+0xe5) [0xb7e3c725] - imap [0x805e2b1] Nov 23 09:43:00 dovecot: Error: child 19671 (imap) killed with signal 6 (core dumped) Gdb trace in attachment. GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as i686-pc-linux-gnu... warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libpthread.so.0...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.1...done. Loaded symbols for /lib/libz.so.1 Reading symbols from /lib/libbz2.so.1...done. Loaded symbols for /lib/libbz2.so.1 Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done. Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1 Core was generated by `imap'. Program terminated with signal 6, Aborted. [New process 19671] #0 0xe424 in __kernel_vsyscall () (gdb) bt full #0 0xe424 in __kernel_vsyscall () No symbol table info available. #1 0xb7e50660 in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb7e51e98 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080d2535 in default_fatal_finish (type=value optimized out, status=0) at failures.c:160 backtrace = 0xb7d72a48 imap [0x80d2521] - imap [0x80d2592] - imap [0x80d1ef9] - imap [0x80c06ef] - imap(mail_index_sync_record+0x210) [0x80af01. #4 0x080d2592 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80ea1cc file %s: line %d (%s): assertion failed: (%s), args=0xbfd9ac24 4x\017\bq\002) at failures.c:443 No locals. #5 0x080d1ef9 in i_panic (format=0x80ea1cc file %s: line %d (%s): assertion failed: (%s)) at failures.c:207 No locals. #6 0x080c06ef in mail_index_sync_ext_hdr_update (ctx=0xbfd9ad2c, offset=2151153664, size=1, data=0xbb71f5c) at mail-index-sync-ext.c:625 map = (struct mail_index_map *) 0xa9e62b0 ext = (const struct mail_index_ext *) 0xb206000 __PRETTY_FUNCTION__ =
Re: [Dovecot] mailbox count folders issues
On Mon, 2009-11-23 at 09:59 +0200, Nikita Koshikov wrote: With above patch applied, I got this one crash: Nov 23 09:42:59 IMAP(al...@domain.com): Panic: file mail-index-sync-ext.c: line 625 (mail_index_sync_ext_hdr_update): assertion failed: (ext-hdr_offset + offset + size = map-hdr.header_size) Fixed: http://hg.dovecot.org/dovecot-1.2/rev/40a6a70b3146 Also that crash doesn't get fixed by itself, so either delete the virtual mailbox's dovecot.index* files or/and apply this patch: http://hg.dovecot.org/dovecot-1.2/rev/9a8eb4d68b04 And I finally created 1000 mailboxes myself and tested that it actually works now. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Mon, 23 Nov 2009 11:46:35 -0500 Timo Sirainen t...@iki.fi wrote: On Mon, 2009-11-23 at 09:59 +0200, Nikita Koshikov wrote: With above patch applied, I got this one crash: Nov 23 09:42:59 IMAP(al...@domain.com): Panic: file mail-index-sync-ext.c: line 625 (mail_index_sync_ext_hdr_update): assertion failed: (ext-hdr_offset + offset + size = map-hdr.header_size) Fixed: http://hg.dovecot.org/dovecot-1.2/rev/40a6a70b3146 Finally it works!!! Thanks. Also that crash doesn't get fixed by itself, so either delete the virtual mailbox's dovecot.index* files or/and apply this patch: http://hg.dovecot.org/dovecot-1.2/rev/9a8eb4d68b04 My indexes are located in the memory (:INDEX=MEMORY:) so I don't apply this patch. If you need me to test it - please let me know. And I finally created 1000 mailboxes myself and tested that it actually works now. Again, Thanks for your work.
Re: [Dovecot] mailbox count folders issues
On Thu, 19 Nov 2009 18:53:19 -0500
Timo Sirainen t...@iki.fi wrote:
On Thu, 2009-11-19 at 18:31 -0500, Timo Sirainen wrote:
On Thu, 2009-11-19 at 09:10 +0200, Nikita Koshikov wrote:
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite:
size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning:
mail_index_update_header_ext: ext_id=2 offset=0 size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning:
log_append_ext_hdr_update: size=65536
Thanks. Pretty simple fix after all that I missed:
http://hg.dovecot.org/dovecot-1.2/rev/3e1ca490dde0
Except .. although that above patch helps a bit, it still breaks after
header size goes to 64k and fixing that requires changing index file
format a bit. This should help there:
http://hg.dovecot.org/dovecot-1.2/rev/e5d38150be58
Sorry, but with version 1.2.8 the crash is still here:
Nov 20 10:16:00 IMAP(al...@domain.com): Panic: file
mail-transaction-log-append.c: line 31 (log_append_buffer): assertion failed:
((type MAIL_TRANSACTION_)
Nov 20 10:16:00 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2411]
- imap [0x80d2482] - imap [0x80d1de9] - imap [0x80b4a9f] - imap(mail_tran]
Nov 20 10:16:00 dovecot: Error: child 29596 (imap) killed with signal 6 (core
dumped)
I have tried delete old indexes, but this didn't help.
current gdb-trace is attached.GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type show copying
and show warranty for details.
This GDB was configured as i686-pc-linux-gnu...
warning: Can't read pathname for load map: Input/output error.
Reading symbols from /lib/libdl.so.2...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/librt.so.1...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libpthread.so.0...done.
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so
Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done.
Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so
Reading symbols from /lib/libz.so.1...done.
Loaded symbols for /lib/libz.so.1
Reading symbols from /lib/libbz2.so.1...done.
Loaded symbols for /lib/libbz2.so.1
Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done.
Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1
Core was generated by `imap'.
Program terminated with signal 6, Aborted.
[New process 29596]
#0 0xe424 in __kernel_vsyscall ()
(gdb) bt full
#0 0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7e2b660 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0xb7e2ce98 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080d2425 in default_fatal_finish (type=value optimized out, status=0) at failures.c:160
backtrace = 0xb7d5d6c0 imap [0x80d2411] - imap [0x80d2482] - imap [0x80d1de9] - imap [0x80b4a9f] - imap(mail_transaction_log_append+0x68d) [0x80b556d] - imap [0x80aa039] - imap(mail_index_sync_commit+0xa5) [0x80ad535]...
#4 0x080d2482 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80ea0cc file %s: line %d (%s): assertion failed: (%s),
args=0xbff736b4 �U\017\b\037) at failures.c:443
No locals.
#5 0x080d1de9 in i_panic (format=0x80ea0cc file %s: line %d (%s): assertion failed: (%s)) at failures.c:207
No locals.
#6 0x080b4a9f in log_append_buffer (ctx=0xbff737a8, buf=0xb7d4d670, hdr_buf=0x0, type=29596) at mail-transaction-log-append.c:31
hdr = {size = 196419352, type = 32824}
hdr_size = value optimized out
hdr_pos = value optimized out
__PRETTY_FUNCTION__ = log_append_buffer
#7 0x080b556d in mail_transaction_log_append (t=0xb205fb8, log_file_seq_r=0xbff73910, log_file_offset_r=0xbff73900) at mail-transaction-log-append.c:348
index = (struct
Re: [Dovecot] mailbox count folders issues
On Fri, 2009-11-20 at 10:23 +0200, Nikita Koshikov wrote: Nov 20 10:16:00 IMAP(al...@domain.com): Panic: file mail-transaction-log-append.c: line 31 (log_append_buffer): assertion failed: ((type MAIL_TRANSACTION_) Oh, missed that one: http://hg.dovecot.org/dovecot-1.2/rev/fa8a438c64ce signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Thu, 2009-11-19 at 09:10 +0200, Nikita Koshikov wrote: Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite: size=32824 Nov 19 08:57:34 IMAP(al...@domain.com): Warning: mail_index_update_header_ext: ext_id=2 offset=0 size=32824 Nov 19 08:57:34 IMAP(al...@domain.com): Warning: log_append_ext_hdr_update: size=65536 Thanks. Pretty simple fix after all that I missed: http://hg.dovecot.org/dovecot-1.2/rev/3e1ca490dde0 signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Thu, 2009-11-19 at 18:31 -0500, Timo Sirainen wrote: On Thu, 2009-11-19 at 09:10 +0200, Nikita Koshikov wrote: Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite: size=32824 Nov 19 08:57:34 IMAP(al...@domain.com): Warning: mail_index_update_header_ext: ext_id=2 offset=0 size=32824 Nov 19 08:57:34 IMAP(al...@domain.com): Warning: log_append_ext_hdr_update: size=65536 Thanks. Pretty simple fix after all that I missed: http://hg.dovecot.org/dovecot-1.2/rev/3e1ca490dde0 Except .. although that above patch helps a bit, it still breaks after header size goes to 64k and fixing that requires changing index file format a bit. This should help there: http://hg.dovecot.org/dovecot-1.2/rev/e5d38150be58 signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
I guess it's just exponentially increasing the buffer size then until it reaches mail_process_size. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to imap process while it's still running? So something like: - open the account so imap process starts - gdb -p imap process pid - gdb command: c - cause imap to crash - gdb should stop - gdb command: bt full Today, I have updated to version 1.2.7 and I was able to get gdb-backtrace with it. This is clean installation without any patches applied. Error-log: Nov 18 09:55:37 IMAP(al...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Nov 18 09:55:37 IMAP(al...@domain.com): Error: Raw backtrace: imap [0x80d2381] - imap [0x80d23f2] - imap [0x80d1d59] - imap [0x80d0b2d] - imap [0x80d0fa7] - imap [0x80db928] - imap [0x80cfeda] - imap(buffer_write+0xd2) [0x80d03d2] - imap(buffer_append+0x26) [0x80d04f6] - imap(mail_transaction_log_append+0x61c) [0x80b54cc] - imap [0x80aa029] - imap(mail_index_sync_commit+0xa5) [0x80ad525] - /usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7ea8bc6] - /usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c) [0xb7eab6ac] - imap(mailbox_sync+0x35) [0x8097595] - imap(cmd_select_full+0x3d8) [0x8062fb8] - imap(cmd_select+0x19) [0x80636f9] - imap [0x806597c] - imap [0x8065a23] - imap(client_handle_input+0x2d) [0x8065b7d] - imap(client_input+0x5f) [0x80664bf] - imap(io_loop_handler_run+0xe0) [0x80da4e0] - imap(io_loop_run+0x20) [0x80d9980] - imap(main+0x5dc) [0x806ee4c] - /lib/libc.so.6(__libc_start_main+0xe5) [0xb7ef6725] - imap [0x805e2 81] Nov 18 09:55:37 dovecot: Error: child 9173 (imap) killed with signal 6 (core dumped) Gdb-backtrace is attached.r...@mail alice 0:0 # gdb /usr/libexec/dovecot/imap core GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as i686-pc-linux-gnu... warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libpthread.so.0...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.1...done. Loaded symbols for /lib/libz.so.1 Reading symbols from /lib/libbz2.so.1...done. Loaded symbols for /lib/libbz2.so.1 Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...done. Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1 Core was generated by `imap'. Program terminated with signal 6, Aborted. [New process 9173] #0 0xe424 in __kernel_vsyscall () (gdb) bt full #0 0xe424 in __kernel_vsyscall () No symbol table info available. #1 0xb7f0a660 in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb7f0be98 in abort () from /lib/libc.so.6 No symbol table info available. #3 0x080d2395 in default_fatal_finish (type=value optimized out, status=0) at failures.c:160 backtrace = 0x92700478 imap [0x80d2381] - imap [0x80d23f2] - imap [0x80d1d59] - imap [0x80d0b2d] - imap [0x80d0fa7] - imap [0x80db928] - imap [0x80cfeda] - imap(buffer_write+0xd2) [0x80d03d2] - imap(buffer_append+0x... #4 0x080d23f2 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80f9adc data stack: Out of memory when allocating %u bytes, args=0xbf853ef4 \020) at failures.c:443 No locals. #5 0x080d1d59 in i_panic (format=0x80f9adc data stack: Out of memory when
Re: [Dovecot] mailbox count folders issues
On Wed, 18 Nov 2009 06:18:06 -0500 Charles Marcus cmar...@media-brokers.com wrote: On 11/18/2009, Nikita Koshikov (koshi...@gmail.com) wrote: Today, I have updated to version 1.2.7 and I was able to get gdb-backtrace with it. This is clean installation without any patches applied. Probably a good idea to always provide dovecot -n output too... It has been showed in my first post. Since that time config file didn't change.
Re: [Dovecot] mailbox count folders issues
On Wed, 2009-11-18 at 10:04 +0200, Nikita Koshikov wrote:
#10 0x080d03d2 in buffer_write (_buf=0xb7e1a670, pos=67100436,
data=0xbb53ae0, data_size=32824) at buffer.c:63
No locals.
#11 0x080d04f6 in buffer_append (buf=0xb7e1a670, data=0xbb53ae0,
data_size=32824) at buffer.c:168
No locals.
#12 0x080b54cc in mail_transaction_log_append (t=0xb205218,
log_file_seq_r=0xbf8541f0, log_file_offset_r=0xbf8541e0) at
mail-transaction-log-append.c:333
Interesting. If you still have the core and do:
fr 12
p *hdr
p offset
p hdrs[ext_id]
What does it print? Also could you try what it logs with the attached patch?
diff -r 96b5d6d8dd2c src/lib-index/mail-index-transaction.c
--- a/src/lib-index/mail-index-transaction.c Thu Nov 12 12:43:13 2009 -0500
+++ b/src/lib-index/mail-index-transaction.c Wed Nov 18 11:54:45 2009 -0500
@@ -1241,6 +1241,10 @@
i_assert(offset = (uint16_t)-1 size = (uint16_t)-1
offset + size = (uint16_t)-1);
+ if (offset + size 1000)
+ i_warning(mail_index_update_header_ext: ext_id=%u offset=%u size=%u,
+ ext_id, (int)offset, (int)size);
+
if (!array_is_created(t-ext_hdr_updates))
i_array_init(t-ext_hdr_updates, ext_id + 2);
diff -r 96b5d6d8dd2c src/lib-index/mail-transaction-log-append.c
--- a/src/lib-index/mail-transaction-log-append.c Thu Nov 12 12:43:13 2009 -0500
+++ b/src/lib-index/mail-transaction-log-append.c Wed Nov 18 11:54:45 2009 -0500
@@ -319,6 +319,10 @@
data = hdr-data;
mask = hdr-mask;
+ if (hdr-alloc_size 1000)
+ i_warning(log_append_ext_hdr_update: size=%u,
+ (int)hdr-alloc_size);
+
buf = buffer_create_dynamic(pool_datastack_create(), 256);
for (offset = 0; offset = hdr-alloc_size; offset++) {
if (offset hdr-alloc_size mask[offset] != 0) {
diff -r 96b5d6d8dd2c src/plugins/virtual/virtual-sync.c
--- a/src/plugins/virtual/virtual-sync.c Thu Nov 12 12:43:13 2009 -0500
+++ b/src/plugins/virtual/virtual-sync.c Wed Nov 18 11:54:45 2009 -0500
@@ -311,6 +311,7 @@
sizeof(struct virtual_mail_index_record),
sizeof(uint32_t));
}
+ i_warning(header rewrite: size=%u, name_pos);
mail_index_update_header_ext(ctx-trans, ctx-mbox-virtual_ext_id,
0, buf-data, name_pos);
}
@@ -1007,6 +1008,10 @@
mailbox_offset = sizeof(struct virtual_mail_index_header) +
bbox-sync_mailbox_idx * sizeof(mailbox);
+ i_warning(header update: offset=%u size=%u, sync_mailbox_idx=%u,
+ (int)(mailbox_offset + uidval_pos),
+ (int)(sizeof(mailbox) - uidval_pos),
+ bbox-sync_mailbox_idx);
mail_index_update_header_ext(ctx-trans, ctx-mbox-virtual_ext_id,
mailbox_offset + uidval_pos,
CONST_PTR_OFFSET(mailbox, uidval_pos),
signature.asc
Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Wed, 18 Nov 2009 11:55:40 -0500
Timo Sirainen t...@iki.fi wrote:
On Wed, 2009-11-18 at 10:04 +0200, Nikita Koshikov wrote:
#10 0x080d03d2 in buffer_write (_buf=0xb7e1a670, pos=67100436,
data=0xbb53ae0, data_size=32824) at buffer.c:63
No locals.
#11 0x080d04f6 in buffer_append (buf=0xb7e1a670, data=0xbb53ae0,
data_size=32824) at buffer.c:168
No locals.
#12 0x080b54cc in mail_transaction_log_append (t=0xb205218,
log_file_seq_r=0xbf8541f0, log_file_offset_r=0xbf8541e0) at
mail-transaction-log-append.c:333
Interesting. If you still have the core and do:
fr 12
p *hdr
p offset
p hdrs[ext_id]
With old core(not sure if it's exactly the same):
(gdb) fr 12
#12 0x080b54cc in mail_transaction_log_append (t=0xb205218,
log_file_seq_r=0xbf8541f0, log_file_offset_r=0xbf8541e0) at
mail-transaction-log-append.c:333
warning: Source file is more recent than executable.
333 } else {
(gdb) p *hdr
No symbol hdr in current context.
(gdb) p offset
No symbol offset in current context.
(gdb) p hdrs[ext_id]
No symbol hdrs in current context.
What does it print? Also could you try what it logs with the attached patch?
Here is logs, from new crash:
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: header rewrite: size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: mail_index_update_header_ext:
ext_id=2 offset=0 size=32824
Nov 19 08:57:34 IMAP(al...@domain.com): Warning: log_append_ext_hdr_update:
size=65536
Nov 19 08:57:35 IMAP(al...@domain.com): Panic: data stack: Out of memory when
allocating 268435472 bytes
Nov 19 08:57:35 IMAP(al...@agromat.ua): Error: Raw backtrace: imap [0x80d23e1]
- imap [0x80d2452] - imap [0x80d1db9] - imap [0x80d0b8d] - imap [0x80d1007]
- imap [0x80db988] - imap [0x80cff3a] - imap(buffer_write+0xd2) [0x80d0432]
- imap(buffer_append+0x26) [0x80d0556] -
imap(mail_transaction_log_append+0x624) [0x80b5504] - imap [0x80aa059] -
imap(mail_index_sync_commit+0xa5) [0x80ad555] -
/usr/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7e6cc06] -
/usr/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x71c)
[0xb7e6f71c] - imap(mailbox_sync+0x35) [0x8097595] -
imap(cmd_select_full+0x3d8) [0x8062fb8] - imap(cmd_select+0x19) [0x80636f9] -
imap [0x806597c] - imap [0x8065a23] - imap(client_handle_input+0x2d)
[0x8065b7d] - imap(client_input+0x5f) [0x80664bf] -
imap(io_loop_handler_run+0xe0) [0x80da540] - imap(io_loop_run+0x20)
[0x80d99e0] - imap(main+0x5dc) [0x806ee4c] -
/lib/libc.so.6(__libc_start_main+0xe5) [0xb7eba725] - imap [0x805e281]
Nov 19 08:57:36 dovecot: Error: child 16124 (imap) killed with signal 6 (core
dumped)
From new core file:
(gdb) fr 12
#12 0x080b5504 in mail_transaction_log_append (t=0xb205218,
log_file_seq_r=0xbf9162b0, log_file_offset_r=0xbf9162a0) at
mail-transaction-log-append.c:337
337 mail-transaction-log-append.c: No such file or directory.
in mail-transaction-log-append.c
(gdb) p *hdr
No symbol hdr in current context.
(gdb) p offset
No symbol offset in current context.
(gdb) p hdrs[ext_id]
No symbol hdrs in current context.
And attached current gdb-trace.
(gdb) bt full
#0 0xe424 in __kernel_vsyscall ()
No symbol table info available.
#1 0xb7ece660 in raise () from /lib/libc.so.6
No symbol table info available.
#2 0xb7ecfe98 in abort () from /lib/libc.so.6
No symbol table info available.
#3 0x080d23f5 in default_fatal_finish (type=value optimized out, status=0) at failures.c:160
backtrace = 0x92700478 imap [0x80d23e1] - imap [0x80d2452] - imap [0x80d1db9] - imap [0x80d0b8d] - imap [0x80d1007] - imap [0x80db988] - imap [0x80cff3a] - imap(buffer_write+0xd2) [0x80d0432] - imap(buffer_append+0x...
#4 0x080d2452 in i_internal_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=0x80f9b9c data stack: Out of memory when allocating %u bytes,
args=0xbf915fb4 \020) at failures.c:443
No locals.
#5 0x080d1db9 in i_panic (format=0x80f9b9c data stack: Out of memory when allocating %u bytes) at failures.c:207
No locals.
#6 0x080d0b8d in mem_block_alloc (min_size=134217728) at data-stack.c:291
block = (struct stack_block *) 0x0
prev_size = value optimized out
alloc_size = 268435456
#7 0x080d1007 in t_malloc_real (size=value optimized out, permanent=true) at data-stack.c:352
block = (struct stack_block *) 0x6
ret = value optimized out
alloc_size = 134217728
#8 0x080db988 in pool_data_stack_realloc (pool=0xbb4c808, mem=0xb28b4018, old_size=67108857, new_size=134217728) at mempool-datastack.c:127
No locals.
#9 0x080cff3a in buffer_alloc (buf=0xbb4c818, size=134217728) at buffer.c:32
__PRETTY_FUNCTION__ = buffer_alloc
#10 0x080d0432 in buffer_write (_buf=0xbb4c818, pos=67100436, data=0xbb711d0, data_size=32824) at buffer.c:63
No locals.
#11 0x080d0556 in buffer_append (buf=0xbb4c818, data=0xbb711d0, data_size=32824) at buffer.c:168
No locals.
#12 0x080b5504 in mail_transaction_log_append (t=0xb205218,
Re: [Dovecot] mailbox count folders issues
On Thu, 29 Oct 2009 12:38:22 -0400 Timo Sirainen t...@iki.fi wrote: On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote: On Wed, 28 Oct 2009 14:17:52 -0400 Timo Sirainen t...@iki.fi wrote: On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. Sorry, but this patch didn't help. Not with the out of memory, but did it get rid of the double free or corruption error? Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. 268435456 is 256M which is max mail_process_size config setting, maybe this values is in use ? I guess it's just exponentially increasing the buffer size then until it reaches mail_process_size. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to imap process while it's still running? So something like: - open the account so imap process starts - gdb -p imap process pid - gdb command: c - cause imap to crash - gdb should stop - gdb command: bt full I found I way, to switch crash under telnet session and attach gdb to process, but this didn't produce valid trace: GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as i686-pc-linux-gnu. Attaching to process 29952 Reading symbols from /usr/libexec/dovecot/imap...(no debugging symbols found)...done. Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libpthread.so.0... (no debugging symbols found)...done. [Thread debugging using libthread_db enabled] [New Thread 0xb7dccac0 (LWP 29952)] Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so... (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.1 Reading symbols from /lib/libbz2.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libbz2.so.1 (no debugging symbols found) 0xe424 in __kernel_vsyscall () (gdb) cont Continuing. (no debugging symbols found) Program received signal SIGABRT, Aborted. [Switching to Thread 0xb7dccac0 (LWP 29952)] 0xe424 in __kernel_vsyscall () (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) yes Detaching from program: /usr/libexec/dovecot/imap, process 29952
Re: [Dovecot] mailbox count folders issues
On Thu, 29 Oct 2009 12:38:22 -0400 Timo Sirainen t...@iki.fi wrote: On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote: On Wed, 28 Oct 2009 14:17:52 -0400 Timo Sirainen t...@iki.fi wrote: On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. Sorry, but this patch didn't help. Not with the out of memory, but did it get rid of the double free or corruption error? Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. 268435456 is 256M which is max mail_process_size config setting, maybe this values is in use ? I guess it's just exponentially increasing the buffer size then until it reaches mail_process_size. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to imap process while it's still running? So something like: - open the account so imap process starts - gdb -p imap process pid - gdb command: c - cause imap to crash - gdb should stop - gdb command: bt full One more note: version 1.2.4 also has this bug.
Re: [Dovecot] mailbox count folders issues
On Thu, 29 Oct 2009 12:38:22 -0400 Timo Sirainen t...@iki.fi wrote: On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote: On Wed, 28 Oct 2009 14:17:52 -0400 Timo Sirainen t...@iki.fi wrote: On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. Sorry, but this patch didn't help. Not with the out of memory, but did it get rid of the double free or corruption error? The valgrind error is the same. I'll attach current valgrid output. Also, I have noticed, that valgrind writes vgcore.* files in user's homedir, I attach gdb backtrace from that file. Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. 268435456 is 256M which is max mail_process_size config setting, maybe this values is in use ? I guess it's just exponentially increasing the buffer size then until it reaches mail_process_size. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to imap process while it's still running? So something like: - open the account so imap process starts - gdb -p imap process pid - gdb command: c - cause imap to crash - gdb should stop - gdb command: bt full The problem, that webmail don't keep permanent connection, so when I login - there are about 2-3 imap-login\disconnected log entries. And when I try to get directory listing the new process created for this operation and then crushes. Can gdb work in non-interactive mode ? so I can write wrapper to handle debug output ? Also I enable imap_debug feature on webmail client, as you say - it stop working after client trying to access All virtual folder. The appropriate imap-session log part is also attached. Note, I can't reproduce panic with Thunderbird, this happens in roundcube only. All virtual folder config is: cat /var/mail/virtual/.all/dovecot-virtual #All messages in all folders * all vcore-gdb.trace Description: Binary data dovecot-error.valgrind Description: Binary data =cut== [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / 1 [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Drafts [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasChildren) / Dealers [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Dealers/Kapriz [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Dealers/1 Inbox [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Dealers/2 Outbox [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Spam [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Trash [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / INBOX [30-Oct-2009 10:30:11 +0200]: S: * LIST (\Noselect \HasChildren) / Company [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Company/all [30-Oct-2009 10:30:11 +0200]: S: * LIST (\HasNoChildren) / Company/favorites [30-Oct-2009 10:30:11 +0200]: S: lmb OK List completed. [30-Oct-2009 10:30:12 +0200]: C: lsb LSUB * [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Sent [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Drafts [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Spam [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / 1 [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / INBOX [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Business correspondence/Inbox [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Business correspondence/Outbox [30-Oct-2009 10:30:12 +0200]: S: * LSUB () / Trash [30-Oct-2009 10:30:12 +0200]: S: lsb OK Lsub completed. [30-Oct-2009 10:30:12 +0200]: C: sel1 SELECT INBOX [30-Oct-2009 10:30:12 +0200]: S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk) [30-Oct-2009 10:30:12 +0200]: S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted. [30-Oct-2009 10:30:12 +0200]: S: * 37 EXISTS [30-Oct-2009 10:30:12 +0200]: S: * 0 RECENT [30-Oct-2009 10:30:12 +0200]: S: * OK [UIDVALIDITY 1245314909] UIDs valid [30-Oct-2009 10:30:12 +0200]: S: * OK [UIDNEXT 39] Predicted next UID [30-Oct-2009 10:30:12 +0200]: S: * OK [HIGHESTMODSEQ 1] Highest [30-Oct-2009 10:30:12 +0200]: S: sel1 OK [READ-WRITE] Select completed. [30-Oct-2009 10:30:12 +0200]: C: sel1 SELECT Drafts [30-Oct-2009 10:30:12 +0200]: S: * OK [CLOSED] Previous mailbox closed. [30-Oct-2009 10:30:12 +0200]: S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) [30-Oct-2009 10:30:12 +0200]: S: * OK
Re: [Dovecot] mailbox count folders issues
On Wed, 28 Oct 2009 14:17:52 -0400 Timo Sirainen t...@iki.fi wrote: On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. Sorry, but this patch didn't help. It's anyway related to one of your virtual maiboxes. I suppose you have a virtual mailbox that includes messages from all real mailboxes? Yes, I have two virtual mailboxes: all - contains all messages favorites - contains flagged letters But this user didn't subscribe to any of them. Backtrace shows that it crashes while trying to refresh a virtual mailbox. Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. 268435456 is 256M which is max mail_process_size config setting, maybe this values is in use ? Can you make sure debugging symbols aren't stripped from imap binary (i.e. file /usr/.../dovecot/imap says not stripped, this is the default when compiling Dovecot from sources) and get gdb backtrace? http://dovecot.org/bugreport.html Then I could see exactly where it's crashing and figuring out this would be a lot easier. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. r...@mail dovecot-1.2.6 0:0 # file /usr/libexec/dovecot/imap /usr/libexec/dovecot/imap: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped r...@mail dovecot-1.2.6 0:130 # gdb /usr/libexec/dovecot/imap /data/mail/domain.com/gozhda.a/core GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type show copying and show warranty for details. This GDB was configured as i686-pc-linux-gnu... (no debugging symbols found) warning: Can't read pathname for load map: Input/output error. Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /usr/lib/dovecot/imap/lib10_quota_plugin.so... (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_expire_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_expire_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_virtual_plugin.so... (no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_virtual_plugin.so Reading symbols from /usr/lib/dovecot/imap/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/imap/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.1 Reading symbols from /lib/libbz2.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libbz2.so.1 Reading symbols from /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/gcc/i686-pc-linux-gnu/4.3.4/libgcc_s.so.1 (no debugging symbols found) Core was generated by `imap'. Program terminated with signal 6, Aborted. [New process 29774] #0 0xe424 in __kernel_vsyscall () (gdb) bt full #0 0xe424 in __kernel_vsyscall () No symbol table info available. #1 0xb7e6e660 in raise () from /lib/libc.so.6 No symbol table info available. #2 0xb7e6fe98
Re: [Dovecot] mailbox count folders issues
On Thu, 2009-10-29 at 12:11 +0200, Nikita Koshikov wrote: On Wed, 28 Oct 2009 14:17:52 -0400 Timo Sirainen t...@iki.fi wrote: On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. Sorry, but this patch didn't help. Not with the out of memory, but did it get rid of the double free or corruption error? Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. 268435456 is 256M which is max mail_process_size config setting, maybe this values is in use ? I guess it's just exponentially increasing the buffer size then until it reaches mail_process_size. I recompile dovecot binaries with debug symbols, but seemed that gdb backtrace is broken. Yeah, it is. Hmm. Since the core isn't working, can you attach gdb to imap process while it's still running? So something like: - open the account so imap process starts - gdb -p imap process pid - gdb command: c - cause imap to crash - gdb should stop - gdb command: bt full signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Tue, 27 Oct 2009 17:50:22 -0400
Timo Sirainen t...@iki.fi wrote:
On Tue, 2009-10-27 at 09:48 +0200, Nikita Koshikov wrote:
I have one user, which have a lot of folders (not it is 414 without INBOX).
When he tries to create one more folder with random name - the folder
successfully created, but LIST command invokes out of memory to imap
process.
From the logs:
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory
when allocating 268435472 bytes
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected ***
imap: double free or corruption (!prev): 0x0812ba00 ***
This is bad. Can you reproduce it easily? Could you cause this crash
under valgrind? For example:
protocol imap {
mail_executable = /usr/local/bin/imap.sh
}
and /usr/local/bin/imap.sh:
#!/bin/sh
if [ $USER = gozhd...@domain.com ]; then
exec /usr/bin/valgrind /usr/libexec/dovecot/imap
else
exec /usr/libexec/dovecot/imap
fi
Valgrind will hopefully then log to Dovecot's log what exactly is the
problem.
In attachement full valgrind output for user's session.
Steps I perform:
1)Login with admin master password to user's mailbox(from webmail)
2)create new directory
3)try to get directories list
It's anyway related to one of your virtual maiboxes. I suppose you have
a virtual mailbox that includes messages from all real mailboxes?
Yes, I have two virtual mailboxes:
all - contains all messages
favorites - contains flagged letters
But this user didn't subscribe to any of them.Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Memcheck, a memory error detector.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Using LibVEX rev 1884, a library for dynamic binary translation.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640== For more details, rerun with: -v
Oct 28 10:04:31 IMAP(gozhd...@domain.ua): Error: ==24640==
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 59 from 2)
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== malloc/free: in use at exit: 129 bytes in 3 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== malloc/free: 165 allocs, 162 frees, 105,524 bytes allocated.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== For counts of detected errors, rerun with: -v
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== searching for pointers to 3 not-freed blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== checked 80,788 bytes.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== LEAK SUMMARY:
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==definitely lost: 41 bytes in 1 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== possibly lost: 0 bytes in 0 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640==still reachable: 88 bytes in 2 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== suppressed: 0 bytes in 0 blocks.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24640== Rerun with --leak-check=full to see details of leaked memory.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Memcheck, a memory error detector.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Using LibVEX rev 1884, a library for dynamic binary translation.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644== For more details, rerun with: -v
Oct 28 10:04:32 IMAP(gozhd...@domain.ua): Error: ==24644==
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644==
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 59 from 2)
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644== malloc/free: in use at exit: 4,225 bytes in 4 blocks.
Oct 28 10:04:35 IMAP(gozhd...@domain.ua): Error: ==24644== malloc/free: 412
Re: [Dovecot] mailbox count folders issues
On Wed, 2009-10-28 at 10:39 +0200, Nikita Koshikov wrote: Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory when allocating 268435472 bytes Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected *** imap: double free or corruption (!prev): 0x0812ba00 *** Oh, the double free is because of broken handling of out-of-memory error. http://hg.dovecot.org/dovecot-1.2/rev/acfef2f0fec3 probably fixes that. It's anyway related to one of your virtual maiboxes. I suppose you have a virtual mailbox that includes messages from all real mailboxes? Yes, I have two virtual mailboxes: all - contains all messages favorites - contains flagged letters But this user didn't subscribe to any of them. Backtrace shows that it crashes while trying to refresh a virtual mailbox. Hmm. So I guess there's no memory corruption causing this, but I don't really see why it would try to allocate that much memory. 268435472 in hex is 0x1010, which is an interesting number but doesn't really help much either. Can you make sure debugging symbols aren't stripped from imap binary (i.e. file /usr/.../dovecot/imap says not stripped, this is the default when compiling Dovecot from sources) and get gdb backtrace? http://dovecot.org/bugreport.html Then I could see exactly where it's crashing and figuring out this would be a lot easier. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] mailbox count folders issues
On Tue, 2009-10-27 at 09:48 +0200, Nikita Koshikov wrote:
I have one user, which have a lot of folders (not it is 414 without INBOX).
When he tries to create one more folder with random name - the folder
successfully created, but LIST command invokes out of memory to imap process.
From the logs:
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Panic: data stack: Out of memory
when allocating 268435472 bytes
Oct 26 15:26:38 IMAP(gozhd...@domain.com): Error: *** glibc detected ***
imap: double free or corruption (!prev): 0x0812ba00 ***
This is bad. Can you reproduce it easily? Could you cause this crash
under valgrind? For example:
protocol imap {
mail_executable = /usr/local/bin/imap.sh
}
and /usr/local/bin/imap.sh:
#!/bin/sh
if [ $USER = gozhd...@domain.com ]; then
exec /usr/bin/valgrind /usr/libexec/dovecot/imap
else
exec /usr/libexec/dovecot/imap
fi
Valgrind will hopefully then log to Dovecot's log what exactly is the
problem.
It's anyway related to one of your virtual maiboxes. I suppose you have
a virtual mailbox that includes messages from all real mailboxes?
signature.asc
Description: This is a digitally signed message part
