Aw: Re: 2 panics, 1 segmentation fault in dovecot 2.3.0 (fts)
Thanks Aki! The fix works! Will give you the debug stuff of the other error next week. Regards, Infoomatic > Gesendet: Samstag, 27. Januar 2018 um 20:38 Uhr > Von: "Aki Tuomi" > An: "Dovecot ML" , Infoomatic > Betreff: Re: 2 panics, 1 segmentation fault in dovecot 2.3.0 (fts) > > > > On January 27, 2018 at 4:19 PM Infoomatic wrote: > > > > > > Hi, > > > > While testing our migration to dovecot 2.3.0, we encountered 2 panics, 1 > > segmentation fault while we wanted to rebuild the solr indizes. Problem [1] > > was encountered while rebuilding the indizes, and now I don't know how many > > mails/documents are indexed or where it got stuck. [2] and [3] encountered > > while just playing around, since I did not know of "tokenize" and did not > > find a lot of information about it. > > > > This is a ubuntu 16.04 machine, dovecot 2.3.0 compiled from source ( + the > > patch for the recent zlib problem). "doveadm index" runs on a ~ 4GB > > mdbox-zipped-mailbox for about 3-4 minutes before aborting with the message > > above, beforehand I did a "/opt/dovecot/bin/doveadm fts rescan -A". When I > > repeat "doveadm index" it aborts after about 4 seconds, after a > > "/opt/dovecot/bin/doveadm fts rescan -A" it runs again for 3-4 minutes. > > > > Apache tika logs, but those are probably not related to the problem, > > however, the devs might want investigate the last line: > > WARN Parse exception Expected ';', got "," determining content disposition > > javax.mail.internet.ParseException: Expected ';', got "," > > and > > org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 > > bytes > > > > > > I would be happy if you could have a look at this, please, and of course > > just tell me what further information you need. > > > > Thanks, > > infoomatic > > > > > > [1] /opt/dovecot/bin/doveadm index -u user1 '*' > > doveadm(user1): Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): > > assertion failed: (uni_is_valid_ucs4(chr)) > > doveadm(user1): Error: Raw backtrace: > > /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) [0x7ff27755beea] -> > > /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) [0x7ff27755bf59] -> > > /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff2774ce5e1] -> > > /opt/dovecot/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0) > > [0x7ff277593300] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xa74f9) > > [0x7ff27753d4f9] -> > > /opt/dovecot/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5) > > [0x7ff27753d705] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xcf5c) > > [0x7ff276460f5c] -> > > /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27) > > [0x7ff276460c37] -> > > /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x511) > > [0x7ff27645eb51] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x110a6) > > [0x7ff2764650a6] -> > > /opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) > > [0x7ff27784618e] -> /opt/dovecot/bin/doveadm(+0x2f697) [0x55dd2f912697] -> > > /opt/dovecot/bin/doveadm(+0x29827) [0x55dd2f90c827] -> > > /opt/dovecot/bin/doveadm(+0x2a4bb) [0x55dd2f90d4bb] -> > > /opt/dovecot/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) > > [0x55dd2f90e2cd] -> /opt/dovecot/bin/doveadm(doveadm_cmd_run_ver2+0x575) > > [0x55dd2f91deb5] -> /opt/dovecot/bin/doveadm(doveadm_cmd_try_run_ver2+0x37) > > [0x55dd2f91df07] -> /opt/dovecot/bin/doveadm(main+0x1d2) [0x55dd2f8fd032] > > -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7ff2770ec830] > > -> /opt/dovecot/bin/doveadm(_start+0x29) [0x55dd2f8fd439] > > Aborted (core dumped) > > > > > > You can fix this with > https://github.com/dovecot/core/commit/5c1837529e6957da3e389683c43bd006859395e5.patch > > > [2] This was just about my curiosity since I haven't used "tokenize" and > > did not find too much info about it: > > /opt/dovecot/bin/doveadm fts tokenize -A '*' > > Panic: file doveadm-print.c: line 142: unreached > > Error: Raw backtrace: /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) > > [0x7f38ac5c2eea] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) > > [0x7f38ac5c2f59] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) > > [0x7f38ac5355e1] -> /opt/dovecot/bin/doveadm(+0x3b3d3) [0x55c728e8b
Re: 2 panics, 1 segmentation fault in dovecot 2.3.0 (fts)
> On January 27, 2018 at 4:19 PM Infoomatic wrote: > > > Hi, > > While testing our migration to dovecot 2.3.0, we encountered 2 panics, 1 > segmentation fault while we wanted to rebuild the solr indizes. Problem [1] > was encountered while rebuilding the indizes, and now I don't know how many > mails/documents are indexed or where it got stuck. [2] and [3] encountered > while just playing around, since I did not know of "tokenize" and did not > find a lot of information about it. > > This is a ubuntu 16.04 machine, dovecot 2.3.0 compiled from source ( + the > patch for the recent zlib problem). "doveadm index" runs on a ~ 4GB > mdbox-zipped-mailbox for about 3-4 minutes before aborting with the message > above, beforehand I did a "/opt/dovecot/bin/doveadm fts rescan -A". When I > repeat "doveadm index" it aborts after about 4 seconds, after a > "/opt/dovecot/bin/doveadm fts rescan -A" it runs again for 3-4 minutes. > > Apache tika logs, but those are probably not related to the problem, however, > the devs might want investigate the last line: > WARN Parse exception Expected ';', got "," determining content disposition > javax.mail.internet.ParseException: Expected ';', got "," > and > org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 > bytes > > > I would be happy if you could have a look at this, please, and of course just > tell me what further information you need. > > Thanks, > infoomatic > > > [1] /opt/dovecot/bin/doveadm index -u user1 '*' > doveadm(user1): Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): > assertion failed: (uni_is_valid_ucs4(chr)) > doveadm(user1): Error: Raw backtrace: > /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) [0x7ff27755beea] -> > /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) [0x7ff27755bf59] -> > /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff2774ce5e1] -> > /opt/dovecot/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0) > [0x7ff277593300] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xa74f9) > [0x7ff27753d4f9] -> > /opt/dovecot/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5) > [0x7ff27753d705] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xcf5c) > [0x7ff276460f5c] -> > /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27) > [0x7ff276460c37] -> > /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x511) > [0x7ff27645eb51] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x110a6) > [0x7ff2764650a6] -> > /opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) > [0x7ff27784618e] -> /opt/dovecot/bin/doveadm(+0x2f697) [0x55dd2f912697] -> > /opt/dovecot/bin/doveadm(+0x29827 ) [0x55dd2f90c827] -> /opt/dovecot/bin/doveadm(+0x2a4bb) [0x55dd2f90d4bb] -> /opt/dovecot/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55dd2f90e2cd] -> /opt/dovecot/bin/doveadm(doveadm_cmd_run_ver2+0x575) [0x55dd2f91deb5] -> /opt/dovecot/bin/doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55dd2f91df07] -> /opt/dovecot/bin/doveadm(main+0x1d2) [0x55dd2f8fd032] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7ff2770ec830] -> /opt/dovecot/bin/doveadm(_start+0x29) [0x55dd2f8fd439] > Aborted (core dumped) > > You can fix this with https://github.com/dovecot/core/commit/5c1837529e6957da3e389683c43bd006859395e5.patch > [2] This was just about my curiosity since I haven't used "tokenize" and did > not find too much info about it: > /opt/dovecot/bin/doveadm fts tokenize -A '*' > Panic: file doveadm-print.c: line 142: unreached > Error: Raw backtrace: /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) > [0x7f38ac5c2eea] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) > [0x7f38ac5c2f59] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f38ac5355e1] -> /opt/dovecot/bin/doveadm(+0x3b3d3) [0x55c728e8b3d3] -> > /opt/dovecot/bin/doveadm(+0x2a63a) [0x55c728e7a63a] -> > /opt/dovecot/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) > [0x55c728e7b2cd] -> /opt/dovecot/bin/doveadm(doveadm_cmd_run_ver2+0x575) > [0x55c728e8aeb5] -> /opt/dovecot/bin/doveadm(doveadm_cmd_try_run_ver2+0x37) > [0x55c728e8af07] -> /opt/dovecot/bin/doveadm(main+0x1d2) [0x55c728e6a032] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f38ac153830] -> > /opt/dovecot/bin/doveadm(_start+0x29) [0x55c728e6a439] > Aborted (core dumped) > This one sounds like a bug, we'll look into this. > > [3] /opt/dovecot/bin/doveadm fts tokenize -u user1 "@" > Segmentation fault (core dumped) > Can you run this in gdb with debug symbols and provide bt full? Aki
2 panics, 1 segmentation fault in dovecot 2.3.0 (fts)
Hi, While testing our migration to dovecot 2.3.0, we encountered 2 panics, 1 segmentation fault while we wanted to rebuild the solr indizes. Problem [1] was encountered while rebuilding the indizes, and now I don't know how many mails/documents are indexed or where it got stuck. [2] and [3] encountered while just playing around, since I did not know of "tokenize" and did not find a lot of information about it. This is a ubuntu 16.04 machine, dovecot 2.3.0 compiled from source ( + the patch for the recent zlib problem). "doveadm index" runs on a ~ 4GB mdbox-zipped-mailbox for about 3-4 minutes before aborting with the message above, beforehand I did a "/opt/dovecot/bin/doveadm fts rescan -A". When I repeat "doveadm index" it aborts after about 4 seconds, after a "/opt/dovecot/bin/doveadm fts rescan -A" it runs again for 3-4 minutes. Apache tika logs, but those are probably not related to the problem, however, the devs might want investigate the last line: WARN Parse exception Expected ';', got "," determining content disposition javax.mail.internet.ParseException: Expected ';', got "," and org.apache.tika.exception.ZeroByteFileException: InputStream must have > 0 bytes I would be happy if you could have a look at this, please, and of course just tell me what further information you need. Thanks, infoomatic [1] /opt/dovecot/bin/doveadm index -u user1 '*' doveadm(user1): Panic: file unichar.c: line 160 (uni_ucs4_to_utf8_c): assertion failed: (uni_is_valid_ucs4(chr)) doveadm(user1): Error: Raw backtrace: /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) [0x7ff27755beea] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) [0x7ff27755bf59] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ff2774ce5e1] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(uni_ucs4_to_utf8_c+0xa0) [0x7ff277593300] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xa74f9) [0x7ff27753d4f9] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(mail_html2text_more+0xc5) [0x7ff27753d705] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0xcf5c) [0x7ff276460f5c] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_parser_more+0x27) [0x7ff276460c37] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(fts_build_mail+0x511) [0x7ff27645eb51] -> /opt/dovecot/lib/dovecot/lib20_fts_plugin.so(+0x110a6) [0x7ff2764650a6] -> /opt/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_precache+0x2e) [0x7ff27784618e] -> /opt/dovecot/bin/doveadm(+0x2f697) [0x55dd2f912697] -> /opt/dovecot/bin/doveadm(+0x29827) [0x55dd2f90c827] -> /opt/dovecot/bin/doveadm(+0x2a4bb) [0x55dd2f90d4bb] -> /opt/dovecot/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55dd2f90e2cd] -> /opt/dovecot/bin/doveadm(doveadm_cmd_run_ver2+0x575) [0x55dd2f91deb5] -> /opt/dovecot/bin/doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55dd2f91df07] -> /opt/dovecot/bin/doveadm(main+0x1d2) [0x55dd2f8fd032] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7ff2770ec830] -> /opt/dovecot/bin/doveadm(_start+0x29) [0x55dd2f8fd439] Aborted (core dumped) [2] This was just about my curiosity since I haven't used "tokenize" and did not find too much info about it: /opt/dovecot/bin/doveadm fts tokenize -A '*' Panic: file doveadm-print.c: line 142: unreached Error: Raw backtrace: /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5eea) [0x7f38ac5c2eea] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(+0xc5f59) [0x7f38ac5c2f59] -> /opt/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f38ac5355e1] -> /opt/dovecot/bin/doveadm(+0x3b3d3) [0x55c728e8b3d3] -> /opt/dovecot/bin/doveadm(+0x2a63a) [0x55c728e7a63a] -> /opt/dovecot/bin/doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21d) [0x55c728e7b2cd] -> /opt/dovecot/bin/doveadm(doveadm_cmd_run_ver2+0x575) [0x55c728e8aeb5] -> /opt/dovecot/bin/doveadm(doveadm_cmd_try_run_ver2+0x37) [0x55c728e8af07] -> /opt/dovecot/bin/doveadm(main+0x1d2) [0x55c728e6a032] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7f38ac153830] -> /opt/dovecot/bin/doveadm(_start+0x29) [0x55c728e6a439] Aborted (core dumped) [3] /opt/dovecot/bin/doveadm fts tokenize -u user1 "@" Segmentation fault (core dumped)
