Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 23:01 schrieb Ken Murchison: > I haven't used it in a long time but you could try tools/masssievec in > the Cyrus distro As mentioned that script fails for some users as well, maybe even for the same reason than lmtpd at runtime. How could I debug these scripts? If I cd into the directory and run (ingo is the filter app from horde groupware): sievec ingo.script ingo.bc that works out fine. I run with disabled sievedir now and try to fix the few users with problematic scripts. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-03 um 09:40 schrieb Stefan G. Weichinger via Info-cyrus: > I run with disabled sievedir now and try to fix the few users with > problematic scripts. I looked through them and rm-ed them all ;) (bofh) They were outdated and inactive anyway. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Upgraded cyrus 2.5.6: lmtpd segfaults
gentoo server here, yesterday I upgraded cyrus-imapd from 2.4.x to 2.5.6 Alongside I reinstalled glibc, cyrus-sasl, postfix ... and corrected all the new config options etc in cyrus.conf etc I now see problems with lmtpd. # cyrus.conf lmtpunixcmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0 # /etc/postfix/main.cf mailbox_transport = lmtp:unix:/var/imap/socket/lmtp # ls -l /var/imap/socket/lmtp srwxrwxrwx 1 root root 0 Nov 2 13:04 /var/imap/socket/lmtp I get segfaults for lmtpd: Nov 02 14:01:59 postler.lichtfels.com kernel: lmtpd[17148]: segfault at 173e450 ip 7f09acc141b5 sp 7ffc379dab30 error 4 in libc-2.21.so[7f09acb9e000+18b000] Nov 02 14:01:59 postler.lichtfels.com master[8091]: process type:SERVICE name:lmtpunix path:/usr/lib64/cyrus/lmtpd age:0.324s pid:17148 signaled to death by signal 11 (Segmentation fault) # dmesg [12630549.982443] lmtpd[17655]: segfault at 778ba0 ip 7f5f3a1091b5 sp 7ffe0dda3470 error 4 in libc-2.21.so[7f5f3a093000+18b000] [12630550.204617] lmtpd[17671]: segfault at 250f6c0 ip 7fd6bdc6b1b5 sp 7fffd49d63a0 error 4 in libc-2.21.so[7fd6bdbf5000+18b000] [12630550.834193] lmtpd[17678]: segfault at 12056b0 ip 7f86e19461b5 sp 7ffe7f076260 error 4 in libc-2.21.so[7f86e18d+18b000] [12630552.372314] show_signal_msg: 5 callbacks suppressed [12630552.372321] lmtpd[17748]: segfault at 214f050 ip 7f65586331b5 sp 7fffbe826030 error 4 in libc-2.21.so[7f65585bd000+18b000] [12630552.666540] lmtpd[17743]: segfault at 143f150 ip 7f095d5ea1b5 sp 7fff5436c9c0 error 4 in libc-2.21.so[7f095d574000+18b000] [12630552.739760] lmtpd[17769]: segfault at 1e16560 ip 7fc9abdeb1b5 sp 7fffa0481150 error 4 in libc-2.21.so[7fc9abd75000+18b000] could anyone give me a pointer here? I can't flush mails from the postfix queue, but *some* mail gets through. maybe this is also relevant: Nov 02 14:06:12 postler.lichtfels.com master[5188]: unable to setsocketopt(IP_TOS) service lmtpunix/unix: Operation not supported Nov 02 14:06:12 postler.lichtfels.com master[5188]: unable to setsocketopt(IP_TOS) service notify/unix: Operation not supported stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 14:06 schrieb Stefan G. Weichinger via Info-cyrus: > > gentoo server here, > > yesterday I upgraded cyrus-imapd from 2.4.x to 2.5.6 > Alongside I reinstalled glibc, cyrus-sasl, postfix ... and corrected all > the new config options etc in cyrus.conf etc > > I now see problems with lmtpd. I tried multiple things over the last few hours, even upgraded the kernel and rebuilt all(?) the relevant packages. I still have over 100 emails in the queue which don't get delivered to cyrus, and I still get the segfaults. Could someone pls advise how to proceed and maybe debug this issue? thanks in advance, Stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 21:26 schrieb Ken Murchison: > Can you get a backtrace from a lmtpd core dump? if you tell me what to do ;) I am not that experienced in doing that, sorry. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 20:06 schrieb Dave McMurtrie via Info-cyrus: > On Mon, 2015-11-02 at 19:11 +0100, Stefan G. Weichinger via Info-cyrus > wrote: >> Am 2015-11-02 um 14:06 schrieb Stefan G. Weichinger via Info-cyrus: >>> >>> gentoo server here, >>> >>> yesterday I upgraded cyrus-imapd from 2.4.x to 2.5.6 >>> Alongside I reinstalled glibc, cyrus-sasl, postfix ... and corrected all >>> the new config options etc in cyrus.conf etc >>> >>> I now see problems with lmtpd. >> >> I tried multiple things over the last few hours, even upgraded the >> kernel and rebuilt all(?) the relevant packages. >> >> I still have over 100 emails in the queue which don't get delivered to >> cyrus, and I still get the segfaults. >> >> Could someone pls advise how to proceed and maybe debug this issue? > > We recently upgraded our MX servers to what will eventually be 3.x and > had a series of issues with lmtpproxyd (which is actually a link to > lmtpd). I'm not sure how much of the code is common with what's in the > 2.5.x releases, but Ken (cc'd on this) managed to track down and fix our > sigsegv issues. Ken, do you know if the things you fixed are present in > the 2.5.x code as well? I am thankful for any help on this as I have to get out these mails to my customers (right now it's 9pm here, but I should have that fixed tmrw morning or so). Maybe I could downgrade to 2.4 as well? I hesitate to do so as I don't want to make things even worse by panicking. Stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 21:34 schrieb Stefan G. Weichinger via Info-cyrus: > Am 2015-11-02 um 21:26 schrieb Ken Murchison: > >> Can you get a backtrace from a lmtpd core dump? > > if you tell me what to do ;) > I am not that experienced in doing that, sorry. I installed gdb now for a start and rebuild cyrus-imapd with gdb enabled following: https://wiki.gentoo.org/wiki/Bugzilla/Guide#Debugging_using_GDB Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 22:47 schrieb Ken Murchison: > lmtpd is crashing inside of the sieve code. It looks like its trying to > compile a regular expression that appears in the recipient's sieve script. > > I don't know if this is a bug, a bad sieve script, or if your version of > Cyrus wasn't compiled with support for the particular regular expression. > > If you want to get email delivered, you can simply disable the user's > sieve script. You could also try recompiling the script with sievec I will try this with one of my own accounts asap. As there are >140 emails piled up right now, is there any quick trick to skip sieve for all the users? maybe a specific "cyrus-lmtpd.conf" for the lmtpd socket? thanks! Stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 22:53 schrieb Ken Murchison: > You could try changing the sievedir option in imapd.conf to something > other than where your sieve scripts currently reside. more than 10 hrs of fiddling ... and now mails are getting delivered. thank you thank you thank you! mailq empty now. cool ;) Now the question is how and if to re-enable the sieve scripts. Can I run something over the whole sievedir to convert/check/rebuild stuff? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 21:45 schrieb Ken Murchison: > On 11/02/2015 03:34 PM, Stefan G. Weichinger wrote: >> Am 2015-11-02 um 21:26 schrieb Ken Murchison: >> >>> Can you get a backtrace from a lmtpd core dump? >> if you tell me what to do ;) >> I am not that experienced in doing that, sorry. > > Is lmtpd dumping core anywhere? You can look in the directory that you > started master from. can't find any core dump, sorry. Is this the way to go -> # systemctl stop cyrus-imapd.service # gdb /usr/lib/cyrus/master (gdb) run Starting program: /usr/lib64/cyrus/master [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". ( it keeps this way now ) thanks, Stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 22:28 schrieb Ken Murchison: > You shouldn't have to run master in gdb. > > Do something like this: > > cd /tmp > ulimit -c unlimited > /usr/lib64/cyrus/master & > > Hopefully any lmtpd cores will end up in /tmp they do! how to interpret them? They are ~2megs in size so I don't want to attach them here. I have >10 of them already. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 22:38 schrieb Ken Murchison: > gdb /usr/local/cyrus/lmtpd /tmo/core.XXX (use proper locations) > > At the (gdb) prompt run the backtrace command (bt) > > It should give you info that you can post here. tmp # gdb /usr/lib64/cyrus/lmtpd core.32682 Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `lmtpd'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7ff3212963ab in ?? () from /lib64/libc.so.6 (gdb) bt #0 0x7ff3212963ab in ?? () from /lib64/libc.so.6 #1 0x7ff3212977ec in ?? () from /lib64/libc.so.6 #2 0x7ff321299b2e in malloc () from /lib64/libc.so.6 #3 0x7ff3212ed1d6 in ?? () from /lib64/libc.so.6 #4 0x7ff3212ee5c9 in regcomp () from /lib64/libc.so.6 #5 0x7ff3225fd4d5 in bc_compile_regex (s=0x7ff32291b1a8 , ctag=97, ctag@entry=33, errmsg=errmsg@entry=0x7ffe30e97d30 "L\t", errsiz=errsiz@entry=100) at sieve/bc_eval.c:131 #6 0x7ff3225fe5eb in eval_bc_test (interp=interp@entry=0x1e5ced0, m=m@entry=0x7ffe30e9be60, bc=bc@entry=0x7ff32291b000, ip=ip@entry=0x7ffe30e97eac, workingflags=, version=version@entry=5) at sieve/bc_eval.c:819 #7 0x7ff3225ffbc5 in sieve_eval_bc (exe=exe@entry=0x1e75910, is_incl=is_incl@entry=0, i=i@entry=0x1e5ced0, sc=sc@entry=0x7ffe30e9a120, m=m@entry=0x7ffe30e9be60, flagvars=flagvars@entry=0x7ffe30e99060, actions=actions@entry=0x1e78310, notify_list=notify_list@entry=0x1e782d0, errmsg=errmsg@entry=0x7ffe30e99030, workingvars=workingvars@entry=0x7ffe30e99080) at sieve/bc_eval.c:1584 #8 0x7ff322605368 in sieve_execute_bytecode (exe=0x1e75910, interp=interp@entry=0x1e5ced0, script_context=script_context@entry=0x7ffe30e9a120, message_context=message_context@entry=0x7ffe30e9be60) at sieve/script.c:877 #9 0x00410af1 in run_sieve (user=0x1e63cd0 "spa3mei", domain=0x0, mailbox=0x0, interp=0x1e5ced0, msgdata=msgdata@entry=0x7ffe30e9be60) at imap/lmtp_sieve.c:904 #10 0x0040916d in deliver (msgdata=0x1e61eb0, authuser=0x0, authstate=0x0) at imap/lmtpd.c:903 #11 0x0040bab9 in lmtpmode (func=func@entry=0x616b40 , pin=0x1e5fc90, pout=, fd=fd@entry=0) at imap/lmtpengine.c:1270 #12 0x004075d1 in service_main (argc=1, argv=0x1e59030, envp=envp@entry=0x7ffe30e9f578) at imap/lmtpd.c:341 #13 0x004066df in main (argc=, argv=, envp=0x7ffe30e9f578) at master/service.c:622 (gdb) Does that tell you anything? thanks, Stefan Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 23:01 schrieb Ken Murchison: > I haven't used it in a long time but you could try tools/masssievec in > the Cyrus distro digging ... aside from that: should that one be filed as a possible bug somewhere? Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Upgraded cyrus 2.5.6: lmtpd segfaults
Am 2015-11-02 um 23:19 schrieb Stefan G. Weichinger via Info-cyrus: > Am 2015-11-02 um 23:01 schrieb Ken Murchison: > >> I haven't used it in a long time but you could try tools/masssievec in >> the Cyrus distro > > digging ... btw: masssievec fails as well: processing user lky1lkuy processing user lnb2ifon Unable to open sgw.script for reading got error compiling sgw.script. *** Error in `/usr/lib64/cyrus/sievec': free(): invalid next size (fast): 0x01bd5f90 *** === Backtrace: = /lib64/libc.so.6(+0x71b23)[0x7f0ea76a2b23] /lib64/libc.so.6(+0x771f5)[0x7f0ea76a81f5] /lib64/libc.so.6(+0x779d3)[0x7f0ea76a89d3] /usr/lib64/libcyrus_sieve.so.0(+0x12a90)[0x7f0ea7e31a90] /usr/lib64/libcyrus_sieve.so.0(+0x12b08)[0x7f0ea7e31b08] /usr/lib64/libcyrus_sieve.so.0(+0x1674b)[0x7f0ea7e3574b] /usr/lib64/libcyrus_sieve.so.0(sieve_script_parse+0xd8)[0x7f0ea7e2dec8] /usr/lib64/cyrus/sievec[0x40150e] /lib64/libc.so.6(__libc_start_main+0xf0)[0x7f0ea76517b0] /usr/lib64/cyrus/sievec[0x401709] === Memory map: 0040-00402000 r-xp 09:01 364071 /usr/lib64/cyrus/sievec 00601000-00602000 r--p 1000 09:01 364071 /usr/lib64/cyrus/sievec 00602000-00603000 rw-p 2000 09:01 364071 /usr/lib64/cyrus/sievec 01bce000-01bef000 rw-p 00:00 0 [heap] 7f0ea51be000-7f0ea51d4000 r-xp 09:01 2021598 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f0ea51d4000-7f0ea53d3000 ---p 00016000 09:01 2021598 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f0ea53d3000-7f0ea53d4000 r--p 00015000 09:01 2021598 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f0ea53d4000-7f0ea53d5000 rw-p 00016000 09:01 2021598 /usr/lib64/gcc/x86_64-pc-linux-gnu/4.9.3/libgcc_s.so.1 7f0ea53d5000-7f0ea5441000 r-xp 09:01 1677955 /lib64/libpcre.so.1.2.4 7f0ea5441000-7f0ea5641000 ---p 0006c000 09:01 1677955 /lib64/libpcre.so.1.2.4 7f0ea5641000-7f0ea5642000 r--p 0006c000 09:01 1677955 /lib64/libpcre.so.1.2.4 7f0ea5642000-7f0ea5643000 rw-p 0006d000 09:01 1677955 /lib64/libpcre.so.1.2.4 7f0ea5643000-7f0ea5645000 r-xp 09:01 1678286 /lib64/libdl-2.21.so 7f0ea5645000-7f0ea5845000 ---p 2000 09:01 1678286 /lib64/libdl-2.21.so 7f0ea5845000-7f0ea5846000 r--p 2000 09:01 1678286 /lib64/libdl-2.21.so 7f0ea5846000-7f0ea5847000 rw-p 3000 09:01 1678286 /lib64/libdl-2.21.so 7f0ea5847000-7f0ea58b r-xp 09:01 2004402 /usr/lib64/libssl.so.1.0.0 7f0ea58b-7f0ea5aaf000 ---p 00069000 09:01 2004402 /usr/lib64/libssl.so.1.0.0 7f0ea5aaf000-7f0ea5ab4000 r--p 00068000 09:01 2004402 /usr/lib64/libssl.so.1.0.0 7f0ea5ab4000-7f0ea5abb000 rw-p 0006d000 09:01 2004402 /usr/lib64/libssl.so.1.0.0 7f0ea5abb000-7f0ea5d38000 r-xp 09:01 2004084 /usr/lib64/libmysqlclient.so.18.1.0 7f0ea5d38000-7f0ea5f38000 ---p 0027d000 09:01 2004084 /usr/lib64/libmysqlclient.so.18.1.0 7f0ea5f38000-7f0ea5f3b000 r--p 0027d000 09:01 2004084 /usr/lib64/libmysqlclient.so.18.1.0 7f0ea5f3b000-7f0ea6008000 rw-p 0028 09:01 2004084 /usr/lib64/libmysqlclient.so.18.1.0 7f0ea6008000-7f0ea600d000 rw-p 00:00 0 7f0ea600d000-7f0ea6186000 r-xp 09:01 2004369 /usr/lib64/libdb-4.8.so 7f0ea6186000-7f0ea6385000 ---p 00179000 09:01 2004369 /usr/lib64/libdb-4.8.so 7f0ea6385000-7f0ea6388000 r--p 00178000 09:01 2004369 /usr/lib64/libdb-4.8.so 7f0ea6388000-7f0ea638b000 rw-p 0017b000 09:01 2004369 /usr/lib64/libdb-4.8.so 7f0ea638b000-7f0ea63a7000 r-xp 09:01 2004649 /usr/lib64/libsasl2.so.3.0.0 7f0ea63a7000-7f0ea65a6000 ---p 0001c000 09:01 2004649 /usr/lib64/libsasl2.so.3.0.0 7f0ea65a6000-7f0ea65a7000 r--p 0001b000 09:01 2004649 /usr/lib64/libsasl2.so.3.0.0 7f0ea65a7000-7f0ea65a8000 rw-p 0001c000 09:01 2004649 /usr/lib64/libsasl2.so.3.0.0 7f0ea65a8000-7f0ea65bd000 r-xp 09:01 1677933 /lib64/libz.so.1.2.8 7f0ea65bd000-7f0ea67bc000 ---p 00015000 09:01 1677933 /lib64/libz.so.1.2.8 7f0ea67bc000-7f0ea67bd000 r--p 00014000 09:01 1677933 /lib64/libz.so.1.2.8 7f0ea67bd000-7f0ea67be000 rw-p 00015000 09:01 1677933 /lib64/libz.so.1.2.8 7f0ea67be000-7f0ea68be000 r-xp 09:01 1678281 /lib64/libm-2.21.so 7f0ea68be000-7f0ea6abd000 ---p 0010 09:01 1678281 /lib64/libm-2.21.so 7f0ea6abd000-7f0ea6abe000 r--p 000ff000 09:01 1678281 /lib64/libm-2.21.so 7f0ea6abe000-7f0ea6abf000 rw-p 0010 09:01 1678281 /lib64/libm-2.21.so 7f0ea6abf000-7f0ea6ac1000 r-xp 09:01 2003446 /usr/lib64/libpcreposix.so.0.0.3 7f0ea6ac1000-7f0ea6cc ---p 2000 09:01 2003446 /usr/lib64/libpcreposix.so.0.0.3 7f0ea6cc-7f0ea6cc1000 r--p 1000 09:01 2003446 /usr/lib64/libpcreposix.so.0.0.3 7f0ea6cc1000-7f0ea6cc2000 rw-p 2000 09:01 2003446 /usr/lib64/libpcreposix.so.0.0.3 7f0ea6cc2000-7f0ea6ecc000 r-xp 09:01 2004071 /usr/lib64/libcrypto.so.1.0.0 7f0ea6ecc000-7f0ea70cb000 ---p 0020a000 09:01 2004071 /usr/lib64/libcrypto.so.1.0.0 7f0ea70cb000-7f0ea70e7000 r--p 00209000 09:01 2004071 /usr/lib64/libcrypto.so.1.0.0 7f0ea70e7000-7f