[ https://issues.apache.org/jira/browse/TS-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824216#comment-13824216 ]
David Carlin commented on TS-2355: ---------------------------------- I don't see error 1408F10B before the crash like how is reported for squid. Host crashed at 21:44 and the previous instance of 1408F10B is 2.5 hours earlier: {noformat} [Nov 15 19:11:15.871] Server {0x2b5f1a931700} ERROR: SSL::25:error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number:s3_pkt.c:337: {noformat} SSL errors just before the crash: {noformat} [Nov 15 21:44:03.572] Server {0x2b524c807700} ERROR: SSL::27:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:03.955] Server {0x2b524da19700} ERROR: SSL::45:error:14094412:SSL routines:SSL3_READ_BYTES:sslv3 alert bad certificate:s3_pkt.c:1256:SSL alert number 42 [Nov 15 21:44:04.313] Server {0x2b524dc1b700} ERROR: SSL::47:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:08.201] Server {0x2b5244100700} NOTE: Traffic Server is skipping the current log entry for squid.log because its size (87144) exceeds the maximum payload space in a log buffer [Nov 15 21:44:12.798] Server {0x2b5244e0d700} NOTE: Traffic Server is skipping the current log entry for squid.log because its size (16528) exceeds the maximum payload space in a log buffer [Nov 15 21:44:14.101] Server {0x2b524cd0c700} ERROR: SSL::32:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:16.352] Server {0x2b524c504700} ERROR: SSL::24:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:16.600] Server {0x2b524c605700} ERROR: SSL::25:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:25.645] Server {0x2b524c807700} ERROR: SSL::27:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:27.051] Server {0x2b524cc0b700} ERROR: SSL::31:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:29.943] Server {0x2b524ce0d700} ERROR: SSL::33:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:31.863] Server {0x2b524cb0a700} ERROR: SSL::30:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:32.284] Server {0x2b524c605700} ERROR: SSL::25:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:33.168] Server {0x2b524c605700} ERROR: SSL::25:error:140943E8:SSL routines:SSL3_READ_BYTES:reason(1000):s3_pkt.c:1256:SSL alert number 0 [Nov 15 21:44:38.135] {0x2aaca1066640} STATUS: opened /home/y/logs/trafficserver/diags.log {noformat} > ATS 4.0.x crashes when using OpenSSL 1.0.1e > ------------------------------------------- > > Key: TS-2355 > URL: https://issues.apache.org/jira/browse/TS-2355 > Project: Traffic Server > Issue Type: Bug > Components: SSL > Reporter: David Carlin > > I upgraded some 4.0.1 and 4.0.2 hosts from OpenSSL 1.0.0 to 1.0.1e which is > supposed to be ABI compatible. I see this crash about 10 times in a given 24 > hour period. > I'm interested in OpenSSL 1.0.1e as there is a CPU usage improvement in my > tests, and for TLS 1.2 support. > I came across this squid bug with a very similar backtrace. The OpenSSL RT > ticket says > "I have discussed this situation with some Squid developers and we decided - > after SSL error 1408F10B calling standard/raw read() instead of SSL_read() > for empty socket buffer and this patch stopped crash Squid." > http://rt.openssl.org/Ticket/Display.html?id=3128&user=guest&pass=guest > {noformat} > #0 0x0000003f842e7154 in EVP_DigestFinal_ex () from > /usr/lib64/libcrypto.so.10 > #1 0x0000003f84636263 in tls1_final_finish_mac () from > /usr/lib64/libssl.so.10 > #2 0x0000003f8462ad62 in ssl3_do_change_cipher_spec () from > /usr/lib64/libssl.so.10 > #3 0x0000003f8462c7f7 in ssl3_read_bytes () from /usr/lib64/libssl.so.10 > #4 0x0000003f8462d5e2 in ssl3_get_message () from /usr/lib64/libssl.so.10 > #5 0x0000003f8461da1c in ssl3_get_cert_verify () from /usr/lib64/libssl.so.10 > #6 0x0000003f84621e78 in ssl3_accept () from /usr/lib64/libssl.so.10 > #7 0x00000000006711aa in SSLNetVConnection::sslServerHandShakeEvent > (this=0x2aadd0024300, > err=@0x2aacab940c5c) at SSLNetVConnection.cc:488 > #8 0x0000000000672b77 in SSLNetVConnection::sslStartHandShake > (this=0x2aadd0024300, > event=<value optimized out>, err=@0x2aacab940c5c) at > SSLNetVConnection.cc:470 > #9 0x0000000000671dd2 in SSLNetVConnection::net_read_io > (this=0x2aadd0024300, nh= > 0x2aacaa02cbf0, lthread=0x2aacaa029010) at SSLNetVConnection.cc:217 > #10 0x000000000067b8c2 in NetHandler::mainNetEvent (this=0x2aacaa02cbf0, > event=<value optimized out>, e=<value optimized out>) at UnixNet.cc:386 > #11 0x00000000006a335f in handleEvent (this=0x2aacaa029010, e=0x1230a30, > calling_code=5) > at I_Continuation.h:146 > #12 EThread::process_event (this=0x2aacaa029010, e=0x1230a30, calling_code=5) > at UnixEThread.cc:141 > #13 0x00000000006a3d43 in EThread::execute (this=0x2aacaa029010) at > UnixEThread.cc:265 > #14 0x00000000006a21fa in spawn_thread_internal (a=0x143ec30) at Thread.cc:88 > #15 0x00002aaca05b9851 in start_thread () from /lib64/libpthread.so.0 > #16 0x000000324f0e890d in clone () from /lib64/libc.so.6 > {noformat} > {noformat} > NOTE: Traffic Server received Sig 11: Segmentation fault > /home/y/bin/traffic_server - STACK TRACE: > /lib64/libpthread.so.0(+0x324f40f500)[0x2b523d64e500] > /usr/lib64/libcrypto.so.10(EVP_DigestFinal_ex+0x24)[0x3f842e7154] > /usr/lib64/libssl.so.10(tls1_final_finish_mac+0x233)[0x3f84636263] > /usr/lib64/libssl.so.10(ssl3_do_change_cipher_spec+0x72)[0x3f8462ad62] > /usr/lib64/libssl.so.10(ssl3_read_bytes+0xb57)[0x3f8462c7f7] > /usr/lib64/libssl.so.10(ssl3_get_message+0x222)[0x3f8462d5e2] > /usr/lib64/libssl.so.10(ssl3_get_cert_verify+0x6c)[0x3f8461da1c] > /usr/lib64/libssl.so.10(ssl3_accept+0x788)[0x3f84621e78] > /home/y/bin/traffic_server(_ZN17SSLNetVConnection23sslServerHandShakeEventERi+0x2a)[0x6711aa] > /home/y/bin/traffic_server(_ZN17SSLNetVConnection17sslStartHandShakeEiRi+0x37)[0x672b77] > /home/y/bin/traffic_server(_ZN17SSLNetVConnection11net_read_ioEP10NetHandlerP7EThread+0x1f2)[0x671dd2] > /home/y/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x1f2)[0x67b8c2] > /home/y/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x8f)[0x6a335f] > /home/y/bin/traffic_server(_ZN7EThread7executeEv+0x4a3)[0x6a3d43] > /home/y/bin/traffic_server[0x6a21fa] > /lib64/libpthread.so.0(+0x324f407851)[0x2b523d646851] > /lib64/libc.so.6(clone+0x6d)[0x324f0e890d] > {noformat} -- This message was sent by Atlassian JIRA (v6.1#6144)