[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 Mark Thomas changed: What|Removed |Added Resolution|--- |FIXED Status|NEEDINFO|RESOLVED --- Comment #32 from Mark Thomas --- No response for several months so I am going to assume the issue is resolved. Most of the errors in the attached logs relate to HTTP/2 but not all. I can't tell if what is shown is a single error that affects HTTP/2 and HTTP/1.1 are two separate issues. I did note an HTTP/2 issue was fixed in 8.5.42 related to multiple threads accessing a single stream - similar scenarios have triggered APR/native crashes in the past. Looking at the changelog, the refactoring to use a single pollset in 8.5.50 looks like a possible change that coudl have fixed these issues. If you still see this issue or something similar please: - update to the latest 8.5.x or 9.0.x release - update to the latest Tomcat Native release - retest - if you still see the issue, feel free to re-open this bug What we really need are the steps to reproduce it. Anything that narrows down the trigger is helpful but a set of steps to reproduce is ideal. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 Michael Osipov changed: What|Removed |Added Status|NEW |NEEDINFO -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #31 from Michael Osipov --- Does this still occur with the most recent versions of Tomcat, libtcnative and libapr? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 Conrad T. Pino changed: What|Removed |Added Status|NEEDINFO|NEW --- Comment #30 from Conrad T. Pino --- (In reply to Christopher Schultz from comment #27) > (In reply to Conrad T. Pino from comment #25) > > Uploading gigabytes is pointless unless a purpose is served so I'll start by > > asking if uploads are wanted. > > Uploading the crash log is useful. Uploading the memory and/or code dumps is > not really useful. Feel free to upload the crash logs. ZIP file with 8 plain text log files attached. Thank you. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #29 from Conrad T. Pino --- Comment on attachment 36730 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36730 ZIP file - contents 8 plain text log files Directory of hs_err_logs.zip 05/18/2018 15:2038,952 hs_err_pid1964.log 11/07/2018 19:0138,793 hs_err_pid2260.log 12/07/2018 18:2437,242 hs_err_pid2440.log 01/13/2019 19:1337,617 hs_err_pid3040.log 06/29/2018 08:3138,864 hs_err_pid3676.log 02/12/2019 22:5638,322 hs_err_pid4224.log 02/07/2019 11:1837,193 hs_err_pid4776.log 01/17/2019 20:3438,874 hs_err_pid6772.log 8 File(s)305,857 bytes 0 Dir(s) 34,551,578,624 bytes free -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #28 from Conrad T. Pino --- Created attachment 36730 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36730=edit ZIP file - contents 8 plain text log files Directory of hs_err_logs.zip 05/18/2018 15:2038,952 hs_err_pid1964.log 11/07/2018 19:0138,793 hs_err_pid2260.log 12/07/2018 18:2437,242 hs_err_pid2440.log 01/13/2019 19:1337,617 hs_err_pid3040.log 06/29/2018 08:3138,864 hs_err_pid3676.log 02/12/2019 22:5638,322 hs_err_pid4224.log 02/07/2019 11:1837,193 hs_err_pid4776.log 01/17/2019 20:3438,874 hs_err_pid6772.log 8 File(s)305,857 bytes 0 Dir(s) 34,551,578,624 bytes free -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #27 from Christopher Schultz --- (In reply to Conrad T. Pino from comment #25) > Uploading gigabytes is pointless unless a purpose is served so I'll start by > asking if uploads are wanted. Uploading the crash log is useful. Uploading the memory and/or code dumps is not really useful. Feel free to upload the crash logs. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #26 from Mark Thomas --- There are some HTTP/2 concurrency bugs fixed in later versions that may explain the crashes you have seen. Testing with the latest stable 9.0.x and/or 8.5.x releases would be helpful. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 Conrad Pino changed: What|Removed |Added CC||con...@pino.com -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #25 from Conrad Pino --- Across two (2) Tomcat releases: apache-tomcat-8.5.27 apache-tomcat-8.5.31 I have accumulated eight (8) tcnative-1.dll crash logs with memory dumps: 05/18/2018 02:20 PM38,952 hs_err_pid1964.log 05/18/2018 02:20 PM 5,265,420,258 hs_err_pid1964.mdmp 11/07/2018 06:01 PM38,793 hs_err_pid2260.log 11/07/2018 06:01 PM 4,974,077,542 hs_err_pid2260.mdmp 12/07/2018 05:24 PM37,242 hs_err_pid2440.log 12/07/2018 05:24 PM 7,737,765,554 hs_err_pid2440.mdmp 01/13/2019 06:13 PM37,617 hs_err_pid3040.log 01/13/2019 06:13 PM 4,264,832,306 hs_err_pid3040.mdmp 06/29/2018 07:31 AM38,864 hs_err_pid3676.log 06/29/2018 07:31 AM 7,986,923,546 hs_err_pid3676.mdmp 02/12/2019 09:56 PM38,322 hs_err_pid4224.log 02/12/2019 09:56 PM 4,964,538,742 hs_err_pid4224.mdmp 02/07/2019 10:18 AM37,193 hs_err_pid4776.log 02/07/2019 10:18 AM 3,177,549,366 hs_err_pid4776.mdmp 01/17/2019 07:34 PM38,874 hs_err_pid6772.log 01/17/2019 07:34 PM 7,579,831,326 hs_err_pid6772.mdmp 16 File(s) 45,951,244,497 bytes Also available as single ZIP file: 02/14/2019 11:14 PM 5,241,734,239 tcnative-1.zip All are consistent: EXCEPTION_ACCESS_VIOLATION (0xc005) Uploading gigabytes is pointless unless a purpose is served so I'll start by asking if uploads are wanted. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #24 from Christopher Schultz --- (In reply to jan.pfeifer from comment #23) > It has been 6 months now, was there any progress with bug? Should I re-test > with newer versions of Tomcat? There is currently a vote being taken on a new version of tcnative (1.2.21) which includes a fix for *a* memory leak. I'd be interested if it is the memory leak you described several months ago. Once the vote passes (it looks like it will pass) and the version is released, try upgrading to that version and run in your NIO2 + HTTP/2 with the updated tcnative. If things work out, we'll call it a win. If not... I guess we'll have to keep investigating. I'm sorry, I'm out of my element when it comes to debugging windows binaries. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #23 from jan.pfei...@centrum.cz --- It has been 6 months now, was there any progress with bug? Should I re-test with newer versions of Tomcat? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 ithan changed: What|Removed |Added CC|jan.pfei...@centrum.cz |ithanr...@gmail.com -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #22 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #21) > (In reply to jan.pfeifer from comment #20) > > Calling NIO2+HTTP/2 stable was quite premature. With this config I get out > > of memory (5G) every few hours. Strange, peak traffic isnt causing any > > obvious problem. > > Just to confirm: > > * NIO2 + HTTP/2 + Java 10 crashes. Same setup with Java 8 crashes. Not actual crash. But freezes with out of memory error. (5Gigs available. Under normal circumstances under 1G is enough) > * Limiting to HTTP/1.1 restores stability Yes. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #21 from Christopher Schultz --- (In reply to jan.pfeifer from comment #20) > Calling NIO2+HTTP/2 stable was quite premature. With this config I get out > of memory (5G) every few hours. Strange, peak traffic isnt causing any > obvious problem. Just to confirm: * NIO2 + HTTP/2 + Java 10 crashes. Same setup with Java 8 crashes. * Limiting to HTTP/1.1 restores stability Correct? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #20 from jan.pfei...@centrum.cz --- Calling NIO2+HTTP/2 stable was quite premature. With this config I get out of memory (5G) every few hours. Strange, peak traffic isnt causing any obvious problem. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #19 from jan.pfei...@centrum.cz --- It doesn't take long to crash. I can confirm RECYCLE_FACADES did not help. Currently I am running stable config NIO2 + HTTP/2 + Java 10. Let me know if you will need any further assistance. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #18 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #16) Its like observing black hole for me but i've came to some conslusions > > But why is it not crashing on Java 8? 1. I have tried org.apache.catalina.connector.RECYCLE_FACADES=true parameter before and left it enabled. But I am quite sure it crashed at least once with parameter enabled. I've disabled it again. 2. HTTP/2. As I mentioned before I didnt know it is possible to have it working with Java 8. So I run it only with HTTP11. I have enabled HTTP/2 on Java 8 yesterday and got three crashes till now. 3. AND/OR "malevolent" traffic. There was some strange access logs for the images. Now its is clear it is related to HTTP/2. As it ceased with HTTP1 and reappeared again now with HTTP/2. It seems some bots have problems with HTTP/2, requesting the same images over and over again and terminating connection in some "bad" way. Last crash log (Java 8 + HTTP/2 RECYCLE_FACADES=false) --- T H R E A D --- Current thread (0x24278800): JavaThread "https-openssl-apr-443-exec-39" daemon [_thread_in_native, id=52448, stack(0x2d85,0x2d95)] siginfo: ExceptionCode=0xc005, writing address 0x01a0 Registers: RAX=0x, RBX=0x2c5a69a0, RCX=0x0001801ccca0, RDX=0xfff0 RSP=0x2d94e190, RBP=0x0009, RSI=0x0017, RDI=0x R8 =0x0006, R9 =0x004b, R10=0x0007, R11=0x R12=0x2c4971c6, R13=0x299b21c0, R14=0x, R15=0x RIP=0x0001800e0a8f, EFLAGS=0x00010286 Top of Stack: (sp=0x2d94e190) 0x2d94e190: 2c5a69a0 0008 0x2d94e1a0: 2c5a69a0 299b21c0 0x2d94e1b0: 0x2d94e1c0: 71f2 0x2d94e1d0: 00170009 0009 0x2d94e1e0: 20c47220 7132cb5d 0x2d94e1f0: 2240cd00 713b0b22 0x2d94e200: 24278800 20c47220 0x2d94e210: 22df4640 0x2d94e220: 2181135f529d 0001800e0075 0x2d94e230: 2d94e370 20c47220 0x2d94e240: 2d94e318 0009 0x2d94e250: 20c47220 0009 0x2d94e260: 2c5a69a0 0001800e5c2e 0x2d94e270: 0x2d94e280: 1fa3da90 24278800 Instructions: (pc=0x0001800e0a8f) 0x0001800e0a6f: f6 41 70 08 75 08 48 8b cb e8 d3 64 00 00 42 8d 0x0001800e0a7f: 04 37 e9 de f7 ff ff 33 ff 48 8b 83 80 00 00 00 0x0001800e0a8f: 44 89 b0 a0 01 00 00 8b c7 e9 c7 f7 ff ff ba 9e 0x0001800e0a9f: 00 00 00 4c 8d 0d a7 c7 0e 00 b9 14 00 00 00 44 Register to memory mapping: RAX=0x is an unknown value RBX=0x2c5a69a0 is an unknown value RCX=0x0001801ccca0 is an unknown value RDX=0xfff0 is an unknown value RSP=0x2d94e190 is pointing into the stack for thread: 0x24278800 RBP=0x0009 is an unknown value RSI=0x0017 is an unknown value RDI=0x is an unknown value R8 =0x0006 is an unknown value R9 =0x004b is an unknown value R10=0x0007 is an unknown value R11=0x is an unknown value R12=0x2c4971c6 is an unknown value R13=0x299b21c0 is an unknown value R14=0x is an unknown value R15=0x is an unknown value Stack: [0x2d85,0x2d95], sp=0x2d94e190, free space=1016k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [tcnative-1.dll+0xe0a8f] C [tcnative-1.dll+0xe5c2e] C [tcnative-1.dll+0x104bc] C [tcnative-1.dll+0x56ff] C 0x02cbc5a5 Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J 7660 org.apache.tomcat.jni.Socket.sendb(JLjava/nio/ByteBuffer;II)I (0 bytes) @ 0x02cbc51f [0x02cbc4c0+0x5f] J 12795 C2 org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doWrite(ZLjava/nio/ByteBuffer;)V (242 bytes) @ 0x043caef0 [0x043cac80+0x270] J 15975 C2 org.apache.coyote.http2.Http2UpgradeHandler.writeBody(Lorg/apache/coyote/http2/Stream;Ljava/nio/ByteBuffer;IZ)V (218 bytes) @ 0x03c10ba0 [0x03c10680+0x520] J 12532 C2 java.io.BufferedOutputStream.write([BII)V (67 bytes) @ 0x04301c3c [0x04300940+0x12fc] J 18229 C2 com.m2000.shop.controllers.DefaultController.image(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/http/HttpServletRequest;)V (1129 bytes) @ 0x0535b604
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #17 from Konstantin Kolinko --- Please add the following line to your catalina.properties file: org.apache.catalina.connector.RECYCLE_FACADES=true as documented at https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics#Troubleshooting_unexpected_Response_state_problems -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #16 from Christopher Schultz --- (In reply to jan.pfeifer from comment #13) > Two last Thread reports: > > --- T H R E A D --- > > Current thread (0x2f30d800): JavaThread > "https-openssl-apr-443-exec-69" daemon [_thread_in_native, id=36360, > stack(0x3f5e,0x3f6e)] > > Stack: [0x3f5e,0x3f6e], sp=0x3f6ddd80, > free space=1015k > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native > code) > C [tcnative-1.dll+0xe0a8f] > No more information about the native frame. That must be a direct-native call... > Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) > J 10218 org.apache.tomcat.jni.Socket.sendb(JLjava/nio/ByteBuffer;II)I (0 > bytes) @ 0x14d3c59f [0x14d3c540+0x005f] > J 15904 c2 org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking()V (90 > bytes) @ 0x1545cbcc [0x1545c880+0x034c] > J 16170 c2 org.apache.tomcat.util.net.SocketWrapperBase.flush(Z)Z (20 bytes) > @ 0x14860a78 [0x14860a40+0x0038] > J 15983 c2 org.apache.catalina.connector.CoyoteOutputStream.write([BII)V (26 > bytes) @ 0x154a8f2c [0x154a51c0+0x3d6c] > J 16090 c2 java.io.BufferedOutputStream.write([BII)V java.base@10.0.2 (67 > bytes) @ 0x154e5660 [0x154e5560+0x0100] > J 18370 c2 > com.m2000.shop.controllers.DefaultController.image(Ljava/lang/String;Ljava/ > lang/String;Ljava/lang/String;Ljavax/servlet/http/HttpServletResponse;Ljavax/ > servlet/http/HttpServletRequest;)V (1129 bytes) @ 0x159f12f4 > [0x159ec920+0x49d4] Okay, this is good information: 1. The crash is occurring in jni.Socket.sendbb (where many crashes have historically been found) 2. Your code is intentionally invoking a write() on the socket/stream That means that we can take a look at your code to see if there is any misuse of the streams. That is the #1 reason for crashes like these: the application misuses a stream (e.g. after it should have been recycled) and it's in an intermediate state. tcnative should not crash, but we may be able to mitigate it by fixing your code. Can you post as much of your code as possible? Feel free to do so privately if you don't want your code published in BZ. > J 19092 c2 > jdk.internal.reflect.GeneratedMethodAccessor159.invoke(Ljava/lang/Object; > [Ljava/lang/Object;)Ljava/lang/Object; (98 bytes) @ 0x15ba9be8 > [0x15ba9b00+0x00e8] > J 19001 c2 > org.springframework.web.method.support.InvocableHandlerMethod. > invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest; > Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/ > Object;)Ljava/lang/Object; (148 bytes) @ 0x15b26018 > [0x15b25ca0+0x0378] Coming through Spring. > J 18530 c2 > org.springframework.web.servlet.mvc.method.annotation. > ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/ > context/request/ServletWebRequest;Lorg/springframework/web/method/support/ > ModelAndViewContainer;[Ljava/lang/Object;)V (142 bytes) @ 0x15a6e884 > [0x15a6e840+0x0044] > J 18568 c2 > org.springframework.web.servlet.mvc.method.annotation. > RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/ > HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/ > springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ > ModelAndView; (328 bytes) @ 0x15a92a48 > [0x15a90d60+0x1ce8] > J 19592 c2 > org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/ > http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (594 > bytes) @ 0x15ca8c20 [0x15ca8140+0x0ae0] > J 18099 c2 > org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/ > http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (360 > bytes) @ 0x15948490 [0x15947ee0+0x05b0] > J 18074 c2 > org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/ > servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V > (298 bytes) @ 0x15919044 [0x159171c0+0x1e84] > J 19591 c2 > javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/ > HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (269 bytes) @ > 0x15ca6d70 [0x15ca6bc0+0x01b0] >From normal Tomcat dispatch. Great. At least there is no odd threading or anything going on, here. That makes it less likely to be a problem in your code. > siginfo: EXCEPTION_ACCESS_VIOLATION (0xc005), writing address > 0x01a0 Interesting. Most of the crashes in the past have been when reading data, not writing it. Therefore, most of
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #15 from Christopher Schultz --- (In reply to jan.pfeifer from comment #13) > (In reply to Christopher Schultz from comment #12) > > > Just to confirm: this is Java 8 with APR+OpenSSL, not NIO+OpenSSL, correct? > > Yes, actual configuration APR+OpenSSL Java 8. Three days without crash. That's good to hear. It's surprising, but at least you have a configuration you can fall-back to if you don't want crashes. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #14 from jan.pfei...@centrum.cz --- If logs did not help, can you point me where to get that DEBUG build of tcnative? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #13 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #12) > Just to confirm: this is Java 8 with APR+OpenSSL, not NIO+OpenSSL, correct? Yes, actual configuration APR+OpenSSL Java 8. Three days without crash. > The Java stack traces are less important than the native stack trace. There > was only a single item in the native stack trace you already posted. Can you > post the part of the native report that is labelled "-- T H R E A D > ---"? > > That should include a lot more relevant information. Two last Thread reports: --- T H R E A D --- Current thread (0x2f30d800): JavaThread "https-openssl-apr-443-exec-69" daemon [_thread_in_native, id=36360, stack(0x3f5e,0x3f6e)] Stack: [0x3f5e,0x3f6e], sp=0x3f6ddd80, free space=1015k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [tcnative-1.dll+0xe0a8f] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J 10218 org.apache.tomcat.jni.Socket.sendb(JLjava/nio/ByteBuffer;II)I (0 bytes) @ 0x14d3c59f [0x14d3c540+0x005f] J 15904 c2 org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking()V (90 bytes) @ 0x1545cbcc [0x1545c880+0x034c] J 16170 c2 org.apache.tomcat.util.net.SocketWrapperBase.flush(Z)Z (20 bytes) @ 0x14860a78 [0x14860a40+0x0038] J 15983 c2 org.apache.catalina.connector.CoyoteOutputStream.write([BII)V (26 bytes) @ 0x154a8f2c [0x154a51c0+0x3d6c] J 16090 c2 java.io.BufferedOutputStream.write([BII)V java.base@10.0.2 (67 bytes) @ 0x154e5660 [0x154e5560+0x0100] J 18370 c2 com.m2000.shop.controllers.DefaultController.image(Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljavax/servlet/http/HttpServletResponse;Ljavax/servlet/http/HttpServletRequest;)V (1129 bytes) @ 0x159f12f4 [0x159ec920+0x49d4] J 19092 c2 jdk.internal.reflect.GeneratedMethodAccessor159.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (98 bytes) @ 0x15ba9be8 [0x15ba9b00+0x00e8] J 19001 c2 org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(Lorg/springframework/web/context/request/NativeWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)Ljava/lang/Object; (148 bytes) @ 0x15b26018 [0x15b25ca0+0x0378] J 18530 c2 org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(Lorg/springframework/web/context/request/ServletWebRequest;Lorg/springframework/web/method/support/ModelAndViewContainer;[Ljava/lang/Object;)V (142 bytes) @ 0x15a6e884 [0x15a6e840+0x0044] J 18568 c2 org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;Lorg/springframework/web/method/HandlerMethod;)Lorg/springframework/web/servlet/ModelAndView; (328 bytes) @ 0x15a92a48 [0x15a90d60+0x1ce8] J 19592 c2 org.springframework.web.servlet.DispatcherServlet.doDispatch(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (594 bytes) @ 0x15ca8c20 [0x15ca8140+0x0ae0] J 18099 c2 org.springframework.web.servlet.DispatcherServlet.doService(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (360 bytes) @ 0x15948490 [0x15947ee0+0x05b0] J 18074 c2 org.springframework.web.servlet.FrameworkServlet.processRequest(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (298 bytes) @ 0x15919044 [0x159171c0+0x1e84] J 19591 c2 javax.servlet.http.HttpServlet.service(Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V (269 bytes) @ 0x15ca6d70 [0x15ca6bc0+0x01b0] J 14832 c2 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x152cec10 [0x152ce2e0+0x0930] J 17775 c2 org.apache.tomcat.websocket.server.WsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (139 bytes) @ 0x1580d614 [0x1580d5c0+0x0054] J 14832 c2 org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (388 bytes) @ 0x152ce6d8 [0x152ce2e0+0x03f8] J 18045 c2 com.m2000.shop.filters.ThreadLocalFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (26
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #12 from Christopher Schultz --- (In reply to jan.pfeifer from comment #11) > (In reply to Christopher Schultz from comment #10) > > Have a look at this page: > > https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues > > If I undestand correctly there is only problem when using ImageIO.write with > response output stream. I do not use it that way. I write with ImageIO to > filesystem. Then, when image is ready, I read it again and write it to > response stream via Buffered IN/Out stream. Hmm, yes, this was in cases where ImageIO was being used to write to the servlet output stream. So it looks like that isn't the problem. > > Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the > > class shown in that wiki page and re-running. If the crashes stop, we can go > > ahead and blame them on ImageIO, but really tcnative should not crash in > > this scenario... I'd expect that you'd just get lots of IOExceptions > > instead. So tcnative isn't doing its job under this circumstance. > > I left if for weekend with Java 8 to see what happens. Yes there are many > IOExceptions already. No crash so far. Just to confirm: this is Java 8 with APR+OpenSSL, not NIO+OpenSSL, correct? If that's the case, there might be some kind of incompatibility between tcnative and Java 9. That would be very surprising to me, but that seems to be where the data are leading us. > > Unfortunately, I don't have win32 debugging environment available to me, so > > I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to > > investigate. It's obviously somewhere in the tcnative implementation of > > SSLSocket.handshake (or something it calls). That function has ~100 lines of > > code in it, but I'm guessing it's crashing towards the beginning, probably > > on a read. > > > > Can you use a DEBUG build of tcnative to get a better crash report? I'm used > > to seeing more of a stack trace in native crash dumps. > > I will try to get DEBUG build on monday. But there is a lot of other debug > info already. As I posted earlier i can attach it. Tomcat generated very > long bug report from which i posted the top part and sysinfo part, and > several gigs big mdmp file. Do you want to see them or we stick to DEBUG > build of tcnative plan? The Java stack traces are less important than the native stack trace. There was only a single item in the native stack trace you already posted. Can you post the part of the native report that is labelled "-- T H R E A D ---"? That should include a lot more relevant information. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #11 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #10) > Have a look at this page: > https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues If I undestand correctly there is only problem when using ImageIO.write with response output stream. I do not use it that way. I write with ImageIO to filesystem. Then, when image is ready, I read it again and write it to response stream via Buffered IN/Out stream. > Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the > class shown in that wiki page and re-running. If the crashes stop, we can go > ahead and blame them on ImageIO, but really tcnative should not crash in > this scenario... I'd expect that you'd just get lots of IOExceptions > instead. So tcnative isn't doing its job under this circumstance. I left if for weekend with Java 8 to see what happens. Yes there are many IOExceptions already. No crash so far. > Unfortunately, I don't have win32 debugging environment available to me, so > I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to > investigate. It's obviously somewhere in the tcnative implementation of > SSLSocket.handshake (or something it calls). That function has ~100 lines of > code in it, but I'm guessing it's crashing towards the beginning, probably > on a read. > > Can you use a DEBUG build of tcnative to get a better crash report? I'm used > to seeing more of a stack trace in native crash dumps. I will try to get DEBUG build on monday. But there is a lot of other debug info already. As I posted earlier i can attach it. Tomcat generated very long bug report from which i posted the top part and sysinfo part, and several gigs big mdmp file. Do you want to see them or we stick to DEBUG build of tcnative plan? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #10 from Christopher Schultz --- Have a look at this page: https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the class shown in that wiki page and re-running. If the crashes stop, we can go ahead and blame them on ImageIO, but really tcnative should not crash in this scenario... I'd expect that you'd just get lots of IOExceptions instead. So tcnative isn't doing its job under this circumstance. (n.b.: http/2 does not require Java 9.) Unfortunately, I don't have win32 debugging environment available to me, so I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to investigate. It's obviously somewhere in the tcnative implementation of SSLSocket.handshake (or something it calls). That function has ~100 lines of code in it, but I'm guessing it's crashing towards the beginning, probably on a read. Can you use a DEBUG build of tcnative to get a better crash report? I'm used to seeing more of a stack trace in native crash dumps. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #9 from jan.pfei...@centrum.cz --- (In reply to Mark Thomas from comment #8) > You mentioned imageio. I'm not sure at what point you stooped using it but > there have been consistent reports over the years of imageio holding on to > references to I/O streams and using them long after they should have been > finished with. This has been the cause of APR crashes in the past. e.g.: > https://markmail.org/message/xr3c5dixxzetpbez Yes, you are right, that was the reason. But there were several chrashes after the change. First was from https://bitbucket.org/luciad/webp-imageio, the second is from https://developers.google.com/speed/webp/download (img2webp.exe). -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #8 from Mark Thomas --- You mentioned imageio. I'm not sure at what point you stooped using it but there have been consistent reports over the years of imageio holding on to references to I/O streams and using them long after they should have been finished with. This has been the cause of APR crashes in the past. e.g.: https://markmail.org/message/xr3c5dixxzetpbez -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #7 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #6) > That's kind of par for the course, isn't it? Any IO operation can fail for > any reason. If you don't catch it and handle it, Tomcat will (eventually) > log it. What else did you expect to happen? I was only surprised by number of different IO exceptions. For some (Stream is not writable) it is not very clear what do they mean. > A "client abort" exception happens when the client makes a request and then > hangs-up the phone before you complete the response. It's fairly common and > I wouldn't really expect tcnative to crash under that circumstance, but it's > certainly possible. > > In general, ClientAbortExceptions can safely be completely ignored, so don't > worry too much about the fact that they are happening. On a busy site with > large responses (e.g. images) I'd expect lots of them. I undestand that. As I stated above it was only mix of different IOException types and quantity what confused me. > Can you try one more thing for me? Downgrade to Java 1.8.0_whatever and > revert your configuration to use APR+tcnative again and let us know if the > crashes (actual segfaults that kill the JVM) continue. I managed the downgrade. Alas lots of traffic ceased at the moment and server seems stable. I removed HTTP/2 upgrade as I understand it needs Java 9. Or am I wrong? > But let's take your ClientAbortException stuff onto the user list to see if > we can't find maybe some more efficient ways to get your images generated. > That synchronized block looks suspicious to me, and creating new threads all > the time is likely to lead to instability. In first version i wasnt using threads at all. It seemed unlikely to get several concurrent request for one image. If it would happened the later one could simply overwrite image or respond with error. But since the crashes started i suspected thread concurency to be a problem. New threads are only spawned when images is requested for the first time. Images is downloaded from different machine, resized and webp version is created. WebP is now created with an external program. I was using native lib and imageio before. I use threads and ConcurentHashMap only to ensure that incoming requests will wait until image processing is finished. This only applies for requests for the same image. Others are free to get cached image or to start new resizing thread. Other option would be to allow only one image process at time, but it would have negative performance impact i guess. Another strange thing i got into was out of memory error (4GB) yesterday evening. No obvious reason of the problem. I have deleted image cache, so they are recreated now. Everything runs well. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #6 from Christopher Schultz --- (In reply to jan.pfeifer from comment #5) > With NIO2 + OpenSSL no crash so far. No problems detected. Except it comes > with new set of client abort IO exceptions: > > "The specified network name is no longer available" and "An existing > connection was forcibly closed by the remote host". This is actually what I was hoping to see happen. tcnative is ... the bare minimum code to get things working; it doesn't have a huge amount of robustness especially when it comes to all the weird possibilities when a network connection is involved. The fact that NIO is telling you that things are in a bad state means that it's actually true: the bug in tcnative that causes this crash is merely bad error-handling and not just tcnative not being able to push bytes around. That bad error-handling can probably be fixed with enough analysis and maybe some trial-and-error testing on your end (if you are willing to be a guinea pig). But the fact remains: something "bad" is really happening with your clients and/or your network and that is the true source of the problem. > Are there any generic way to detect it? I was forced to enclose > stream.write() to its own try/catch and swallow any IOException it produces. That's kind of par for the course, isn't it? Any IO operation can fail for any reason. If you don't catch it and handle it, Tomcat will (eventually) log it. What else did you expect to happen? > To our problem: I cant see any other way how to find culprit on my side. > Some kind of stress test would probably trigger it, but with the same result > we have now. I guess it is still related to image serving part as it is only > "complex" part of webapp. Crawlers and "image thieves" spams it a lot. There > can be let say 50 request per second, sometimes for same image, sometimes > all ends with some kind of "client abort exception". A "client abort" exception happens when the client makes a request and then hangs-up the phone before you complete the response. It's fairly common and I wouldn't really expect tcnative to crash under that circumstance, but it's certainly possible. In general, ClientAbortExceptions can safely be completely ignored, so don't worry too much about the fact that they are happening. On a busy site with large responses (e.g. images) I'd expect lots of them. Let's keep this BZ issue open and continue to talk about the Java + native stack traces you have and try to get this resolved. ERROR_ACCESS_VIOLATION is the same as a segfault (basically null-pointer exception and/or use-after-free). In this case, it looks like dereferencing a pointer which probably isn't a pointer (its value is way too low to be valid). Can you try one more thing for me? Downgrade to Java 1.8.0_whatever and revert your configuration to use APR+tcnative again and let us know if the crashes (actual segfaults that kill the JVM) continue. But let's take your ClientAbortException stuff onto the user list to see if we can't find maybe some more efficient ways to get your images generated. That synchronized block looks suspicious to me, and creating new threads all the time is likely to lead to instability. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #5 from jan.pfei...@centrum.cz --- With NIO2 + OpenSSL no crash so far. No problems detected. Except it cames with new set of client abort IO exceptions: "The specified network name is no longer available" and "An existing connection was forcibly closed by the remote host". Are there any generic way to detect it? I was forced to enclose stream.write() to its own try/catch and swallow any IOException it produces. To our problem: I cant see any other way how to find culprit on my side. Some kind of stress test would propably trigger it, but with the same result we have now. I guess it is still related to image serving part as it is only "complex" part of webapp. Crawlers and "image thieves" spams it a lot. There can be let say 50 request per second, sometimes for same image, sometimes all ends with some kind of "client abort exception". -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626
--- Comment #4 from jan.pfei...@centrum.cz ---
(In reply to Christopher Schultz from comment #3)
Current configuration is for HTTP/2 so APR+openSSL
I am using Spring framework, so only "direct write" is when i manualy read
images and write them to the response output stream. There is also synchronized
block where current thread checks if requested image is being resized. if so
then it waits on task to finish. Actual code:
//tasks is ConcurentHashMap
try {
String syncKey = target.getAbsolutePath();
Thread task;
synchronized (this) {
task = tasks.get(syncKey);
if (task == null && !target.exists()) {
//debug("START", request);
ResizeTask resizeTask = new ResizeTask(source, target,
size);
resizeTask.setDebug(isDebug);
resizeTask.setLink(getFullURL(request));
task = new Thread(resizeTask);
tasks.put(syncKey, task);
task.start();
}
}
if(task != null) task.join();
tasks.remove(syncKey);
//debug("FINISH "+(System.currentTimeMillis() - time),
request);
} catch (InterruptedException e) {
e.printStackTrace();
}
But it seems lower level problem. To answer your question, yes i can try NIO(2)
jsse/openssl.
--
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #3 from Christopher Schultz --- (In reply to jan.pfeifer from comment #2) > It happens several times a day or even several times a hour. At least it's reproducible, even if you can't guarantee a trigger. > After latest crash i have found this (only relevant log output before crash): > > Caused by: java.io.IOException: Unexpected error [120,001] writing data to > the APR/native socket [975,220,944] with wrapper > [org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper@1ce3e909:975220944]. That error code [120001] is: APR_UTIL_START_STATUS (10) + APR_OS_START_ERROR ( 2) +(os error code) ( 1) The OS error code is, unhelpfully, "Operation not permitted", at least on *NIX. I'm not sure how that maps to Windows, which usually has error codes with negative values. Are you maybe trying to write-back to a socket which should be closed? Is this NIO+OpenSSL or APR+OpenSSL? I'm guessing APR+OpenSSL. Are you able to try using NIO+JSSE or NIO+OpenSSL? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #2 from jan.pfei...@centrum.cz --- It happens several times a day or even several times a hour. But it happens in random manner. No single particular request can do it. There are no clues in logs. At least for me. I have one qutie busy image serving servlet which produces tons of different client abort exceptions, closeNowException, Stream is not writable etc. I can only guess it is about concurent access to it. I have other logs available, if you want to look. After latest crash i have found this (only relevant log output before crash): Caused by: java.io.IOException: Unexpected error [120,001] writing data to the APR/native socket [975,220,944] with wrapper [org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper@1ce3e909:975220944]. at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doWriteInternal(AprEndpoint.java:2712) at org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper.doWrite(AprEndpoint.java:2640) at org.apache.tomcat.util.net.SocketWrapperBase.doWrite(SocketWrapperBase.java:691) at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBase.java:628) at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:618) at org.apache.coyote.http2.Http2UpgradeHandler.writeBody(Http2UpgradeHandler.java:653) at org.apache.coyote.http2.Stream$StreamOutputBuffer.flush(Stream.java:777) at org.apache.coyote.http2.Stream$StreamOutputBuffer.doWrite(Stream.java:734) at org.apache.coyote.http2.Http2OutputBuffer.doWrite(Http2OutputBuffer.java:59) at org.apache.coyote.Response.doWrite(Response.java:599) at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:328) Do you want me to run some specific test or post server config or other logs? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 Mark Thomas changed: What|Removed |Added Status|NEW |NEEDINFO --- Comment #1 from Mark Thomas --- I haven't seen any APR crashes for a while now on any platform. Some indication of how to reproduce this crash is going to be needed to investigate this further. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 jan.pfei...@centrum.cz changed: What|Removed |Added OS||All CC||jan.pfei...@centrum.cz -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
