Hello, It turned out that the "sjsxp" library which JAX-WS RI v2.1.3 uses makes Tomcat behave this way. I tried a different version of JAX-WS RI (v2.1.7) which doesn't use the "sjsxp" library anymore and it solved the issue.
A very similar issue posted on the Metro mailing list: http://metro.1045641.n5.nabble.com/JAX-WS-RI-2-1-5-returning-malformed-response-tp1063518.html It's surprising that a bug of a framework which is built on the Servlet API can make such an issue happen, but anyway thank you very much everyone who helped me out. Regards, Kohei > On Dec 4, 2018, at 3:39, Christopher Schultz <ch...@christopherschultz.net> > wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Kohei, > > On 12/1/18 10:06, Kohei Nozaki wrote: >> Hello Konstantin, thanks for sharing the valuable information. >> >>> On Dec 1, 2018, at 19:00, Konstantin Kolinko >>> <knst.koli...@gmail.com> wrote: >>> >>>> * Our downstream Nginx instance (The client of our Tomcat >>>> instance) recorded the error "upstream sent no valid HTTP/1.0 >>>> header while reading response header from upstream" at that >>>> time and the error makes perfect sense concerning the response >>>> which has neither HTTP status line nor HTTP headers. >>> >>> 1. See the official FAQ / Troubleshoting page: >>> https://wiki.apache.org/tomcat/FAQ/Troubleshooting_and_Diagnostics >>> >>> >>> > Especially pay attention to >>> 1) configuring an access log 2) setting system property >>> org.apache.catalina.connector.RECYCLE_FACADES=true >> >> I've investigated "RECYCLE_FACADES" and understand that an >> improperly implemented application which keeps a reference to >> Request or Response objects outside of their lifecycle can make >> such an issue like mine happen (please correct me if I'm wrong..). >> >> >> But I still don't quite understand what does "RECYCLE_FACADES=true" >> do. The Wiki page says "This makes it easier to spot illegal access >> when it happens, instead of waiting until side effects of such >> access become visible" but how does it make easier? Does this >> property make Tomcat produce an Exception or make Tomcat produce >> some warning message to a log file or something when such accesses >> happen, for example? > > Tomcat usually handles requests something like this. Imagine a > single-threaded server where Tomcat only accepts a single connection > at a time (just to simplify the code to the point where it fits into a > ML post). > > Many of these methods are made-up. There is no > TomcatHttpServletRequest class or a .setRequestLine method in it > (though there are ... siilar concepts in there, way down deep). The > point is how the objects are used, or rather *re* used. > > HttpServletRequest request = new TomcatHttpServletRequest(); > HttpServletResponse response = new TomcatHttpServletResponse(); > > Connection conn = null; > > while(null != (conn = socket.acceptConnection()) { > request.setRequestLine(conn.getRequestLine()); > request.setInputStream(conn.getInputStream()); > response.setOutputStream(conn.getOutputStream()); > > Servlet servlet = getServletForRequest(request); > if(null == servlet) > servlet = defaultServlet; > > servlet.service(request, response); > > request.reset(); > response.reset(); > } > > In "real" Tomcat, each Connection object holds its own Request and > Response objects ad manages them in a similar way, and of course, > Tomcat can accept multiple simultaneous connections -- including > multiple requests over a single connection -- simultaneously -- in the > case of HTTP/2. > > If you enable the RECYCLE_FACADES in Tomcat, the code changes to > behave like this: > > Connection conn = null; > > while(null != (conn = socket.acceptConnection()) { > HttpServletRequest request = new TomcatHttpServletRequest(); > HttpServletResponse response = new TomcatHttpServletResponse(); > request.setRequestLine(conn.getRequestLine()); > request.setInputStream(conn.getInputStream()); > response.setOutputStream(conn.getOutputStream()); > > Servlet servlet = getServletForRequest(request); > if(null == servlet) > servlet = defaultServlet; > > servlet.service(request, response); > > request.dispose(); > response.dispose(); > } > > Note how the request and response objects are no longer re-used across > requests. This represents a trade-off between security/stability > (always getting a fresh object) versus performance (less > garbage-collection for a given request). An application can do things > to the request or response that can break the way the server works, so > an untrusted application should always be run with RECYCLE_FACADES set > to ON. > > If the application keeps a reference to a request or response object > after the request has completed (as defined by returning from > servlet.service()), then Bad Things can happen. If you write to that > response's OutputStream, for example, you might write into the > response of *another request* that is being handled after your code > should be finished. > >> >> p.s. Speaking of the possibility of an improper implementation, >> I've found some discussions which seem to have some similarity to >> my case like the following: >> >> * >> http://tomcat.10.x6.nabble.com/NullPointerException-in-MimeHeaders-td2 > 054107.html >> : I've seen some similar Exceptions at the "MimeHeaders" class >> which were recorded in my Tomcat's log too >> >> * >> http://tomcat.10.x6.nabble.com/Tomcat-occasionally-duplicating-respons > es-td5034710.html >> : A case where a bug in a Filter make Tomcat produce an invalid >> HTTP response >> >> Those discussions made me think of this possibility where my app >> might be improperly implemented. I'm going to check the application >> code from this perspective on Monday. > > When you use RECYCLE_FACADES, your application will only retain a > reference to a useless object.... one which will likely NPE at some > point when your code calls it. This makes the application safer > (because one misbehaving servlet won't damage requests/responses > currently in-use by subsequent requests) and also helps you find the > error in your own code (because the problem will usually manifest > itself as an earlier NPE instead of some "weirdness" which cannot > really be readily explained because of the complexities of how objects > are re-used, etc.). > > Hope that helps, > - -chris > -----BEGIN PGP SIGNATURE----- > Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ > <https://www.enigmail.net/> > > iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlwFeG4ACgkQHPApP6U8 > pFgxWxAAmG1UgpXnQUf2WdI6KQrgMa+Xx6nCkrr5XU/QK2EHvGO1jAE6Oc/qBmh4 > ak25KzQJbC25BDCar/JdFuUlxAVwt2qht7M3RoCVqKm/BoZuamnnVXO5fq8a15cW > NWMXFm9Unmh2wJGI12U8ZuuaPwte0rs4MSn+84gIzI81iaPmQB2TD6ohrl4mtJAT > LsZQQks0IJKpu6/kpx2z0DPcqRTIynRIeegBzkvUjXOHFzu5wF99ZnS2etsTEbr6 > bqKmEiSHSp9OjfiwggWMiHmnIPuZNwXziNj8cPpMxTWGLwMNXP1LUe7JSQ+UTKj4 > tF5HYZovYW95h9vRHV/SkiJULmzi2nR/TGCe6noz0cNYon3y7kQJdG3i4GPaQ+bs > +aww7homCjIceWREQbUKIp+/WUNXmtN8f8EwRjq94Av73CTi5inT+gv377D0Y6fG > ui/p2vxkENsRDn/E6khTywxYjxOTruT0cmeYJ4Ee45k4EqMTb7142fxwblq1sgUy > Of3ziwYVW7i6gz5zJaxv6oHxYCKVlHwj8aaqaETon5clGE7yFrRyAsilsLhu86OO > 0ZwFLGUw7JSsiNs/KPODhmsY7XhNevmUu7ejhyfPe/TMle+TQyUmjTrdnjc+yX3C > NwNUnZQUoQhkCseMaC2tp+6I1fo4/iFBLYdfCTrZpffQy4L0Okg= > =uLmM > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > <mailto:users-unsubscr...@tomcat.apache.org> > For additional commands, e-mail: users-h...@tomcat.apache.org > <mailto:users-h...@tomcat.apache.org>