We seem to have resolved the problem. The initial apache configuration was using mod_proxy ProxyPass to send traffic to ajp:// endpoints on the jboss server. We switched this configuration to use http:// endpoints on jboss and the problem has gone away.
Thanks to all for the input. On Mon, Oct 20, 2014 at 1:18 PM, Fiedler Roman <roman.fied...@ait.ac.at> wrote: > > Von: Tom Purcell [mailto:tpurc...@chariotsolutions.com] > > > > Roman > > > > I've used several clients (curl, a java application, postman) as well as > the > > mobile apps. All exhibit the same behavior: sometimes returning a 200 > when > > a 400 is expected. > > > > I ran tcpdump on the server this morning. It shows no difference between > > properly reported 400s and incorrect 200s. > > Could you compare (or even send) the tcpdumps of a 200 and one 400 > response at > a) interface, where data is entering/leaving to client and b) between jboss > and apache? > > If jboss does not send the 400 response, operation of Apache on reply data > from jboss has to be the cause, hence request/response data should give a > hint. > > > > > On Mon, Oct 20, 2014 at 3:57 AM, Fiedler Roman <roman.fied...@ait.ac.at> > > wrote: > > > > > > Hello Tom, > > > > > Von: Tom Purcell [mailto:tpurc...@chariotsolutions.com] > > > > > > > > Hello > > > > > > We have an application that consists of REST endpoints on a jboss > > > server(5.1.0) fronted by Apache httpd(2.2.15). When a client > makes > > a bad > > > request it usually gets the expected 400 http response code but > > sometimes > > > the client sees a 200. This happens sporadically. Two days ago I > ran a > > test > > > where it happened 11 out of 20 times. Today the highest > > occurrence has > > > been 3 out of 40. > > > > > > To add some context here's some output from the test. Note both > > calls are > > > identical but one gets a 400, the other 200: > > > > > > curl -s -D- -u user:passwd -X POST --data @uc.json > > > https://ourdomain.com/ourapp/rest/v3/subscriber/<id>/user/0 > > 2>&1 > > > > > > 21 : HTTP/1.1 400 Bad Request^M Date: Wed, 15 Oct 2014 > > 18:44:22 > > > GMT^M X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1^M > > Content- > > > Type: application/json^M Content-Length: 361^M Connection: > > close^M > > > ^M^M { "ErrorMessage" : { "errorMessage" : "Json validation > failure: > > > {level=\"error\", > > > > > > schema={\"loadingURI\":\"#\",\"pointer\":\"/properties/userConfig/propertie > > > s/userType\"}, instance={\"pointer\":\"/userConfig/userType\"}, > > > domain=\"validation\", keyword=\"enum\", message=\"instance > > does not > > > match any enum value\", enum=[\"0\",\"1\",\"3\"], value=0}" } } > > > > > > curl -s -D- -u user:passwd -X POST --data @uc.json > > > https://ourdomain.com/ourapp/rest/v3/subscriber/<id>/user/0 > > 2>&1 > > > > > > 22 : HTTP/1.1 200 OK^M Date: Wed, 15 Oct 2014 18:44:24 > > GMT^M > > > Transfer-Encoding: chunked^M Content-Type: text/plain; > > charset=UTF-8^M > > > ^M { "ErrorMessage" : { "errorMessage" : "Json validation > failure: > > > {level=\"error\", > > > > > > schema={\"loadingURI\":\"#\",\"pointer\":\"/properties/userConfig/propertie > > > s/userType\"}, instance={\"pointer\":\"/userConfig/userType\"}, > > > domain=\"validation\", keyword=\"enum\", message=\"instance > > does not > > > match any enum value\", enum=[\"0\",\"1\",\"3\"], value=0}" } } > > > > > > The following are the Apache ssl_access.log entries that > correspond > > to the > > > above calls. Note both got a 400: > > > > > > > > > 10.102.211.152 - - [15/Oct/2014:14:44:22 -0400] "POST > > > /ourapp/rest/v3/subscriber/<id>/user/0 HTTP/1.1" 400 361 > > > > > > 10.102.211.152 - - [15/Oct/2014:14:44:24 -0400] "POST > > > /ourapp/rest/v3/subscriber/<id>/user/0 HTTP/1.1" 400 361 > > > > > > More context: > > > > > > > > * This never happens going directly against the jboss server > > > * It does happen both with and without SSL when hitting > Apache > > > * The tests results shown above were run with using curl as a > > client but > > > we have also seen it happen with other clients(Charles, > wireshark, > > IOS apps, > > > etc) > > > * Note that the 200 response above does not mention the jboss > > server > > > but that the 400 does. I have verified that both requests hit > the jboss > > > server > > > by locating the stack traces in the jboss log. > > > * Normally there is an F5 in the mix and when hitting the app > > through it > > > we get the same results. That said the tests referred to here > > bypassed the > > > F5 > > > and hit the Apache server directly > > > > > > So we should get a 400 but when it gets to the client the > response > > code is > > > 200. Any thoughts? > > > > Two questions: > > > > * Did you run all your tests via local interface on server or did > you use > > mobile carriers all the time? Mobile carriers can do quite > > unbelievable > > things, even on SSL (the partially run MITM to proxy/optimize the > > connections). We had problems with error codes in such a scenario > > tool > > * What did Apache send on the wire? A tcpdump on the server, > > perhaps compared > > to one on your firewall next to the server would be nice. > > > > Roman > > > > > > > > > > > > > > -- > > > > Thanks, > > Tom Purcell > > Cell: 215-779-1963 > > -- Thanks, Tom Purcell Cell: 215-779-1963
