Yes the module that hangs is the driver for remote endpoints and stands up a Fuseki server and communicates with it using HTTP which of course now all goes through HttpOp
Problem is that I never seem to get an actual exception just hangs on the build server. This might also explain why DEBUG level logging makes the build succeed because HttpClient is very noisy at DEBUG level and all that logging likely introduces the delays in the right parts of the code to allow resources to be freed up. Rob On 8/8/13 3:40 PM, "Andy Seaborne" <[email protected]> wrote: >On 08/08/13 19:42, Rob Vesse wrote: >> So I am officially stumped >> >> Adding the delay still causes the builds to hang so I really don't >> understand why the builds fail on Apache Jenkins. Note that I've been >> building the JDBC module on our internal Jenkins server for some time >>and >> never had an issue there. Plus the builds run fine on a local machine. >> >> If anyone else can take a look or has any suggestions please jump in > ><straw-grasping mode> > >Are you using HttpOp? Apache HttpClient? > >I'm fairly certain HttpOp can cause resource starvation by improper use >of HttpClient. However, I haven't managed to find out where for certain >[HTTP Exceptions are my current best guess]. (I can perturb the >situation by tweaking pooling numbers.) > > Andy > >> >> Rob >> >> >> On 8/8/13 11:12 AM, "Rob Vesse" <[email protected]> wrote: >> >>> Ok, so turning the log level back down causes the build to go back to >>> failing >>> >>> This starts to look like some kind of timing issue manifesting on the >>> build server causing the tests to get into a hung state. Apparently >>> having the high log level adds sufficient delay into the process to >>>avoid >>> this. >>> >>> My next idea is to simply insert a delay between the tests in question >>>and >>> see if that solves things. >>> >>> Rob >>> >>> >>> >>> On 8/8/13 10:55 AM, "Rob Vesse" <[email protected]> wrote: >>> >>>> Ok that is very very weird, after turning up the logging for that >>>>module >>>> the build ran through to success (and generated a ridiculously large >>>>log >>>> file at the same time). >>>> >>>> Next step is to try turning down the log level and see if the build >>>>still >>>> succeeds. >>>> >>>> Rob >>>> >>>> >>>> >>>> On 8/8/13 10:35 AM, "Rob Vesse" <[email protected]> wrote: >>>> >>>>> The problem is that nothing is blowing up, the build just gets stuck >>>>>and >>>>> hangs until the build timeout plugin steps in and aborts the build >>>>> >>>>> >>>>> The hang is in the tests for the remote endpoint driver which are >>>>> standing >>>>> up Fuseki instances. However if there was some contention for ports >>>>>in >>>>> the tests I would expect the tests to just plain fail. >>>>> >>>>> I suspect there may be some deadlock of some sort happening when >>>>>running >>>>> the tests on the server but it's hard to tell where/what the deadlock >>>>> is. >>>>> I am turning the log level for the tests in question to DEBUG and >>>>>will >>>>> re-run a build to see if that yields anything more useful. >>>>> >>>>> Rob >>>>> >>>>> On 8/8/13 6:53 AM, "Andy Seaborne" <[email protected]> wrote: >>>>> >>>>>> On 01/08/13 20:56, Rob Vesse wrote: >>>>>>> I've removed it from the main build for now. For some reason it is >>>>>>> getting stuck (but not crashing) on the Apache build server. This >>>>>>>is >>>>>>> despite it building fine locally and on our internal build servers. >>>>>>> >>>>>>> Not sure how to proceed on this - is it worth setting up a separate >>>>>>> build >>>>>>> for JDBC on the Apache build servers to help try and isolate the >>>>>>> problem? >>>>>>> >>>>>>> Rob >>>>>> >>>>>> What exactly is blowing up? >>>>>> >>>>>> The Apache build servers have all sorts of things on them and a wide >>>>>> range of plugins, which itself can a problem. >>>>>> >>>>>> Andy >>>>>> >>>>>>> >>>>>>> >>>>>>> On 8/1/13 11:45 AM, "Rob Vesse" <[email protected]> wrote: >>>>>>> >>>>>>>> I've moved Jena JDBC from Experimental into Trunk and added it to >>>>>>>> the >>>>>>>> main build. The builds are a little nosier that some of the other >>>>>>>> modules so may want some tweaking to avoid spurious build output. >>>>>>>> >>>>>>>> I haven't attempted to figure out how to add it to the distro >>>>>>>> because >>>>>>>> I >>>>>>>> know nothing about Maven Assembly plugin >>>>>>>> >>>>>>>> Rob >>>>>>> >>>>>> >>>>> >>>> >>> >> >
