Re: Buildbot slowness

2019-03-08 Thread Daniel Cunha
Nice! +1 Em sex, 8 de mar de 2019 às 05:07, j4fm escreveu: > Hi Jon > > Would you be able to check and merge PR 438 for me? I've broken the work > down into bite sizes. This first one is just to get Cxf/MP Rest client > back > in (all tests succeed and without slowness). Later I will submit s

Re: Buildbot slowness

2019-03-08 Thread j4fm
Hi Jon Would you be able to check and merge PR 438 for me? I've broken the work down into bite sizes. This first one is just to get Cxf/MP Rest client back in (all tests succeed and without slowness). Later I will submit some work around TCK+testng to improve memory handling when running a larg

Re: Buildbot slowness

2019-03-05 Thread j4fm
This is fixed now. PR on it's way, it was mostly just new exclusions needed adding, duh. -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-04 Thread j4fm
When running tests in full here are the tests that appear to cause a leak... just to see if anything common between them is obvious from your experience? Just to be sure, the leak exceptions from these do not appear in the logs at all with CXF 3.2.7. org.jboss.cdi.tck.tests.extensions.configurat

Re: Buildbot slowness

2019-03-04 Thread Jonathan Gallimore
Thanks for the detail. I'm slammed myself, but hopefully will get some time to help tomorrow. I'd encourage others to dig in if they can / wish to. A starting point could be: try pulling out one of the CDI TCK apps, deploy it manually, and then undeploy, and see what's not getting freed and what i

Re: Buildbot slowness

2019-03-04 Thread j4fm
Thanks for merging. Buildbot is succeeding again. So my testng changes help speed up the tests and improve the heap usage (worth adding the changes in at some point) but it didn't solve the root cause. I see these in the logs: SEVERE [http-nio-37745-exec-211] org.apache.catalina.loader.WebappCl

Re: Buildbot slowness

2019-03-04 Thread Jonathan Gallimore
Shoot over a PR. I would definitely be interested in a little bit of detail before merging so we all understand what's happening. Cheers Jon On Mon, Mar 4, 2019 at 3:10 AM j4fm wrote: > So I have a fix for the cdi-tomee tck which means we can add cxf 3.3.0 back > in. I just want to wait for f

Re: Buildbot slowness

2019-03-03 Thread j4fm
So I have a fix for the cdi-tomee tck which means we can add cxf 3.3.0 back in. I just want to wait for full tests to complete before explaining in detail but to sum it up the bug was in testng which was keeping hold of threads. -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-

Re: Buildbot slowness

2019-03-03 Thread j4fm
Hiya, would you be able to merge PR 430 to allow buildbot to pass again for me? Thanks -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-03 Thread Jean-Louis Monteiro
I agree. It would hide the problem Le dim. 3 mars 2019 à 03:23, j4fm a écrit : > I ran it again with 768MB extra head and it crashed right near the end. I > expect 1GB or 1.5GB would definitely let it finish but it must not be > releasing memory on each application undeployment. I don't think

Re: Buildbot slowness

2019-03-02 Thread j4fm
I ran it again with 768MB extra head and it crashed right near the end. I expect 1GB or 1.5GB would definitely let it finish but it must not be releasing memory on each application undeployment. I don't think just increasing the memory and covering up an underlying problem is a good idea. -- S

Re: Buildbot slowness

2019-03-02 Thread j4fm
I ran the :cdi-tomee tck to successful completion with an extra 756MB added to the max heap for this arquillian run. I'm running it again just to make sure. Would updating master with the config be acceptable? We could have cxf 3.3.0+mp rest client back in if so. -- Sent from: http://tomee-op

Re: Buildbot slowness

2019-03-02 Thread j4fm
Hey Jon, do you know if your test run with an increased java heap froze up or succeeded? Thanks -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-02 Thread j4fm
Slowness is fixed, so buildbot is happier but not quite happy. Just for awareness to avoid someone else looking at it too... 1. I forgot to update the CXF version in the examples\polling-parent. So polling-web failed. 2. Also, it turns out CXF didn't add support for mp rest client 1.2.0 until 3

Re: Buildbot slowness

2019-03-02 Thread Daniel Cunha
nice! Em sáb, 2 de mar de 2019 às 06:47, Jonathan Gallimore < jonathan.gallim...@gmail.com> escreveu: > Done > > On Sat, 2 Mar 2019, 09:37 j4fm, wrote: > > > CXF 3.3.1 is also causing the problem. > > > > I've submitted PR 429 just to make buildbot happy again for the time > being. > > Would you

Re: Buildbot slowness

2019-03-02 Thread Jonathan Gallimore
Done On Sat, 2 Mar 2019, 09:37 j4fm, wrote: > CXF 3.3.1 is also causing the problem. > > I've submitted PR 429 just to make buildbot happy again for the time being. > Would you mind merging just to get the builds going again? Will be good to > confirm it's happy with the other PRs. > > Once a r

Re: Buildbot slowness

2019-03-02 Thread j4fm
CXF 3.3.1 is also causing the problem. I've submitted PR 429 just to make buildbot happy again for the time being. Would you mind merging just to get the builds going again? Will be good to confirm it's happy with the other PRs. Once a root cause/solution is found can bump CXF back up again.

Re: Buildbot slowness

2019-03-02 Thread Jonathan Gallimore
I was seeing the same, but didn't get to dig into it deeper yet - its unlikely I'll be able to do much over the weekend, but we'll see. I did bump up the heap on my build to 6G (from 1.5) to see how far it gets. Jon On Sat, 2 Mar 2019, 08:54 j4fm, wrote: > It's just changing CXF version to 3.3

Re: Buildbot slowness

2019-03-02 Thread j4fm
It's just changing CXF version to 3.3 on it's own that causes it. Reverting back to 3.2.7 solves it. All the JAXB 2.8.2 and related dependency work seems to be fine. Though I see the CXF mp client version bump adds in activation-api which needs excluding. I'm testing 3.3.1 out now just to check (

Re: Buildbot slowness

2019-03-02 Thread Jonathan Gallimore
Did anyone take a look at the heap dump (I can make it available if needed)? If you did, what did you find? I left visualvm computing sizes when I went to bed last night. Not sure where it's at. Hoping to get a look over the weekend. Jon On Sat, 2 Mar 2019, 07:27 j4fm, wrote: > Building is no

Re: Buildbot slowness

2019-03-02 Thread Daniel Cunha
What is the mvn command which you are using to run it? Em sáb, 2 de mar de 2019 às 04:27, j4fm escreveu: > Building is not a problem, it's the CDI TCK test run causing out of memory > exception in tomee > > > > -- > Sent from: > http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html >

Re: Buildbot slowness

2019-03-01 Thread j4fm
Building is not a problem, it's the CDI TCK test run causing out of memory exception in tomee -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-01 Thread Daniel Cunha
Hi folks, it is happening in a specif module? I'm building tomee without samples and seems fine to me. mvn clean install -Pquick Em sáb, 2 de mar de 2019 às 00:36, j4fm escreveu: > Hey Jon, thanks for the test. Do you still have all my PRs in there after > just removing the PR409? > > Yeah, I

Re: Buildbot slowness

2019-03-01 Thread j4fm
Hey Jon, thanks for the test. Do you still have all my PRs in there after just removing the PR409? Yeah, I was looking around all the dependency changes that CXF (and JAXB which it depends on) brings in. I do have something I am working on, instead of excluding the older javax conflicting depende

Re: Buildbot slowness

2019-03-01 Thread César Hernández Mendoza
Thanks for the update James. I reverted locally: https://github.com/apache/tomee/pull/409 , build tome without tests and then: mvn -pl tck/cdi-tomee --fail-at-end -Djava.io.tmpdir=/tmp/tomee-trunk-ubuntu-jvm8 clean install -Pall-adapters -Dsurefire.useFile=false -DdisableXmlReport=true -Dopenejb.

Re: Buildbot slowness

2019-03-01 Thread j4fm
False fix, it just got a lot further before an oom exception. -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-01 Thread j4fm
I have a fix for the slow build (works locally without memory error now). If somebody wouldn't mind stopping the current buildbot task and I will create a small PR. Thanks in advance -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-01 Thread j4fm
Not sure it's ever going to finish. Seems to be grinding to a halt. You're right, no point letting it run. -- Sent from: http://tomee-openejb.979440.n4.nabble.com/TomEE-Dev-f982480.html

Re: Buildbot slowness

2019-03-01 Thread j4fm
Would like to get the stats at the end of the CI build if possible too. PR409 was the first merge to start the issue but there are no code changes in there except to examples and the bumps of CXF and JAXB. Do you see it just being cdi-tomee that's slowed up? I'm checking out it's (indirect) depe

Re: Buildbot slowness

2019-03-01 Thread Jonathan Gallimore
Waiting on my machine to crunch some numbers, but here's the summary: Total Bytes: 2,726,889,443 Total Classes: 124,138 Total Instances: 54,413,232 Classloaders: 790 GC Roots: 3,842 Number of Objects Pending for Finalization: 0 That seems like a lot of cl

Re: Buildbot slowness

2019-03-01 Thread Jonathan Gallimore
On the plus side, I have a heap dump here, so we can take a look. Jon On Fri, Mar 1, 2019 at 9:28 PM Jonathan Gallimore < jonathan.gallim...@gmail.com> wrote: > Seeing the same with the java11-experimentation build here: > > # Created on 2019-03-01T18:01:43.414 > Corrupted STDOUT by directly wri

Re: Buildbot slowness

2019-03-01 Thread Jonathan Gallimore
Seeing the same with the java11-experimentation build here: # Created on 2019-03-01T18:01:43.414 Corrupted STDOUT by directly writing to native stream in forked JVM 1. Stream 'java.lang.OutOfMemoryError: Java heap space'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma a

Buildbot slowness

2019-03-01 Thread j4fm
Getting this WARNING on :cdi-tomee on buildbot which is taking forever... # Created on 2019-03-01T20:48:20.010 Corrupted STDOUT by directly writing to native stream in forked JVM 1. Stream 'java.lang.OutOfMemoryError: Java heap space'. java.lang.IllegalArgumentException: Stream stdin corrupted. Ex

Buildbot slowness

2019-03-01 Thread j4fm
Buildbot is slow with :cdi-tomee tests causing out of memory warnings. The PR409 (CXF 3.3.0 and JAXB 2.3.2) updates was the first merge to cause the slowness. The rolled up merges after show the same. # Created on 2019-03-01T20:48:20.010 Corrupted STDOUT by directly writing to native stream in