I have been hitting these issues and investigating them. - HCatalog pentaho dependency: https://issues.apache.org/jira/browse/BEAM-3621 - We rerun way way more than we need to: https://issues.apache.org/jira/browse/BEAM-3253 but dependency stuff occurs during config, not tasks, I think
The biggest source of flakes with Maven is dependency download. We only got any green with maven once we saved .m2 across builds. It seems like gradle is in that state, and is not caching them. I'm not sure why - by default it should cache to $HOME/.gradle. I think some versions of the Jenkins plugin caches to $WORKSPACE/.gradle but that should also work just fine unless we are wiping out the workspace, which we shouldn't do as we have isolated the build to the $WORKSPACE/src directory. When I look in a workspace I don't see a cache anyhow. There's an issue with the Jenkins workers so I can't ssh to them at the moment to investigate. I am hoping that https://github.com/apache/beam/pull/4644 will also make it easier to see what is happening. Kenn On Fri, Feb 9, 2018 at 5:29 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi Romain, > > that's actually part of my investigations ;) > > Regards > JB > > On 02/09/2018 02:26 PM, Romain Manni-Bucau wrote: > > did you check the concurrency? For me it is plain wrong since it is > hardcoded in > > the build file and doesnt let me or the tool customize it. This means it > just > > blocks my computer in general and makes some timeout related tests fail > easily. > > (this kind of config should always be customized on the CI and never > hardcoded IMHO) > > > > > > Romain Manni-Bucau > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > <https://rmannibucau.metawerx.net/> | Old Blog > > <http://rmannibucau.wordpress.com> | Github <https://github.com/ > rmannibucau> | > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > <https://www.packtpub.com/application-development/java- > ee-8-high-performance> > > > > 2018-02-09 12:00 GMT+01:00 Aljoscha Krettek <aljos...@apache.org > > <mailto:aljos...@apache.org>>: > > > > Yes, I was just about to write about this as well. In my recent PRs > this > > always failed for different reasons. > > > > Thanks for looking into this! > > > > > On 9. Feb 2018, at 11:35, Jean-Baptiste Onofré <j...@nanthrax.net > > <mailto:j...@nanthrax.net>> wrote: > > > > > > Hi guys, > > > > > > I noticed that the Gradle build on Jenkins is flaky: it almost > always > > fails for > > > different reason (I can't download pentaho artifact sometime, > interrupted > > other > > > times). > > > > > > Jenkins is doing: > > > > > > Jenkins: ./gradlew --continue --rerun-tasks :javaPreCommit > > > > > > I gonna investigate how to improve the situation. > > > > > > Regards > > > JB > > > -- > > > Jean-Baptiste Onofré > > > jbono...@apache.org <mailto:jbono...@apache.org> > > > http://blog.nanthrax.net > > > Talend - http://www.talend.com > > > > > > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com >
