I had all of the tests passing at least once for 1.7.2, some had to be rerun however.
On Wednesday, August 3, 2016, Christopher <ctubb...@apache.org> wrote: > On Wed, Aug 3, 2016 at 5:47 PM Sean Busbey <bus...@cloudera.com > <javascript:;>> wrote: > > > My understanding was that maintenance releases (aka double dot, e.g. > > 1.7.2) had relaxed criteria because we expected the scope of changes > > in them to be more limited. Even so, the release notes for 1.7.2, > > 1.7.1, and 1.7.0 all claim the ITs passed. > > > > > Even those releases have periodic IT failure. > > > > Is there a reason we can't parallelize the ITs? > > > We can. Eric's mrit effort was all intended towards that. But, that's not > the same as CI passing. I don't know what it would take to parallelize them > in a CI server. > > > > What's stopping > > builds.a.o from running them? Specific requests from projects to asf > > infra can get us resources if that's the problem. > > > > > I spoke to infra in HipChat about this a a few weeks ago, and mentioned a > few things which impact builds on ASF jenkins (builds.apache.org): > > 1. Accumulo has an excessive number of tests to run. > 2. Build timeouts with Jenkins can abort builds. > 3. Tests are timing sensitive, and are affected by VM/host configuration > and contention with other concurrent builds from other projects. > 4. Tests need lots of RAM and storage (at least 4GB RAM, but ideally no > less than 16GB, and at least 6 GB for a workspace) > 5. Tests need specialized system configuration, (increasing ulimits, > optimizing kernel settings for swappiness, etc.) > > What we really need for reliable IT passing in CI, is exclusive use of > dedicated, bare-metal beefy build machines, for 6+ hours per build x 4 > branches minimum, plus another 6+ hours for each pull request and other > builds which skipITs, so we can get immediate feedback on unit tests and > compilation errors. > > I don't think it's reasonable for us to even ask for such resources from > INFRA. Using ASF Jenkins for -DskipITs I think is reasonable, with > individual testing of full IT suite from the developers/contributors, as > well as feedback from independent dedicated CI builds (such as the Jenkins > server I'm running, as well as any Josh might still be running, and > others). > > The minimum passing to create a release candidate is -Psunny. Usually folks > submit test results on the full ITs which informed their vote. Sometimes, > but not always, the release manager will do some pre-vote testing and > prepare draft release notes. Testing information also gets aggregated from > the vote thread onto the release notes. >
