Another option would be to make the Jenkins build status non-mandatory for merging the PRs.
In this case, if we see any known flaky tests, we could proceed with merging. For any new failing test, we should re-run the build and create a new issue to track it. Pros: * We don't have to spend big amount of time rebuilding the same PRs to merge Cons: * We'll get acquainted with the failing tests and possibly never fix them. Any thoughts or preferences? -- Matteo Merli <mme...@apache.org> On Wed, Aug 2, 2017 at 12:22 PM, Matteo Merli <mme...@apache.org> wrote: > Reading on the Jenkins Github plugin documentation at > https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin > > > You can trigger the build of a PR by adding a comment "retest this please" > on the PR itself. This will kick a new build that will properly update the > PR status. > > This will work for all committers. > > > > -- > Matteo Merli > <mme...@apache.org> > > On Wed, Aug 2, 2017 at 10:24 AM, Matteo Merli <mme...@apache.org> wrote: > >> As most of you have seen, we have a bunch of intermittent test failures. >> >> This, added to the difficulties in rebuilding a pull-request and updating >> the status lead to a bit of a cumbersome process to get a PR merged. >> >> I have created issues for the tests I've seen failing and marked them >> with label "component/test". The full list is at >> https://github.com/apache/incubator-pulsar/issues?q=is%3Aiss >> ue+is%3Aopen+label%3Acomponent%2Ftest >> >> Please anyone that has time to investigate these, take one, assign to >> yourself and try to reproduce. Fortunately, in new builds we now have >> individual INFO logs for each of the test, so that it might help in the >> debugging. >> >> >> >> -- >> Matteo Merli >> <mme...@apache.org> >> >> On Tue, Aug 1, 2017 at 9:40 AM, Matteo Merli <mme...@apache.org> wrote: >> >>> As we discussed some time back, I have added CI builds in Jenkins. >>> >>> Jenkins seems to be much more stable compare to Travis and it's bit more >>> flexible in terms of what we can do. For example, for C++ build we can use >>> custom Docker image with all the dependencies we require to build the >>> Pulsar client lib. >>> >>> I have asked INFRA to disable the mandatory check on Travis (to merge >>> the PRs) and instead use Jenkins for the same purpose. >>> >>> The build page links: >>> >>> https://builds.apache.org/job/pulsar-master/ >>> >>> https://builds.apache.org/job/pulsar-pull-request/ >>> >>> The only minor issue that I can see is that the "Rebuild" button the on >>> the PR builder is not updating back the status on the Github pull request. >>> The workaround is to "repush" the branch on github. If there are no >>> changes, you can do a "git commit --amend" and it will just update the >>> timestamp of the commit and force a new sha hash. Then push --force to >>> update the PR. >>> >>> If anyone knows or wants to take a look for a better solution.. :) >>> >>> >>> >>> >>> -- >>> Matteo Merli >>> <mme...@apache.org> >>> >> >> >
