> Another option would be to make the Jenkins build status non-mandatory for merging the PRs.
Then we might want to be careful before merging because flaky tests might be failing due to genuine reason or bug. Probably, we can keep it enable for sometime and disable if we really think that flaky tests are giving lot of pain for merging? Thanks, Rajan On Wed, Aug 2, 2017 at 1:49 PM, Matteo Merli <mme...@apache.org> wrote: > Another option would be to make the Jenkins build status non-mandatory for > merging the PRs. > > In this case, if we see any known flaky tests, we could proceed with > merging. For any new failing test, we > should re-run the build and create a new issue to track it. > > Pros: > * We don't have to spend big amount of time rebuilding the same PRs to > merge > > Cons: > * We'll get acquainted with the failing tests and possibly never fix them. > > Any thoughts or preferences? > > > > -- > Matteo Merli > <mme...@apache.org> > > On Wed, Aug 2, 2017 at 12:22 PM, Matteo Merli <mme...@apache.org> wrote: > > > Reading on the Jenkins Github plugin documentation at > > https://wiki.jenkins.io/display/JENKINS/GitHub+pull+ > request+builder+plugin > > > > > > You can trigger the build of a PR by adding a comment "retest this > please" > > on the PR itself. This will kick a new build that will properly update > the > > PR status. > > > > This will work for all committers. > > > > > > > > -- > > Matteo Merli > > <mme...@apache.org> > > > > On Wed, Aug 2, 2017 at 10:24 AM, Matteo Merli <mme...@apache.org> wrote: > > > >> As most of you have seen, we have a bunch of intermittent test failures. > >> > >> This, added to the difficulties in rebuilding a pull-request and > updating > >> the status lead to a bit of a cumbersome process to get a PR merged. > >> > >> I have created issues for the tests I've seen failing and marked them > >> with label "component/test". The full list is at > >> https://github.com/apache/incubator-pulsar/issues?q=is%3Aiss > >> ue+is%3Aopen+label%3Acomponent%2Ftest > >> > >> Please anyone that has time to investigate these, take one, assign to > >> yourself and try to reproduce. Fortunately, in new builds we now have > >> individual INFO logs for each of the test, so that it might help in the > >> debugging. > >> > >> > >> > >> -- > >> Matteo Merli > >> <mme...@apache.org> > >> > >> On Tue, Aug 1, 2017 at 9:40 AM, Matteo Merli <mme...@apache.org> wrote: > >> > >>> As we discussed some time back, I have added CI builds in Jenkins. > >>> > >>> Jenkins seems to be much more stable compare to Travis and it's bit > more > >>> flexible in terms of what we can do. For example, for C++ build we can > use > >>> custom Docker image with all the dependencies we require to build the > >>> Pulsar client lib. > >>> > >>> I have asked INFRA to disable the mandatory check on Travis (to merge > >>> the PRs) and instead use Jenkins for the same purpose. > >>> > >>> The build page links: > >>> > >>> https://builds.apache.org/job/pulsar-master/ > >>> > >>> https://builds.apache.org/job/pulsar-pull-request/ > >>> > >>> The only minor issue that I can see is that the "Rebuild" button the on > >>> the PR builder is not updating back the status on the Github pull > request. > >>> The workaround is to "repush" the branch on github. If there are no > >>> changes, you can do a "git commit --amend" and it will just update the > >>> timestamp of the commit and force a new sha hash. Then push --force to > >>> update the PR. > >>> > >>> If anyone knows or wants to take a look for a better solution.. :) > >>> > >>> > >>> > >>> > >>> -- > >>> Matteo Merli > >>> <mme...@apache.org> > >>> > >> > >> > > >
