Given the length of time precheckin seems to run, would it make sense to break it up?
-Sean. On Thu, Nov 2, 2017 at 11:49 AM, Dan Smith <dsm...@pivotal.io> wrote: > Looks good. Should we go ahead and change this to run precheckin instead of > build? > > -Dan > > On Thu, Nov 2, 2017 at 9:53 AM, Anthony Baker <aba...@pivotal.io> wrote: > > > If you’d like to check this out, here’s the PR containing the pipeline > and > > job scripts: > > https://github.com/apache/geode/pull/1006 > > > > And the pipeline itself: > > https://concourse.apachegeode-ci.info > > > > There are three pipelines defined: > > > > - develop: runs `gradle build`. Can be extended to include other > > precheckin tests based on feedback. > > - docker-images: builds the container used for the develop pipeline. > > - meta: watches for changes to the pipeline files and automatically > > updates the runtime pipelines. > > > > Authentication is integrated with GitHub. If you want the ability to > > manually stop/start jobs please request on the dev@g.a.o mailing list > > (same as for Jenkins) and include your GitHub id. > > > > What do you think? > > > > Anthony > > > > > On Oct 6, 2017, at 7:08 AM, Anthony Baker <aba...@pivotal.io> wrote: > > > > > > Hi all, > > > > > > I’d like to propose the following that we switch our continuous > > > integration (CI) system from Jenkins [1] to Concourse [2]. I suggest > > > this because we continue to experience a significant number of > > > environmental-related test failures. > > > > > > These issues include CPU interference from other Jenkins jobs on the > > > same host, running out of disk space, port conflicts, and other > > > gremlins. The net effect is that we are only getting 1-2 successful > > > builds per month. Certainly not all test failures can be traced back > > > to environmental issues. However, internal testing on isolated VM’s > > > shows a combined success rate of about 3X higher compared to ASF > > > Jenkins for the same tests. This is still definitely NotAwesome, but > > > removing environmental factors will let us focus on stabilizing flaky > > > tests. > > > > > > Concourse is an Apache-licensed open source CI system based on > > > pipelines. The pipelines are defined in a YML file containing job > > > definitions—inputs, outputs, resources, and tasks. A task is simply a > > > bash script that returns 0/1 for success/failure. A web UI displays > > > build status. Importantly, each job runs inside an isolated > > > container. The containers are load-balanced across a pool of workers. > > > For an example of a build pipeline, see [3] for the pipeline used to > > > build concourse itself. > > > > > > A Concourse environment is deployed and managed in cloud environments > > > through bosh [4]. Pivotal has agreed to donate AWS and/or GCP compute > > > and storage resources as well as manage the infrastructure. These > > > project resources would be available for use by all committers and > > > community members regardless of corporate affiliations. Note that > > > AFAIK there is no explicit requirement to host CI on ASF > > > infrastructure—unlike for critical project resources such as source > > > code, mailing lists, and issue tracking. > > > > > > The source for the pipeline and job scripts would reside within the > > > geode-* repos. Geode committers would be able to modify those, same > > > as with our .travis.yml scripts. All test results and build artifacts > > > would be publicly viewable just like with our Jenkins build output > > > today. Requests for admin assistance would go through the dev@geode > > > mailing list. > > > > > > Thoughts? As a first step we could run both CI systems side-by-side > > > and see how the Concourse approach works for our project. > > > > > > Thanks, > > > Anthony > > > > > > > > > [1] https://builds.apache.org/job/Geode-nightly/ > > > [2] https://concourse.ci > > > [3] https://ci.concourse.ci > > > [4] https://bosh.io > > > > >
