I'd be hugely in favor, however, this is not what the Apache Jenkins supports right now, AFAIK. I've asked Infra about this awhile ago, but nothing has moved yet. There was also a Jira issue about it, INFRA-11610.
On Thu, Oct 20, 2016 at 12:24 PM, Amit Sela <amitsel...@gmail.com> wrote: > Hi all, > > I'd like to discuss options to execute ROS tests (per runner) more > efficiently, and explore the option of running them on PreCommit, as > opposed to PostCommit as they run today. > > The SDK provides a set of tests called "RunnableOnService" (aka ROS) that > can be applied to a runner and validate it (correctly) supports SDK > features. > It's 300+ tests in total (batch + streaming) and it clearly takes time, and > that is why it runs on PostCommit. > I think we should look for a configuration where this is executed > more efficiently and if possible run on PreCommit since runners are > encouraged to rely on those tests and it's better to know of breaking > changes before hand. > > This came up somewhere in this > <https://github.com/apache/incubator-beam/pull/1055> conversation, and the > highlights are basically: > > Kenneth Knowles suggested we might parallelize sub-builds in the following > manner: > > 1. Run unit tests. > 2. (sub tasks) Run ROS tests for each runner in parallel, skipping unit > tests. > > I was wondering if we could setup Jenkins to run ROS per runner only of > there was a code change for that runner - of course SDK changes will > probably have to run ROS for all runners, but that might still be an > optimization. > > I think one of Beam's best sell-points is it's extensive testing framework, > and the fact that runners can be validated across capabilities, but it > would be best to know of runner-braking changes before merging to master. > > Thoughts ? > > Thanks, > Amit >
