It's really easy to create and modify those builds. If the issue is that we need to add SBT or Maven to the existing one, it's a short change. We can just have it build both of them. I wasn't aware of things breaking before in one build but not another.
- Patrick On Mon, Oct 12, 2015 at 9:21 AM, Sean Owen <so...@cloudera.com> wrote: > Yeah, was the issue that it had to be built vs Maven to show the error > and this uses SBT -- or vice versa? that's why the existing test > didn't detect it. Was just thinking of adding one more of these non-PR > builds, but I forget if there was a reason this is hard. Certainly not > worth building for each PR. > > On Mon, Oct 12, 2015 at 5:16 PM, Patrick Wendell <pwend...@gmail.com> > wrote: > > We already do automated compile testing for Scala 2.11 similar to Hadoop > > versions: > > > > https://amplab.cs.berkeley.edu/jenkins/view/Spark-QA-Compile/ > > > https://amplab.cs.berkeley.edu/jenkins/view/Spark-QA-Compile/job/Spark-master-Scala211-Compile/buildTimeTrend > > > > > > If you look, this build takes 7-10 minutes, so it's a nontrivial > increase to > > add it to all new PR's. Also, it's only broken once in the last few > months > > (despite many patches going in) - a pretty low failure rate. For > scenarios > > like this it's better to test it asynchronously. We can even just revert > a > > patch immediately if it's found to break 2.11. > > > > Put another way - we typically have 1000 patches or more per release. > Even > > at one jenkins run per patch: 7 minutes * 1000 = 7 days of developer > > productivity loss. Compare that to having a few times where we have to > > revert a patch and ask someone to resubmit (which maybe takes at most one > > hour)... it's not worth it. > > > > - Patrick > > > > On Mon, Oct 12, 2015 at 8:24 AM, Sean Owen <so...@cloudera.com> wrote: > >> > >> There are many Jenkins jobs besides the pull request builder that > >> build against various Hadoop combinations, for example, in the > >> background. Is there an obstacle to building vs 2.11 on both Maven and > >> SBT this way? > >> > >> On Mon, Oct 12, 2015 at 2:55 PM, Iulian Dragoș > >> <iulian.dra...@typesafe.com> wrote: > >> > Anything that can be done by a machine should be done by a machine. I > am > >> > not > >> > sure we have enough data to say it's only once or twice per release, > and > >> > even if we were to issue a PR for each breakage, it's additional load > on > >> > committers and reviewers, not to mention our own work. I personally > >> > don't > >> > see how 2-3 minutes of compute time per PR can justify hours of work > >> > plus > >> > reviews. > > > > >
