(Renaming thread so as to un-hijack Marcelo's request.) Sure, we definitely want tests running faster.
Part of "testing all the things" will be factoring out stuff from the various builds that can be run just once. We've also tried in the past (with little success) to parallelize test execution <https://issues.apache.org/jira/browse/SPARK-3431>. That still needs work before it becomes possible. Nick On Thu, Apr 2, 2015 at 11:59 AM shane knapp <skn...@berkeley.edu> wrote: > i agree with all of this. but can we please break up the tests and make > them shorter? :) > > On Thu, Apr 2, 2015 at 8:54 AM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> This is secondary to Marcelo’s question, but I wanted to comment on this: >> >> Its main limitation is more cultural than technical: you need to get >> people >> to care about intermittent test runs, otherwise you can end up with >> failures that nobody keeps on top of >> >> This is a problem that plagues Spark as well, but there *is* a technical >> solution. >> >> The solution is simple: *All* the builds that we care about run for >> *every* >> proposed change. If *any* build fails, the change doesn’t make it into the > > >> repository. >> >> Spark already has a pull request builder that tests and reports back on >> PRs. Committers don’t merge in PRs when this builder reports that it >> failed >> some tests. That’s a good thing. >> >> The problem is that there are several other builds that we run on a fixed >> interval, independent of the pull request builder. These builds test >> different configurations, dependency versions, and environments than what >> the PR builder covers. If one of those builds fails, it fails on its own >> little island, with no-one to hear it scream. The build failure is >> detached >> from the PR that caused it to fail. >> >> What should happen is that the whole matrix of stuff we care to test gets >> run for every PR. No PR goes in if any build we care about fails for that >> PR, and every build we care about runs for every commit of every PR. >> >> Really, this is just an extension of the basic idea of the PR builder. It >> > doesn’t make much sense to test stuff *after* it has been committed and > > >> potentially broken things. And it becomes exponentially more difficult to >> find and fix a problem the longer it has been festering in the repo. It’s >> best to keep such problems out in the first place. >> >> With some more work on our CI infrastructure, I think this can be done. >> Maybe even later this year. >> >> Nick >> > >> On Thu, Apr 2, 2015 at 6:02 AM Steve Loughran ste...@hortonworks.com >> > <http://mailto:ste...@hortonworks.com> wrote: >> >> >> > > On 2 Apr 2015, at 06:31, Patrick Wendell <pwend...@gmail.com> wrote: >> > > >> > > Hey Marcelo, >> > > >> > > Great question. Right now, some of the more active developers have an >> > > account that allows them to log into this cluster to inspect logs (we >> > > copy the logs from each run to a node on that cluster). The >> > > infrastructure is maintained by the AMPLab. >> > > >> > > I will put you in touch the someone there who can get you an account. >> > > >> > > This is a short term solution. The longer term solution is to have >> > > these scp'd regularly to an S3 bucket or somewhere people can get >> > > access to them, but that's not ready yet. >> > > >> > > - Patrick >> > > >> > >> >> > >> > >> > ASF Jenkins is always there to play with; committers/PMC members should >> > just need to file a BUILD JIRA to get access. >> > >> > Its main limitation is more cultural than technical: you need to get >> > people to care about intermittent test runs, otherwise you can end up >> with >> > failures that nobody keeps on top of >> > https://builds.apache.org/view/H-L/view/Hadoop/ >> > >> > Someone really needs to own the "keep the builds working" problem -and >> > have the ability to somehow kick others into fixing things. The latter >> is >> > pretty hard cross-organisation >> > >> > >> > >> That would be really helpful to debug build failures. The scalatest >> > >> output isn't all that helpful. >> > >> >> > >> > Potentially an issue with the test runner, rather than the tests >> > themselves. >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> > For additional commands, e-mail: dev-h...@spark.apache.org >> > >> > >> >
