Re: Adding/Using More Resolution Types on JIRA

2015-05-15 Thread Patrick Wendell
If there is no further feedback on this I will ask ASF Infra to add the new fields "Out of Scope" and "Inactive". - Patrick On Tue, May 12, 2015 at 9:02 AM, Nicholas Chammas wrote: > I tend to find that any large project has a lot of walking dead JIRAs, and > pretending they are simply Open caus

Re: Recent Spark test failures

2015-05-15 Thread Patrick Wendell
Sorry premature send: The PR builder currently builds against Hadoop 2.3 https://github.com/apache/spark/blob/master/dev/run-tests#L54 We can set this to whatever we want. 2.2 might make sense since it's the default in our published artifacts. - Patrick On Fri, May 15, 2015 at 11:53 AM, Patrick

Re: Recent Spark test failures

2015-05-15 Thread Patrick Wendell
The PR builder currently builds against Hadoop 2.3. - Patrick On Fri, May 15, 2015 at 11:40 AM, Marcelo Vanzin wrote: > Funny thing, since I asked this question in a PR a few minutes ago... > > Ignoring the rotation suggestion for a second, can the PR builder at least > cover hadoop 2.2? That's

Re: Recent Spark test failures

2015-05-15 Thread Marcelo Vanzin
Funny thing, since I asked this question in a PR a few minutes ago... Ignoring the rotation suggestion for a second, can the PR builder at least cover hadoop 2.2? That's the actual version used to create the official Spark artifacts for maven, and the oldest version Spark supports for YARN.. Kind

Re: Recent Spark test failures

2015-05-15 Thread Ted Yu
bq. would be prohibitive to build all configurations for every push Agreed. Can PR builder rotate testing against hadoop 2.3, 2.4, 2.6 and 2.7 (each test run still uses one hadoop profile) ? This way we would have some coverage for each of the major hadoop releases. Cheers On Fri, May 15, 2015

Re: Tentative due dates for Spark 1.3.2 release

2015-05-15 Thread Patrick Wendell
Hi Niranda, Maintenance releases are not done on a predetermined schedule but instead according to which fixes show up and their severity. Since we just did a 1.3.1 release I'm not sure I see 1.3.2 on the immediate horizon. However, the maintenance releases are simply builds at the head of the re

Re: Change for submitting to yarn in 1.3.1

2015-05-15 Thread Marcelo Vanzin
Hi Chester, Writing a design / requirements doc sounds great. One comment though: On Thu, May 14, 2015 at 11:18 PM, Chester At Work wrote: >For #5 yes, it's about the command line args. These are args are > the input for the spark jobs. Seems a bit too much to create a file just to > sp

Re: Recent Spark test failures

2015-05-15 Thread Sean Owen
You all are looking only at the pull request builder. It just does one build to sanity-check a pull request, since that already takes 2 hours and would be prohibitive to build all configurations for every push. There is a different set of Jenkins jobs that periodically tests master against a lot mo

Re: Recent Spark test failures

2015-05-15 Thread Frederick R Reiss
The PR builder seems to be building against Hadoop 2.3. In the log for the most recent successful build ( https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32805/consoleFull ) I see: = Building Spark

Re: Recent Spark test failures

2015-05-15 Thread Ted Yu
From https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32831/consoleFull : [info] Building Spark with these arguments: -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -Pkinesis-asl -Phive -Phive-thriftserver Should PR builder cover hadoop 2.4 as well ? Thanks On Fri, May 15, 2015

Re: Recent Spark test failures

2015-05-15 Thread Ted Yu
Jenkins build against hadoop 2.4 has been unstable recently: https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-with-YARN/HADOOP_PROFILE=hadoop-2.4,label=centos/ I haven't found the test which hung / failed in recent Jenkins builds. But PR builder has several green builds la

Tentative due dates for Spark 1.3.2 release

2015-05-15 Thread Niranda Perera
Hi, May I know the tentative release dates for spark 1.3.2? rgds -- Niranda