Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

this is still ongoing. the update is running 'chown -R jenkins' on the jenkins root directory, which is a hair under 3T. this might take a while... :\ shane On Fri, Nov 13, 2015 at 6:36 AM, shane knapp wrote: > this is happening now. > > On Thu, Nov 12, 2015 at 12:14 PM,

Re: Seems jenkins is down (or very slow)?

were you hitting any particular URL when you noticed this, or was it generally slow? On Thu, Nov 12, 2015 at 6:21 PM, Yin Huai wrote: > Hi Guys, > > Seems Jenkins is down or very slow? Does anyone else experience it or just > me? > > Thanks, > > Yin

Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

phew. this is finally done... jenkins is up and building. On Fri, Nov 13, 2015 at 7:16 AM, shane knapp wrote: > this is still ongoing. the update is running 'chown -R jenkins' on > the jenkins root directory, which is a hair under 3T. > > this might take a while... :\ >

Re: A proposal for Spark 2.0

We have veered off the topic of Spark 2.0 a little bit here - yes we can talk about RDD vs. DS/DF more but lets refocus on Spark 2.0. I'd like to propose we have one more 1.x release after Spark 1.6. This will allow us to stabilize a few of the new features that were added in 1.6: 1) the

let spark streaming sample come to stop

Hi, I try to run the following 1.4.1 sample by putting a words.txt under localdir bin\run-example org.apache.spark.examples.streaming.HdfsWordCount localdir 2 questions 1. it does not pick up words.txt because it's 'old' I guess - any option to let it picked up? 2. I managed to put a 'new'

Re: SparkPullRequestBuilder coverage

It only runs tests that are impacted by the change. E.g. if you only modify SQL, it won't run the core or streaming tests. On Fri, Nov 13, 2015 at 11:17 AM, Ted Yu wrote: > Hi, > I noticed that SparkPullRequestBuilder completes much faster than maven > Jenkins build. > >

Re: Spark 1.4.2 release and votes conversation?

I actually tried to build a binary for 1.4.2 and wanted to start voting, but there was an issue with the release script that failed the jenkins job. Would be great to kick off a 1.4.2 release. On Fri, Nov 13, 2015 at 1:00 PM, Andrew Lee wrote: > Hi All, > > > I'm wondering

Incubator Proposal for Spark-Kernel

Hello, I wanted to make known a new Apache Incubator proposal for "Spark-Kernel", https://wiki.apache.org/incubator/SparkKernelProposal, which provides applications with a mechanism to interactively and remotely access Spark. The proposal is just starting to be discussed on the general incubator

Re: SparkPullRequestBuilder coverage

Can I assume that for any particular test, if it passes reliably on SparkPullRequestBuilder, it should pass on maven Jenkins ? If so, should flaky test(s) be disabled, strengthened and enabled again ? Cheers On Fri, Nov 13, 2015 at 11:20 AM, Reynold Xin wrote: > It only

Re: pyspark with pypy not work for spark-1.5.1

We already test CPython 2.6, CPython 3.4 and PyPy 2.5, it took more than 30 min to run (without parallelization), I think it should be enough. PyPy 2.2 is too old that we have not enough resource to support that. On Fri, Nov 6, 2015 at 2:27 AM, Chang Ya-Hsuan wrote: > Hi I

Re: SparkPullRequestBuilder coverage

Yes. And those have been happening too. On Fri, Nov 13, 2015 at 12:09 PM, Ted Yu wrote: > Can I assume that for any particular test, if it passes reliably on > SparkPullRequestBuilder, > it should pass on maven Jenkins ? > > If so, should flaky test(s) be disabled,

Spark 1.4.2 release and votes conversation?

Hi All, I'm wondering if Spark 1.4.2 had been voted by any chance or if I have overlooked and we are targeting 1.4.3? By looking at the JIRA https://issues.apache.org/jira/browse/SPARK/fixforversion/12332833/?selectedTab=com.atlassian.jira.jira-projects-plugin:version-summary-panel All

Problem with Breadcast variable not deserialized

Hi, I've a simple Spark job that tries to broadcast an *Externalizable* object across workers. A simple System.out confirm that the object is only serialized with writeExternal and than deserialized without readExternal. Personally I think this is a bug. I'm using the default JavaSerializer

Re: Seems jenkins is down (or very slow)?

It was generally slow. But, after 5 or 10 minutes, it's all good. On Fri, Nov 13, 2015 at 9:16 AM, shane knapp wrote: > were you hitting any particular URL when you noticed this, or was it > generally slow? > > On Thu, Nov 12, 2015 at 6:21 PM, Yin Huai

Re: A proposal for Spark 2.0

Why does stabilization of those two features require a 1.7 release instead of 1.6.1? On Fri, Nov 13, 2015 at 11:40 AM, Kostas Sakellis wrote: > We have veered off the topic of Spark 2.0 a little bit here - yes we can > talk about RDD vs. DS/DF more but lets refocus on Spark

Re: Spark 1.4.2 release and votes conversation?

In the interim, you can just build it off branch-1.4 if you want. On Fri, Nov 13, 2015 at 1:30 PM, Reynold Xin wrote: > I actually tried to build a binary for 1.4.2 and wanted to start voting, > but there was an issue with the release script that failed the jenkins job. >

Re: [build system] short jenkins downtime tomorrow morning, 11-13-2015 @ 7am PST

this is happening now. On Thu, Nov 12, 2015 at 12:14 PM, shane knapp wrote: > i will admit that it does seem like a bad idea to poke jenkins on > friday the 13th, but there's a release that fixes a lot of security > issues: > >