Re: Tungsten's Vectorized Execution

[Reynold Xin]

Yijie,

As Davies said, it will take us a while to get to vectorized execution.
However, before that, we are going to refactor code generation to push it
into each expression: https://issues.apache.org/jira/browse/SPARK-7813

Once this one is in (probably in the next 2 or 3 weeks), there will be lots
of expressions to create code-gen versions, and it'd be great to get as
much help as possible from the community.




On Thu, May 21, 2015 at 1:59 PM, Davies Liu  wrote:

> We have not start to prototype the vectorized one yet, will evaluated
> in 1.5 and may targeted for 1.6.
>
> We're glad to hear some feedback/suggestions/comments from your side!
>
> On Thu, May 21, 2015 at 9:37 AM, Yijie Shen 
> wrote:
> > Hi all,
> >
> > I’ve seen the Blog of Project Tungsten here, it sounds awesome to me!
> >
> > I’ve also noticed there is a plan to change the code generation from
> > record-at-a-time evaluation to a vectorized one, which interests me most.
> >
> > What’s the status of vectorized evaluation?  Is this an inner effort of
> > Databricks or welcome to be involved?
> >
> > Since I’ve done similar stuffs on Spark SQL, I would like to get
> involved if
> > that’s possible.
> >
> >
> > Yours,
> >
> > Yijie
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
>

Available Functions in SparkR

[Eskilson,Aleksander]

I’ve built Spark 1.4.0 for Hadoop 2.6 in a CDH5.4 and am testing SparkR. I’ve 
loaded up SparkR using the executable in /bin. The library import 
library(SparkR) seems to no longer import some of the same functions as it did 
for SparkR before the merge, e.g. textFile, lapply, etc. but it does include 
sparkR.init, take, and other original functions. How is it planned to access 
the full set of functions in the repl with the coming version of SparkR?

Thanks,
Alek Eskilson

CONFIDENTIALITY NOTICE This message and any included attachments are from 
Cerner Corporation and are intended only for the addressee. The information 
contained in this message is confidential and may constitute inside or 
non-public information under international, federal, or state securities laws. 
Unauthorized forwarding, printing, copying, distribution, or use of such 
information is strictly prohibited and may be unlawful. If you are not the 
addressee, please promptly delete this message and notify the sender of the 
delivery error by e-mail or you may call Cerner's corporate offices in Kansas 
City, Missouri, U.S.A at (+1) (816)221-1024.

Unable to build from assembly

[Manoj Kumar]

Hello,

I updated my master from upstream recently, and on running

build/sbt assembly

it gives me this error

[error]
/home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:106:
error: MyJavaLogisticRegression is not abstract and does not override
abstract method setDefault(ParamPair...) in Params
[error] class MyJavaLogisticRegression
[error] ^
[error]
/home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:168:
error: MyJavaLogisticRegressionModel is not abstract and does not override
abstract method setDefault(ParamPair...) in Params
[error] class MyJavaLogisticRegressionModel
[error] ^
[error] 2 errors
[error] (examples/compile:compile) javac returned nonzero exit code

It was working fine before this.

Could someone please guide me on what could be wrong?



-- 
Godspeed,
Manoj Kumar,
http://manojbits.wordpress.com

http://github.com/MechCoder

Spark Bug: Counting twice with different results

[Niklas Wilcke]

Hi,

I have recognized a strange behavior of spark core in combination with
mllib. Running my pipeline results in a RDD.
Calling count() on this RDD results in 160055.
Calling count() directly afterwards results in 160044 and so on.
The RDD seems to be unstable.

How can that be? Do you maybe have an explanation or guidance for
further investigation? I'm investigating for 3 days now and can't
isolate the bug.

Unfortunately I can't provide a minimal working example only using
Spark. At the moment I try to reproduce the bug with only using the
Spark API to hand it over to someone more experienced.

I recognized this behavior while investigating SPARK-5480. Trying to
build a graph and calculate the transitive closure on such a unstable
RDD results in a IndexOutOfBoundsException -1.

My first suspicion is that
org.apache.spark.mllib.rdd.RDDFunctions.sliding causes the problems.
Replacing my algorithm which uses the sliding window solves the problem.

The bug only occurs on large data sets. On small ones the pipeline works
fine. That makes it hard to investigate because every run takes several
minutes. Also generated data does not produce the bug.

I didn't open a Jira ticket yet because I can't tell how to reproduce it.

I'm running Spark 1.3.1 in standalone mode with HDFS on a 10 node cluster.

Thanks for your advise,
Niklas


-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: Unable to build from assembly

[Ted Yu]

What version of Java do you use ?

Can you run this command first ?
build/sbt clean

BTW please see [SPARK-7498] [MLLIB] add varargs back to setDefault

Cheers

On Fri, May 22, 2015 at 7:34 AM, Manoj Kumar  wrote:

> Hello,
>
> I updated my master from upstream recently, and on running
>
> build/sbt assembly
>
> it gives me this error
>
> [error]
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:106:
> error: MyJavaLogisticRegression is not abstract and does not override
> abstract method setDefault(ParamPair...) in Params
> [error] class MyJavaLogisticRegression
> [error] ^
> [error]
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:168:
> error: MyJavaLogisticRegressionModel is not abstract and does not override
> abstract method setDefault(ParamPair...) in Params
> [error] class MyJavaLogisticRegressionModel
> [error] ^
> [error] 2 errors
> [error] (examples/compile:compile) javac returned nonzero exit code
>
> It was working fine before this.
>
> Could someone please guide me on what could be wrong?
>
>
>
> --
> Godspeed,
> Manoj Kumar,
> http://manojbits.wordpress.com
> 
> http://github.com/MechCoder
>

Re: Unable to build from assembly

[Edoardo Vacchi]

confirming. master has been broken in the morning; currently it should
be ok, though

On Fri, May 22, 2015 at 4:34 PM, Manoj Kumar
 wrote:
> Hello,
>
> I updated my master from upstream recently, and on running
>
> build/sbt assembly
>
> it gives me this error
>
> [error]
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:106:
> error: MyJavaLogisticRegression is not abstract and does not override
> abstract method setDefault(ParamPair...) in Params
> [error] class MyJavaLogisticRegression
> [error] ^
> [error]
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:168:
> error: MyJavaLogisticRegressionModel is not abstract and does not override
> abstract method setDefault(ParamPair...) in Params
> [error] class MyJavaLogisticRegressionModel
> [error] ^
> [error] 2 errors
> [error] (examples/compile:compile) javac returned nonzero exit code
>
> It was working fine before this.
>
> Could someone please guide me on what could be wrong?
>
>
>
> --
> Godspeed,
> Manoj Kumar,
> http://manojbits.wordpress.com
> http://github.com/MechCoder

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

UDTs and StringType upgrade issue for Spark 1.4.0

[Justin Uang]

We ran into an issue regarding Strings in UDTs when upgrading to Spark
1.4.0-rc. I understand that it's a non-public APIs, so it's expected, but I
just wanted to bring it up for awareness and so we can maybe change the
release notes to mention them =)

Our UDT was serializing to a StringType, but now strings are represented
internally as UTF8String, so we had to change our UDT to use
UTF8String.apply() and UTF8String.toString() to convert back to String.

Re: Change for submitting to yarn in 1.3.1

[Marcelo Vanzin]

Hi Kevin,

One thing that might help you in the meantime, while we work on a better
interface for all this...

On Thu, May 21, 2015 at 5:21 PM, Kevin Markey 
wrote:

> Making *yarn.Client* private has prevented us from moving from Spark
> 1.0.x to Spark 1.2 or 1.3 despite many alluring new features.
>

Since you're not afraid to use private APIs, and to avoid using ugly
reflection hacks, you could abuse the fact that private things in Scala are
not really private most of the time. For example (trimmed to show just
stuff that might be interesting to you):

# javap -classpath
/opt/cloudera/parcels/CDH/jars/spark-assembly-1.3.0-cdh5.4.0-hadoop2.6.0-cdh5.4.0.jar
org.apache.spark.deploy.yarn.Client
Compiled from "Client.scala"
public class org.apache.spark.deploy.yarn.Client implements
org.apache.spark.Logging {
  ...
  public org.apache.hadoop.yarn.client.api.YarnClient
org$apache$spark$deploy$yarn$Client$$yarnClient();
  public void run();
  public
org.apache.spark.deploy.yarn.Client(org.apache.spark.deploy.yarn.ClientArguments,
org.apache.hadoop.conf.Configuration, org.apache.spark.SparkConf);
  public
org.apache.spark.deploy.yarn.Client(org.apache.spark.deploy.yarn.ClientArguments,
org.apache.spark.SparkConf);
  public
org.apache.spark.deploy.yarn.Client(org.apache.spark.deploy.yarn.ClientArguments);
}

So it should be easy to write a small Java wrapper around this. No less
hacky than relying on the "private-but-public" code of before.

-- 
Marcelo

Re: Spark Bug: Counting twice with different results

[Sean Owen]

This is expected for example if your RDD is the result of random
sampling, or if the underlying source is not consistent. You haven't
shown any code.

On Fri, May 22, 2015 at 3:34 PM, Niklas Wilcke
<1wil...@informatik.uni-hamburg.de> wrote:
> Hi,
>
> I have recognized a strange behavior of spark core in combination with
> mllib. Running my pipeline results in a RDD.
> Calling count() on this RDD results in 160055.
> Calling count() directly afterwards results in 160044 and so on.
> The RDD seems to be unstable.
>
> How can that be? Do you maybe have an explanation or guidance for
> further investigation? I'm investigating for 3 days now and can't
> isolate the bug.
>
> Unfortunately I can't provide a minimal working example only using
> Spark. At the moment I try to reproduce the bug with only using the
> Spark API to hand it over to someone more experienced.
>
> I recognized this behavior while investigating SPARK-5480. Trying to
> build a graph and calculate the transitive closure on such a unstable
> RDD results in a IndexOutOfBoundsException -1.
>
> My first suspicion is that
> org.apache.spark.mllib.rdd.RDDFunctions.sliding causes the problems.
> Replacing my algorithm which uses the sliding window solves the problem.
>
> The bug only occurs on large data sets. On small ones the pipeline works
> fine. That makes it hard to investigate because every run takes several
> minutes. Also generated data does not produce the bug.
>
> I didn't open a Jira ticket yet because I can't tell how to reproduce it.
>
> I'm running Spark 1.3.1 in standalone mode with HDFS on a 10 node cluster.
>
> Thanks for your advise,
> Niklas
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: Unable to build from assembly

[Manoj Kumar]

A clean build worked.

Thanks everyone for the help!

On Fri, May 22, 2015 at 8:42 PM, Edoardo Vacchi 
wrote:

> confirming. master has been broken in the morning; currently it should
> be ok, though
>
> On Fri, May 22, 2015 at 4:34 PM, Manoj Kumar
>  wrote:
> > Hello,
> >
> > I updated my master from upstream recently, and on running
> >
> > build/sbt assembly
> >
> > it gives me this error
> >
> > [error]
> >
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:106:
> > error: MyJavaLogisticRegression is not abstract and does not override
> > abstract method setDefault(ParamPair...) in Params
> > [error] class MyJavaLogisticRegression
> > [error] ^
> > [error]
> >
> /home/manoj/spark/examples/src/main/java/org/apache/spark/examples/ml/JavaDeveloperApiExample.java:168:
> > error: MyJavaLogisticRegressionModel is not abstract and does not
> override
> > abstract method setDefault(ParamPair...) in Params
> > [error] class MyJavaLogisticRegressionModel
> > [error] ^
> > [error] 2 errors
> > [error] (examples/compile:compile) javac returned nonzero exit code
> >
> > It was working fine before this.
> >
> > Could someone please guide me on what could be wrong?
> >
> >
> >
> > --
> > Godspeed,
> > Manoj Kumar,
> > http://manojbits.wordpress.com
> > http://github.com/MechCoder
>



-- 
Godspeed,
Manoj Kumar,
http://manojbits.wordpress.com

http://github.com/MechCoder

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[Justin Uang]

I'm working on one of the Palantir teams using Spark, and here is our
feedback:

We have encountered three issues when upgrading to spark 1.4.0. I'm not
sure they qualify as a -1, as they come from using non-public APIs and
multiple spark contexts for the purposes of testing, but I do want to bring
them up for awareness =)

   1. Our UDT was serializing to a StringType, but now strings are
   represented internally as UTF8String, so we had to change our UDT to use
   UTF8String.apply() and UTF8String.toString() to convert back to String.
   2. createDataFrame when using UDTs used to accept things in the
   serialized catalyst form. Now, they're supposed to be in the UDT java class
   form (I think this change would've affected us in 1.3.1 already, since we
   were in 1.3.0)
   3. derby database lifecycle management issue with HiveContext. We have
   been using a SparkContextResource JUnit Rule that we wrote, and it sets up
   then tears down a SparkContext and HiveContext between unit test runs
   within the same process (possibly the same thread as well). Multiple
   contexts are not being used at once. It used to work in 1.3.0, but now when
   we try to create the HiveContext for the second unit test, then it
   complains with the following exception. I have a feeling it might have
   something to do with the Hive object being thread local, and us not
   explicitly closing the HiveContext and everything it holds. The full stack
   trace is here: https://gist.github.com/justinuang/0403d49cdeedf91727cd

Caused by: java.sql.SQLException: Failed to start database
'metastore_db' with class loader
org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@5dea2446,
see the next exception for details.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)


On Wed, May 20, 2015 at 10:35 AM Imran Rashid  wrote:

> -1
>
> discovered I accidentally removed master & worker json endpoints, will
> restore
> https://issues.apache.org/jira/browse/SPARK-7760
>
> On Tue, May 19, 2015 at 11:10 AM, Patrick Wendell 
> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.4.0!
>>
>> The tag to be voted on is v1.4.0-rc1 (commit 777a081):
>>
>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-1.4.0-rc1/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1092/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/
>>
>> Please vote on releasing this package as Apache Spark 1.4.0!
>>
>> The vote is open until Friday, May 22, at 17:03 UTC and passes
>> if a majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.4.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see
>> http://spark.apache.org/
>>
>> == How can I help test this release? ==
>> If you are a Spark user, you can help us test this release by
>> taking a Spark 1.3 workload and running on this release candidate,
>> then reporting any regressions.
>>
>> == What justifies a -1 vote for this release? ==
>> This vote is happening towards the end of the 1.4 QA period,
>> so -1 votes should only occur for significant regressions from 1.3.1.
>> Bugs already present in 1.3.X, minor regressions, or bugs related
>> to new features will not block this release.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>> For additional commands, e-mail: dev-h...@spark.apache.org
>>
>>
>

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[Michael Armbrust]

Thanks for the feedback.  As you stated UDTs are explicitly not a public
api as we knew we were going to be making breaking changes to them.  We
hope to stabilize / open them up in future releases.  Regarding the Hive
issue, have you tried using TestHive instead.  This is what we use for
testing and it takes care of creating temporary directories for all
storage.  It also has a reset() function that you can call in-between
tests.  If this doesn't work for you, maybe open a JIRA and we can discuss
more there.

On Fri, May 22, 2015 at 12:56 PM, Justin Uang  wrote:

> I'm working on one of the Palantir teams using Spark, and here is our
> feedback:
>
> We have encountered three issues when upgrading to spark 1.4.0. I'm not
> sure they qualify as a -1, as they come from using non-public APIs and
> multiple spark contexts for the purposes of testing, but I do want to bring
> them up for awareness =)
>
>1. Our UDT was serializing to a StringType, but now strings are
>represented internally as UTF8String, so we had to change our UDT to use
>UTF8String.apply() and UTF8String.toString() to convert back to String.
>2. createDataFrame when using UDTs used to accept things in the
>serialized catalyst form. Now, they're supposed to be in the UDT java class
>form (I think this change would've affected us in 1.3.1 already, since we
>were in 1.3.0)
>3. derby database lifecycle management issue with HiveContext. We have
>been using a SparkContextResource JUnit Rule that we wrote, and it sets up
>then tears down a SparkContext and HiveContext between unit test runs
>within the same process (possibly the same thread as well). Multiple
>contexts are not being used at once. It used to work in 1.3.0, but now when
>we try to create the HiveContext for the second unit test, then it
>complains with the following exception. I have a feeling it might have
>something to do with the Hive object being thread local, and us not
>explicitly closing the HiveContext and everything it holds. The full stack
>trace is here: https://gist.github.com/justinuang/0403d49cdeedf91727cd
>
> Caused by: java.sql.SQLException: Failed to start database 'metastore_db' 
> with class loader 
> org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@5dea2446, see 
> the next exception for details.
>   at 
> org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
>
>
> On Wed, May 20, 2015 at 10:35 AM Imran Rashid 
> wrote:
>
>> -1
>>
>> discovered I accidentally removed master & worker json endpoints, will
>> restore
>> https://issues.apache.org/jira/browse/SPARK-7760
>>
>> On Tue, May 19, 2015 at 11:10 AM, Patrick Wendell 
>> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 1.4.0!
>>>
>>> The tag to be voted on is v1.4.0-rc1 (commit 777a081):
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-1.4.0-rc1/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1092/
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/
>>>
>>> Please vote on releasing this package as Apache Spark 1.4.0!
>>>
>>> The vote is open until Friday, May 22, at 17:03 UTC and passes
>>> if a majority of at least 3 +1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.4.0
>>> [ ] -1 Do not release this package because ...
>>>
>>> To learn more about Apache Spark, please see
>>> http://spark.apache.org/
>>>
>>> == How can I help test this release? ==
>>> If you are a Spark user, you can help us test this release by
>>> taking a Spark 1.3 workload and running on this release candidate,
>>> then reporting any regressions.
>>>
>>> == What justifies a -1 vote for this release? ==
>>> This vote is happening towards the end of the 1.4 QA period,
>>> so -1 votes should only occur for significant regressions from 1.3.1.
>>> Bugs already present in 1.3.X, minor regressions, or bugs related
>>> to new features will not block this release.
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>>> For additional commands, e-mail: dev-h...@spark.apache.org
>>>
>>>
>>

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[Andrew Psaltis]

All,
Should all the docs work from
http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/ ? If so the R API
docs 404.


>>> On Tue, May 19, 2015 at 11:10 AM, Patrick Wendell 
>>> wrote:
>>>
 Please vote on releasing the following candidate as Apache Spark
 version 1.4.0!

 The tag to be voted on is v1.4.0-rc1 (commit 777a081):

 https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e

 The release files, including signatures, digests, etc. can be found at:
 http://people.apache.org/~pwendell/spark-1.4.0-rc1/

 Release artifacts are signed with the following key:
 https://people.apache.org/keys/committer/pwendell.asc

 The staging repository for this release can be found at:
 https://repository.apache.org/content/repositories/orgapachespark-1092/

 The documentation corresponding to this release can be found at:
 http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/

 Please vote on releasing this package as Apache Spark 1.4.0!

 The vote is open until Friday, May 22, at 17:03 UTC and passes
 if a majority of at least 3 +1 PMC votes are cast.

 [ ] +1 Release this package as Apache Spark 1.4.0
 [ ] -1 Do not release this package because ...

 To learn more about Apache Spark, please see
 http://spark.apache.org/

 == How can I help test this release? ==
 If you are a Spark user, you can help us test this release by
 taking a Spark 1.3 workload and running on this release candidate,
 then reporting any regressions.

 == What justifies a -1 vote for this release? ==
 This vote is happening towards the end of the 1.4 QA period,
 so -1 votes should only occur for significant regressions from 1.3.1.
 Bugs already present in 1.3.X, minor regressions, or bugs related
 to new features will not block this release.

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org


>>>
>

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[Shivaram Venkataraman]

Thanks for catching this. I'll check with Patrick to see why the R API docs
are not getting included.

On Fri, May 22, 2015 at 2:44 PM, Andrew Psaltis 
wrote:

> All,
> Should all the docs work from
> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/ ? If so the R
> API docs 404.
>
>
 On Tue, May 19, 2015 at 11:10 AM, Patrick Wendell 
 wrote:

> Please vote on releasing the following candidate as Apache Spark
> version 1.4.0!
>
> The tag to be voted on is v1.4.0-rc1 (commit 777a081):
>
> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-1.4.0-rc1/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1092/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/
>
> Please vote on releasing this package as Apache Spark 1.4.0!
>
> The vote is open until Friday, May 22, at 17:03 UTC and passes
> if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.apache.org/
>
> == How can I help test this release? ==
> If you are a Spark user, you can help us test this release by
> taking a Spark 1.3 workload and running on this release candidate,
> then reporting any regressions.
>
> == What justifies a -1 vote for this release? ==
> This vote is happening towards the end of the 1.4 QA period,
> so -1 votes should only occur for significant regressions from 1.3.1.
> Bugs already present in 1.3.X, minor regressions, or bugs related
> to new features will not block this release.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
>

>>
>

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[Patrick Wendell]

Thanks Andrew, the doc issue should be fixed in RC2 (if not, please
chine in!). R was missing in the build envirionment.

- Patrick

On Fri, May 22, 2015 at 3:33 PM, Shivaram Venkataraman
 wrote:
> Thanks for catching this. I'll check with Patrick to see why the R API docs
> are not getting included.
>
> On Fri, May 22, 2015 at 2:44 PM, Andrew Psaltis 
> wrote:
>>
>> All,
>> Should all the docs work from
>> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/ ? If so the R API
>> docs 404.
>>
>
> On Tue, May 19, 2015 at 11:10 AM, Patrick Wendell 
> wrote:
>>
>> Please vote on releasing the following candidate as Apache Spark
>> version 1.4.0!
>>
>> The tag to be voted on is v1.4.0-rc1 (commit 777a081):
>>
>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e
>>
>> The release files, including signatures, digests, etc. can be found
>> at:
>> http://people.apache.org/~pwendell/spark-1.4.0-rc1/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>>
>> https://repository.apache.org/content/repositories/orgapachespark-1092/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/
>>
>> Please vote on releasing this package as Apache Spark 1.4.0!
>>
>> The vote is open until Friday, May 22, at 17:03 UTC and passes
>> if a majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.4.0
>> [ ] -1 Do not release this package because ...
>>
>> To learn more about Apache Spark, please see
>> http://spark.apache.org/
>>
>> == How can I help test this release? ==
>> If you are a Spark user, you can help us test this release by
>> taking a Spark 1.3 workload and running on this release candidate,
>> then reporting any regressions.
>>
>> == What justifies a -1 vote for this release? ==
>> This vote is happening towards the end of the 1.4 QA period,
>> so -1 votes should only occur for significant regressions from 1.3.1.
>> Bugs already present in 1.3.X, minor regressions, or bugs related
>> to new features will not block this release.
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
>> For additional commands, e-mail: dev-h...@spark.apache.org
>>
>
>>>
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [VOTE] Release Apache Spark 1.4.0 (RC1)

[jameszhouyi]

We came across a Spark SQL issue
(https://issues.apache.org/jira/browse/SPARK-7119) that cause query to fail.
I not sure that if vote -1 to this RC1.



--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-4-0-RC1-tp12321p12403.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org