spark git commit: [SPARK-6299][CORE] ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL

2015-03-17 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 47cce984e - 5c16ced1e [SPARK-6299][CORE] ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL ``` case class ClassA(value: String) val rdd = sc.parallelize(List((k1, ClassA(v1)), (k1,

spark git commit: [SQL][docs][minor] Fixed sample code in SQLContext scaladoc

2015-03-17 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 5c16ced1e - 426816b5c [SQL][docs][minor] Fixed sample code in SQLContext scaladoc Error in the code sample of the `implicits` object in `SQLContext`. Author: Lomig Mégard lomig.meg...@gmail.com Closes #5051 from tarfaa/simple and

spark git commit: [SPARK-6299][CORE] ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL

2015-03-17 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9667b9f9c - f0edeae7f [SPARK-6299][CORE] ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL ``` case class ClassA(value: String) val rdd = sc.parallelize(List((k1, ClassA(v1)), (k1, ClassA(v2)) ))

spark git commit: [SPARK-6357][GraphX] Add unapply in EdgeContext

2015-03-17 Thread rxin
Repository: spark Updated Branches: refs/heads/master 68707225f - b3e6eca81 [SPARK-6357][GraphX] Add unapply in EdgeContext This extractor is mainly used for Graph#aggregateMessages*. Author: Takeshi YAMAMURO linguin@gmail.com Closes #5047 from maropu/AddUnapplyInEdgeContext and

spark git commit: [SPARK-6226][MLLIB] add save/load in PySpark's KMeansModel

2015-03-17 Thread meng
Repository: spark Updated Branches: refs/heads/master d9f3e0168 - c94d06264 [SPARK-6226][MLLIB] add save/load in PySpark's KMeansModel Use `_py2java` and `_java2py` to convert Python model to/from Java model. yinxusen Author: Xiangrui Meng m...@databricks.com Closes #5049 from

spark git commit: [SPARK-6336] LBFGS should document what convergenceTol means

2015-03-17 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.3 ac0e7cc7f - 476c4e117 [SPARK-6336] LBFGS should document what convergenceTol means LBFGS uses convergence tolerance. This value should be written in document as an argument. Author: lewuathe lewua...@me.com Closes #5033 from

spark git commit: [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append.

2015-03-17 Thread lian
Repository: spark Updated Branches: refs/heads/branch-1.3 9d88f0cbd - 3ea38bc3d [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append. https://issues.apache.org/jira/browse/SPARK-6366 Author: Yin Huai yh...@databricks.com Closes

spark git commit: [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append.

2015-03-17 Thread lian
Repository: spark Updated Branches: refs/heads/master 4633a87b8 - dc9c9196d [SPARK-6366][SQL] In Python API, the default save mode for save and saveAsTable should be error instead of append. https://issues.apache.org/jira/browse/SPARK-6366 Author: Yin Huai yh...@databricks.com Closes #5053

spark git commit: [SPARK-6383][SQL]Fixed compiler and errors in Dataframe examples

2015-03-17 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.3 3ea38bc3d - cee6d0877 [SPARK-6383][SQL]Fixed compiler and errors in Dataframe examples Author: Tijo Thomas tijopara...@gmail.com Closes #5068 from tijoparacka/fix_sql_dataframe_example and squashes the following commits: 6953ac1

spark git commit: [SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases

2015-03-17 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a6ee2f794 - 357900311 [SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases We need to handle ambiguous `exprId`s that are produced by new aliases as well as those caused by leaf nodes (`MultiInstanceRelation`).

spark git commit: [SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases

2015-03-17 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.3 cee6d0877 - ba8352c76 [SPARK-6247][SQL] Fix resolution of ambiguous joins caused by new aliases We need to handle ambiguous `exprId`s that are produced by new aliases as well as those caused by leaf nodes (`MultiInstanceRelation`).

spark git commit: [SPARK-5908][SQL] Resolve UdtfsAlias when only single Alias is used

2015-03-17 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master a012e0863 - 5c80643d1 [SPARK-5908][SQL] Resolve UdtfsAlias when only single Alias is used `ResolveUdtfsAlias` in `hiveUdfs` only considers the `HiveGenericUdtf` with multiple alias. When only single alias is used with `HiveGenericUdtf`,

spark git commit: [SPARK-5404] [SQL] Update the default statistic number

2015-03-17 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 5c80643d1 - 78cb08a5d [SPARK-5404] [SQL] Update the default statistic number By default, the statistic for logical plan with multiple children is quite aggressive, and those statistic are quite critical for the join optimization, hence

spark git commit: [SPARK-5651][SQL] Add input64 in blacklist and add test suit for create table within backticks

2015-03-17 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 78cb08a5d - a6ee2f794 [SPARK-5651][SQL] Add input64 in blacklist and add test suit for create table within backticks Now spark version is only support ```create table table_in_database_creation.test1 as select * from src limit 1;``` in

spark git commit: [SPARK-6330] [SQL] Add a test case for SPARK-6330

2015-03-17 Thread lian
Repository: spark Updated Branches: refs/heads/branch-1.3 476c4e117 - 9d88f0cbd [SPARK-6330] [SQL] Add a test case for SPARK-6330 When getting file statuses, create file system from each path instead of a single one from hadoop configuration. Author: Pei-Lun Lee pl...@appier.com Closes

[2/2] spark git commit: [SPARK-4011] tighten the visibility of the members in Master/Worker class

2015-03-17 Thread srowen
[SPARK-4011] tighten the visibility of the members in Master/Worker class https://issues.apache.org/jira/browse/SPARK-4011 Currently, most of the members in Master/Worker are with public accessibility. We might wish to tighten the accessibility of them a bit more discussion is here:

[1/2] spark git commit: [SPARK-4011] tighten the visibility of the members in Master/Worker class

2015-03-17 Thread srowen
Repository: spark Updated Branches: refs/heads/master b2d8c0222 - 25f35806e http://git-wip-us.apache.org/repos/asf/spark/blob/25f35806/core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala -- diff --git

spark git commit: [SPARK-6269] [CORE] Use ScalaRunTime's array methods instead of java.lang.reflect.Array in size estimation

2015-03-17 Thread srowen
Repository: spark Updated Branches: refs/heads/master 25f35806e - 005d1c5f2 [SPARK-6269] [CORE] Use ScalaRunTime's array methods instead of java.lang.reflect.Array in size estimation This patch switches the usage of java.lang.reflect.Array in Size estimation to using scala's RunTime

spark git commit: [SPARK-6331] Load new master URL if present when recovering streaming context from checkpoint

2015-03-17 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.3 426816b5c - 95f8d1c51 [SPARK-6331] Load new master URL if present when recovering streaming context from checkpoint In streaming driver recovery, when the SparkConf is reconstructed based on the checkpointed configuration, it

spark git commit: [SPARK-6331] Load new master URL if present when recovering streaming context from checkpoint

2015-03-17 Thread tdas
Repository: spark Updated Branches: refs/heads/master e26db9be4 - c928796ad [SPARK-6331] Load new master URL if present when recovering streaming context from checkpoint In streaming driver recovery, when the SparkConf is reconstructed based on the checkpointed configuration, it recovers

spark git commit: [SPARK-6365] jetty-security also needed for SPARK_PREPEND_CLASSES to work

2015-03-17 Thread irashid
Repository: spark Updated Branches: refs/heads/master c928796ad - e9f22c612 [SPARK-6365] jetty-security also needed for SPARK_PREPEND_CLASSES to work https://issues.apache.org/jira/browse/SPARK-6365 thanks vanzin for helping me figure this out Author: Imran Rashid iras...@cloudera.com

spark git commit: [SPARK-3266] Use intermediate abstract classes to fix type erasure issues in Java APIs

2015-03-17 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master e9f22c612 - 0f673c21f [SPARK-3266] Use intermediate abstract classes to fix type erasure issues in Java APIs This PR addresses a Scala compiler bug ([SI-8905](https://issues.scala-lang.org/browse/SI-8905)) that was breaking some of the

spark git commit: [SPARK-3266] Use intermediate abstract classes to fix type erasure issues in Java APIs

2015-03-17 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 95f8d1c51 - 29e39e178 [SPARK-3266] Use intermediate abstract classes to fix type erasure issues in Java APIs This PR addresses a Scala compiler bug ([SI-8905](https://issues.scala-lang.org/browse/SI-8905)) that was breaking some of

spark git commit: [SPARK-6313] Add config option to disable file locks/fetchFile cache to ...

2015-03-17 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.3 29e39e178 - febb12308 [SPARK-6313] Add config option to disable file locks/fetchFile cache to ... ...support NFS mounts. This is a work around for now with the goal to find a more permanent solution.

spark git commit: [SPARK-6365] jetty-security also needed for SPARK_PREPEND_CLASSES to work

2015-03-17 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-1.3 febb12308 - ac0e7cc7f [SPARK-6365] jetty-security also needed for SPARK_PREPEND_CLASSES to work https://issues.apache.org/jira/browse/SPARK-6365 thanks vanzin for helping me figure this out Author: Imran Rashid iras...@cloudera.com