spark git commit: [SPARK-4925] Publish Spark SQL hive-thriftserver maven artifact

2015-04-27 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 82bb7fd41 -> 998aac21f [SPARK-4925] Publish Spark SQL hive-thriftserver maven artifact turned on hive-thriftserver profile in release script Author: Misha Chernetsov Closes #5429 from chernetsov/master and squashes the following commits:

spark git commit: [SPARK-4925] Publish Spark SQL hive-thriftserver maven artifact

2015-04-27 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.3 c4470b93f -> 81de30ae5 [SPARK-4925] Publish Spark SQL hive-thriftserver maven artifact turned on hive-thriftserver profile in release script Author: Misha Chernetsov Closes #5429 from chernetsov/master and squashes the following comm

spark git commit: [SPARK-6856] [R] Make RDD information more useful in SparkR

2015-04-27 Thread shivaram
Repository: spark Updated Branches: refs/heads/master 998aac21f -> 7078f6028 [SPARK-6856] [R] Make RDD information more useful in SparkR Author: Jeff Harrison Closes #5667 from His-name-is-Joof/joofspark and squashes the following commits: f8814a6 [Jeff Harrison] newline added after RDD sho

spark git commit: SPARK-7107 Add parameter for zookeeper.znode.parent to hbase_inputformat...

2015-04-27 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7078f6028 -> ef82bddc1 SPARK-7107 Add parameter for zookeeper.znode.parent to hbase_inputformat... py Author: tedyu Closes #5673 from tedyu/master and squashes the following commits: ab7c72b [tedyu] SPARK-7107 Adjust indentation to

spark git commit: [SPARK-6991] [SPARKR] Adds support for zipPartitions.

2015-04-27 Thread shivaram
Repository: spark Updated Branches: refs/heads/master ef82bddc1 -> ca9f4ebb8 [SPARK-6991] [SPARKR] Adds support for zipPartitions. Author: hlin09 Closes #5568 from hlin09/zipPartitions and squashes the following commits: 12c08a5 [hlin09] Fix comments d2d32db [hlin09] Merge branch 'master' i

spark git commit: [SPARK-7103] Fix crash with SparkContext.union when RDD has no partitioner

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 81de30ae5 -> d13080aa2 [SPARK-7103] Fix crash with SparkContext.union when RDD has no partitioner Added a check to the SparkContext.union method to check that a partitioner is defined on all RDDs when instantiating a PartitionerAwareUn

spark git commit: [SPARK-7103] Fix crash with SparkContext.union when RDD has no partitioner

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master ca9f4ebb8 -> b9de9e040 [SPARK-7103] Fix crash with SparkContext.union when RDD has no partitioner Added a check to the SparkContext.union method to check that a partitioner is defined on all RDDs when instantiating a PartitionerAwareUnionR

spark git commit: [SPARK-6738] [CORE] Improve estimate the size of a large array

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master b9de9e040 -> 8e1c00dbf [SPARK-6738] [CORE] Improve estimate the size of a large array Currently, SizeEstimator.visitArray is not correct in the follow case, ``` array size > 200, elem has the share object ``` when I add a debug log in Size

spark git commit: [SPARK-3090] [CORE] Stop SparkContext if user forgets to.

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8e1c00dbf -> 5d45e1f60 [SPARK-3090] [CORE] Stop SparkContext if user forgets to. Set up a shutdown hook to try to stop the Spark context in case the user forgets to do it. The main effect is that any open logs files are flushed and closed,

spark git commit: [SPARK-7145] [CORE] commons-lang (2.x) classes used instead of commons-lang3 (3.x); commons-io used without dependency

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master 5d45e1f60 -> ab5adb7a9 [SPARK-7145] [CORE] commons-lang (2.x) classes used instead of commons-lang3 (3.x); commons-io used without dependency Remove use of commons-lang in favor of commons-lang3 classes; remove commons-io use in favor of

spark git commit: [SPARK-7162] [YARN] Launcher error in yarn-client

2015-04-27 Thread srowen
Repository: spark Updated Branches: refs/heads/master ab5adb7a9 -> 62888a4de [SPARK-7162] [YARN] Launcher error in yarn-client jira: https://issues.apache.org/jira/browse/SPARK-7162 Author: GuoQiang Li Closes #5716 from witgo/SPARK-7162 and squashes the following commits: b64564c [GuoQiang

spark git commit: [SPARK-7090] [MLLIB] Introduce LDAOptimizer to LDA to further improve extensibility

2015-04-27 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 62888a4de -> 4d9e560b5 [SPARK-7090] [MLLIB] Introduce LDAOptimizer to LDA to further improve extensibility jira: https://issues.apache.org/jira/browse/SPARK-7090 LDA was implemented with extensibility in mind. And with the development of

spark git commit: [SPARK-7174][Core] Move calling `TaskScheduler.executorHeartbeatReceived` to another thread

2015-04-27 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4d9e560b5 -> 874a2ca93 [SPARK-7174][Core] Move calling `TaskScheduler.executorHeartbeatReceived` to another thread `HeartbeatReceiver` will call `TaskScheduler.executorHeartbeatReceived`, which is a blocking operation because `TaskSchedul

spark git commit: [SPARK-6829] Added math functions for DataFrames

2015-04-27 Thread rxin
Repository: spark Updated Branches: refs/heads/master 874a2ca93 -> 29576e786 [SPARK-6829] Added math functions for DataFrames Implemented almost all math functions found in scala.math (max, min and abs were already present). cc mengxr marmbrus Author: Burak Yavuz Closes #5616 from brkyvz/

spark git commit: [SPARK-5946] [STREAMING] Add Python API for direct Kafka stream

2015-04-27 Thread tdas
Repository: spark Updated Branches: refs/heads/master 29576e786 -> 9e4e82b7b [SPARK-5946] [STREAMING] Add Python API for direct Kafka stream Currently only added `createDirectStream` API, I'm not sure if `createRDD` is also needed, since some Java object needs to be wrapped in Python. Please