git commit: [SPARK-1760]: fix building spark with maven documentation

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 32868f31f -> bd67551ee [SPARK-1760]: fix building spark with maven documentation Author: witgo Closes #712 from witgo/building-with-maven and squashes the following commits: 215523b [witgo] fix building spark with maven documentation P

git commit: Update GradientDescentSuite.scala

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 bb90e87f6 -> 0972b6251 Update GradientDescentSuite.scala use more faster way to construct an array Author: baishuo(白硕) Closes #588 from baishuo/master and squashes the following commits: 45b95fb [baishuo(白硕)] Update Gradient

git commit: String interpolation + some other small changes

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 94c6c06ea -> 601e37198 String interpolation + some other small changes After having been invited to make the change in https://github.com/apache/spark/commit/6bee01dd04ef73c6b829110ebcdd622d521ea8ff#commitcomment-6284165 by @witgo. Autho

[1/2] SPARK-1565, update examples to be used with spark-submit script.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 8f3b9250c -> 30cfa8dc1 http://git-wip-us.apache.org/repos/asf/spark/blob/30cfa8dc/examples/src/main/scala/org/apache/spark/examples/streaming/MQTTWordCount.scala -- diff

git commit: Add language tabs and Python version to interactive part of quick-start

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ad4e60ee7 -> f10de042b Add language tabs and Python version to interactive part of quick-start This is an addition of some stuff that was missed in https://issues.apache.org/jira/browse/SPARK-1567. I've also updated the doc to show submit

svn commit: r1593657 - in /spark: community.md documentation.md site/community.html site/documentation.html

2014-05-15 Thread andrew
Author: andrew Date: Sat May 10 02:20:32 2014 New Revision: 1593657 URL: http://svn.apache.org/r1593657 Log: add link to meetup videos from community page Modified: spark/community.md spark/documentation.md spark/site/community.html spark/site/documentation.html Modified: spark/c

git commit: SPARK-1829 Sub-second durations shouldn't round to "0 s"

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master fde82c154 -> a3315d7f4 SPARK-1829 Sub-second durations shouldn't round to "0 s" As "99 ms" up to 99 ms As "0.1 s" from 0.1 s up to 0.9 s https://issues.apache.org/jira/browse/SPARK-1829 Compare the first image to the second here: http://i

[1/3] git commit: Revert "[maven-release-plugin] prepare for next development iteration"

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 9ff9078fc -> e8ca3970c Revert "[maven-release-plugin] prepare for next development iteration" This reverts commit d08e9604fc9958b7c768e91715c8152db2ed6fd0. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-

git commit: [FIX] do not load defaults when testing SparkConf in pyspark

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 8e13ab2fe -> 31faec790 [FIX] do not load defaults when testing SparkConf in pyspark The default constructor loads default properties, which can fail the test. Author: Xiangrui Meng Closes #775 from mengxr/pyspark-conf-fix and squashe

git commit: Documentation: Encourage use of reduceByKey instead of groupByKey.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master f10de042b -> 21570b463 Documentation: Encourage use of reduceByKey instead of groupByKey. Author: Patrick Wendell Closes #784 from pwendell/group-by-key and squashes the following commits: 9b4505f [Patrick Wendell] Small fix 6347924 [Pat

[1/2] git commit: Revert "[maven-release-plugin] prepare for next development iteration"

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 31b853ce8 -> aa2ac7060 Revert "[maven-release-plugin] prepare for next development iteration" This reverts commit e480bcfbd269ae1d7a6a92cfb50466cf192fe1fb. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-

git commit: default task number misleading in several places

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 fdf9717da -> 9f0f2ecb8 default task number misleading in several places private[streaming] def defaultPartitioner(numPartitions: Int = self.ssc.sc.defaultParallelism){ new HashPartitioner(numPartitions) } it represents that th

git commit: [SPARK-1826] fix the head notation of package object dsl

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 5ca3096dd -> fdf9717da [SPARK-1826] fix the head notation of package object dsl Author: wangfei Closes #765 from scwf/dslfix and squashes the following commits: d2d1a9d [wangfei] Update package.scala 66ff53b [wangfei] fix the head no

git commit: SPARK-1775: Unneeded lock in ShuffleMapTask.deserializeInfo

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 06b15baab -> 7db47c463 SPARK-1775: Unneeded lock in ShuffleMapTask.deserializeInfo This was used in the past to have a cache of deserialized ShuffleMapTasks, but that's been removed, so there's no need for a lock. It slows down Spark when

[2/2] git commit: Revert "[maven-release-plugin] prepare release v1.0.0-rc6"

2014-05-15 Thread pwendell
Revert "[maven-release-plugin] prepare release v1.0.0-rc6" This reverts commit 54133abdce0246f6643a1112a5204afb2c4caa82. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/aa2ac706 Tree: http://git-wip-us.apache.org/repos/asf/sp

git commit: Fixing typo in als.py

2014-05-15 Thread shivaram
Repository: spark Updated Branches: refs/heads/master c3f8b78c2 -> 5c5e7d580 Fixing typo in als.py XtY should be Xty. Author: Evan Sparks Closes #696 from etrain/patch-2 and squashes the following commits: 634cb8d [Evan Sparks] Fixing typo in als.py Project: http://git-wip-us.apache.org/

git commit: [SPARK-1460] Returning SchemaRDD instead of normal RDD on Set operations...

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 3eb53bd59 -> 967635a24 [SPARK-1460] Returning SchemaRDD instead of normal RDD on Set operations... ... that do not change schema Author: Kan Zhang Closes #448 from kanzhang/SPARK-1460 and squashes the following commits: 111e388 [Kan Zha

[2/3] git commit: Revert "[maven-release-plugin] prepare release v1.0.0-rc5"

2014-05-15 Thread pwendell
Revert "[maven-release-plugin] prepare release v1.0.0-rc5" This reverts commit 18f062303303824139998e8fc8f4158217b0dbc3. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/fdadad06 Tree: http://git-wip-us.apache.org/repos/asf/sp

git commit: [SPARK-1157][MLlib] Bug fix: lossHistory should exclude rejection steps, and remove miniBatch

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 d81d6261b -> 34529975e [SPARK-1157][MLlib] Bug fix: lossHistory should exclude rejection steps, and remove miniBatch Getting the lossHistory from Breeze's API which already excludes the rejection steps in line search. Also, remove the

git commit: [SPARK-1826] fix the head notation of package object dsl

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9ad096d55 -> 44165fc91 [SPARK-1826] fix the head notation of package object dsl Author: wangfei Closes #765 from scwf/dslfix and squashes the following commits: d2d1a9d [wangfei] Update package.scala 66ff53b [wangfei] fix the head notati

git commit: [maven-release-plugin] prepare release v1.0.0-rc7

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 aa2ac7060 -> 9212b3e5b [maven-release-plugin] prepare release v1.0.0-rc7 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9212b3e5 Tree: http://git-wip-us.apache.org

Git Push Summary

2014-05-15 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.0.0-rc7 [created] bc81ecfdc

git commit: SPARK-1579: Clean up PythonRDD and avoid swallowing IOExceptions

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 da9f9e05b -> 18caa8cbc SPARK-1579: Clean up PythonRDD and avoid swallowing IOExceptions This patch includes several cleanups to PythonRDD, focused around fixing [SPARK-1579](https://issues.apache.org/jira/browse/SPARK-1579) cleanly. Li

git commit: [SPARK-1644] The org.datanucleus:* should not be packaged into spark-assembly-*.jar

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 2f452cbaf -> 561510867 [SPARK-1644] The org.datanucleus:* should not be packaged into spark-assembly-*.jar Author: witgo Closes #688 from witgo/SPARK-1644 and squashes the following commits: 56ad6ac [witgo] review commit 87c03e4 [witgo]

git commit: default task number misleading in several places

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master 44165fc91 -> 2f639957f default task number misleading in several places private[streaming] def defaultPartitioner(numPartitions: Int = self.ssc.sc.defaultParallelism){ new HashPartitioner(numPartitions) } it represents that the de

git commit: [maven-release-plugin] prepare for next development iteration

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 9212b3e5b -> c4746aa6f [maven-release-plugin] prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c4746aa6 Tree: http://git-wip-u

git commit: [SPARK-1754] [SQL] Add missing arithmetic DSL operations.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 ee633210c -> 6f701ff55 [SPARK-1754] [SQL] Add missing arithmetic DSL operations. Add missing arithmetic DSL operations: `unary_-`, `%`. Author: Takuya UESHIN Closes #689 from ueshin/issues/SPARK-1754 and squashes the following commit

git commit: Typo fix: fetchting -> fetching

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7f6f4a103 -> d00981a95 Typo fix: fetchting -> fetching Author: Andrew Ash Closes #680 from ash211/patch-3 and squashes the following commits: 9ce3746 [Andrew Ash] Typo fix: fetchting -> fetching Project: http://git-wip-us.apache.org/re

git commit: [SQL] Improve SparkSQL Aggregates

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 010040fd0 -> 8f3b9250c [SQL] Improve SparkSQL Aggregates * Add native min/max (was using hive before). * Handle nulls correctly in Avg and Sum. Author: Michael Armbrust Closes #683 from marmbrus/aggFixes and squashes the following co

git commit: SPARK-1569 Spark on Yarn, authentication broken by pr299

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 82c8e89c9 -> 879bd SPARK-1569 Spark on Yarn, authentication broken by pr299 Pass the configs as java options since the executor needs to know before it registers whether to create the connection using authentication or not.We

git commit: fix different versions of commons-lang dependency and apache/spark#746 addendum

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 c02d6144a -> 31b853ce8 fix different versions of commons-lang dependency and apache/spark#746 addendum Author: witgo Closes #754 from witgo/commons-lang and squashes the following commits: 3ebab31 [witgo] merge master f3b8fa2 [witgo]

git commit: Add language tabs and Python version to interactive part of quick-start

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 ba871235f -> 976784b1e Add language tabs and Python version to interactive part of quick-start This is an addition of some stuff that was missed in https://issues.apache.org/jira/browse/SPARK-1567. I've also updated the doc to show su

git commit: [SPARK-1840] SparkListenerBus prints out scary error message when terminated normally

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 9f0f2ecb8 -> ba871235f [SPARK-1840] SparkListenerBus prints out scary error message when terminated normally Running SparkPi example gave this error. ``` Pi is roughly 3.14374 14/05/14 18:16:19 ERROR Utils: Uncaught exception in thread

git commit: [SPARK-1688] Propagate PySpark worker stderr to driver

2014-05-15 Thread adav
Repository: spark Updated Branches: refs/heads/master d00981a95 -> 520087224 [SPARK-1688] Propagate PySpark worker stderr to driver When at least one of the following conditions is true, PySpark cannot be loaded: 1. PYTHONPATH is not set 2. PYTHONPATH does not contain the python directory (or

git commit: [SPARK-1840] SparkListenerBus prints out scary error message when terminated normally

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 2f639957f -> ad4e60ee7 [SPARK-1840] SparkListenerBus prints out scary error message when terminated normally Running SparkPi example gave this error. ``` Pi is roughly 3.14374 14/05/14 18:16:19 ERROR Utils: Uncaught exception in thread Spa

git commit: [SPARK-1631] Correctly set the Yarn app name when launching the AM.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 666bebe63 -> 7e1933451 [SPARK-1631] Correctly set the Yarn app name when launching the AM. Author: Marcelo Vanzin Closes #539 from vanzin/yarn-app-name and squashes the following commits: 7d1ca4f [Marcelo Vanzin] [SPARK-1631] Correct

git commit: MLlib documentation fix

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 98944a973 -> d81d6261b MLlib documentation fix Fixed the documentation for that `loadLibSVMData` is changed to `loadLibSVMFile`. Author: DB Tsai Closes #703 from dbtsai/dbtsai-docfix and squashes the following commits: 71dd508 [DB

git commit: Converted bang to ask to avoid scary warning when a block is removed

2014-05-15 Thread tdas
Repository: spark Updated Branches: refs/heads/master 4c60fd1e8 -> 32868f31f Converted bang to ask to avoid scary warning when a block is removed Removing a block through the blockmanager gave a scary warning messages in the driver. ``` 2014-05-08 20:16:19,172 WARN BlockManagerMasterActor: Go

git commit: [SPARK-1696][MLLIB] use alpha in dense dspr

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 31faec790 -> d6f1a75bc [SPARK-1696][MLLIB] use alpha in dense dspr It doesn't affect existing code because only `alpha = 1.0` is used in the code. Author: Xiangrui Meng Closes #778 from mengxr/mllib-dspr-fix and squashes the followin

git commit: [SQL] Fix Performance Issue in data type casting

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master 3308722ca -> ca4318686 [SQL] Fix Performance Issue in data type casting Using lazy val object instead of function in the class Cast, which improved the performance nearly by 2X in my local micro-benchmark. Author: Cheng Hao Closes #679

git commit: SPARK-1579: Clean up PythonRDD and avoid swallowing IOExceptions

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 967635a24 -> 3308722ca SPARK-1579: Clean up PythonRDD and avoid swallowing IOExceptions This patch includes several cleanups to PythonRDD, focused around fixing [SPARK-1579](https://issues.apache.org/jira/browse/SPARK-1579) cleanly. Listed

git commit: [SPARK-1631] Correctly set the Yarn app name when launching the AM.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8b7841299 -> 3f779d872 [SPARK-1631] Correctly set the Yarn app name when launching the AM. Author: Marcelo Vanzin Closes #539 from vanzin/yarn-app-name and squashes the following commits: 7d1ca4f [Marcelo Vanzin] [SPARK-1631] Correctly s

git commit: fix different versions of commons-lang dependency and apache/spark#746 addendum

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 46324279d -> bae07e36a fix different versions of commons-lang dependency and apache/spark#746 addendum Author: witgo Closes #754 from witgo/commons-lang and squashes the following commits: 3ebab31 [witgo] merge master f3b8fa2 [witgo] mer

git commit: Documentation: Encourage use of reduceByKey instead of groupByKey.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 976784b1e -> f2faa37f6 Documentation: Encourage use of reduceByKey instead of groupByKey. Author: Patrick Wendell Closes #784 from pwendell/group-by-key and squashes the following commits: 9b4505f [Patrick Wendell] Small fix 6347924

git commit: [FIX] do not load defaults when testing SparkConf in pyspark

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master 65533c7ec -> 94c6c06ea [FIX] do not load defaults when testing SparkConf in pyspark The default constructor loads default properties, which can fail the test. Author: Xiangrui Meng Closes #775 from mengxr/pyspark-conf-fix and squashes th

git commit: SPARK-1544 Add support for deep decision trees.

2014-05-15 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.0 0972b6251 -> c7b27043a SPARK-1544 Add support for deep decision trees. @etrain and I came with a PR for arbitrarily deep decision trees at the cost of multiple passes over the data at deep tree levels. To summarize: 1) We take a param

git commit: [Typo] propertes -> properties

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.0 d6f1a75bc -> 5ca3096dd [Typo] propertes -> properties Author: andrewor14 Closes #780 from andrewor14/submit-typo and squashes the following commits: e70e057 [andrewor14] propertes -> properties (cherry picked from commit 9ad096d55a3

git commit: [Typo] propertes -> properties

2014-05-15 Thread rxin
Repository: spark Updated Branches: refs/heads/master e3d72a74a -> 9ad096d55 [Typo] propertes -> properties Author: andrewor14 Closes #780 from andrewor14/submit-typo and squashes the following commits: e70e057 [andrewor14] propertes -> properties Project: http://git-wip-us.apache.org/rep