spark git commit: [SQL] support udt to hive types conversion (hive-udt is not supported)

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master bd86cb173 - 894a7245c [SQL] support udt to hive types conversion (hive-udt is not supported) marmbrus Author: Xiangrui Meng m...@databricks.com Closes #3164 from mengxr/hive-udt and squashes the following commits: 57c7519 [Xiangrui

spark git commit: [SPARK-4169] [Core] Accommodate non-English Locales in unit tests

2014-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 894a7245c - ed8bf1eac [SPARK-4169] [Core] Accommodate non-English Locales in unit tests For me the core tests failed because there are two locale dependent parts in the code. Look at the Jira ticket for details. Why is it necessary to

spark git commit: [SPARK-4169] [Core] Accommodate non-English Locales in unit tests

2014-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 9b781c816 - 69dd2997f [SPARK-4169] [Core] Accommodate non-English Locales in unit tests For me the core tests failed because there are two locale dependent parts in the code. Look at the Jira ticket for details. Why is it necessary

spark git commit: [SPARK-4169] [Core] Accommodate non-English Locales in unit tests

2014-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 78cd3ab88 - dc38defd2 [SPARK-4169] [Core] Accommodate non-English Locales in unit tests For me the core tests failed because there are two locale dependent parts in the code. Look at the Jira ticket for details. Why is it necessary

spark git commit: SPARK-2548 [STREAMING] JavaRecoverableWordCount is missing

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master ed8bf1eac - 3a02d416c SPARK-2548 [STREAMING] JavaRecoverableWordCount is missing Here's my attempt to re-port `RecoverableNetworkWordCount` to Java, following the example of its Scala and Java siblings. I fixed a few minor doc/formatting

spark git commit: SPARK-2548 [STREAMING] JavaRecoverableWordCount is missing

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.1 dc38defd2 - cdcf5467a SPARK-2548 [STREAMING] JavaRecoverableWordCount is missing Here's my attempt to re-port `RecoverableNetworkWordCount` to Java, following the example of its Scala and Java siblings. I fixed a few minor

spark git commit: Update RecoverableNetworkWordCount.scala

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master 3a02d416c - 0340c56a9 Update RecoverableNetworkWordCount.scala Trying this example, I missed the moment when the checkpoint was iniciated Author: comcmipi pito...@fns.uniba.sk Closes #2735 from comcmipi/patch-1 and squashes the following

spark git commit: Update RecoverableNetworkWordCount.scala

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.1 cdcf5467a - 254b13570 Update RecoverableNetworkWordCount.scala Trying this example, I missed the moment when the checkpoint was iniciated Author: comcmipi pito...@fns.uniba.sk Closes #2735 from comcmipi/patch-1 and squashes the

spark git commit: [SPARK-4312] bash doesn't have die

2014-11-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 0340c56a9 - c5db8e2c0 [SPARK-4312] bash doesn't have die sbt-launch-lib.bash includes `die` command but it's not valid command for Linux, MacOS X or Windows. Closes #2898 Author: Jey Kottalam j...@kottalam.net Closes #3182 from

spark git commit: [SPARK-4312] bash doesn't have die

2014-11-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 ca3fe8c12 - 7917f27a3 [SPARK-4312] bash doesn't have die sbt-launch-lib.bash includes `die` command but it's not valid command for Linux, MacOS X or Windows. Closes #2898 Author: Jey Kottalam j...@kottalam.net Closes #3182 from

spark git commit: SPARK-4230. Doc for spark.default.parallelism is incorrect

2014-11-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.2 7917f27a3 - 04a79b616 SPARK-4230. Doc for spark.default.parallelism is incorrect Author: Sandy Ryza sa...@cloudera.com Closes #3107 from sryza/sandy-spark-4230 and squashes the following commits: 37a1d19 [Sandy Ryza] Clear up a

spark git commit: [SPARK-4047] - Generate runtime warnings for example implementation of PageRank

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 dd1b2a0a9 - 19dcb5714 [SPARK-4047] - Generate runtime warnings for example implementation of PageRank Based on SPARK-2434, this PR generates runtime warnings for example implementations (Python, Scala) of PageRank. Author:

spark git commit: [SPARK-4047] - Generate runtime warnings for example implementation of PageRank

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/master b32734e12 - 974d334cf [SPARK-4047] - Generate runtime warnings for example implementation of PageRank Based on SPARK-2434, this PR generates runtime warnings for example implementations (Python, Scala) of PageRank. Author: Varadharajan

spark git commit: Revert [SPARK-2703][Core]Make Tachyon related unit tests execute without deploying a Tachyon system locally.

2014-11-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 974d334cf - 6e7a309b8 Revert [SPARK-2703][Core]Make Tachyon related unit tests execute without deploying a Tachyon system locally. This reverts commit bd86cb1738800a0aa4c88b9afdba2f97ac6cbf25. Project:

spark git commit: [SPARK-4319][SQL] Enable an ignored test null count.

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 19dcb5714 - 0089a4f64 [SPARK-4319][SQL] Enable an ignored test null count. Author: Takuya UESHIN ues...@happy-camper.st Closes #3185 from ueshin/issues/SPARK-4319 and squashes the following commits: a44a38e [Takuya UESHIN] Enable an

spark git commit: [SPARK-4000][Build] Uploads HiveCompatibilitySuite logs

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master dbf10588d - 534b23141 [SPARK-4000][Build] Uploads HiveCompatibilitySuite logs This is a follow up of #2845. In addition to unit-tests.log files, also upload failure output files generated by `HiveCompatibilitySuite` to Jenkins master.

spark git commit: [SPARK-4308][SQL] Follow up of #3175 for branch 1.1

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.1 86b1bd031 - b3ef06b75 [SPARK-4308][SQL] Follow up of #3175 for branch 1.1 PR #3175 is for master branch only and can't be backported to branch 1.1 directly because Hive 0.13.1 support. Author: Cheng Lian l...@databricks.com Closes

spark git commit: [SPARK-3971][SQL] Backport #2843 to branch-1.1

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.1 b3ef06b75 - 64945f868 [SPARK-3971][SQL] Backport #2843 to branch-1.1 This PR backports #2843 to branch-1.1. The key difference is that this one doesn't support Hive 0.13.1 and thus always returns `0.12.0` when

spark git commit: [SQL] remove a decimal case branch that has no effect at runtime

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 0089a4f64 - 1ed1c68c0 [SQL] remove a decimal case branch that has no effect at runtime it generates warnings at compile time marmbrus Author: Xiangrui Meng m...@databricks.com Closes #3192 from mengxr/dtc-decimal and squashes the

spark git commit: [SQL] remove a decimal case branch that has no effect at runtime

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master acb55aedd - d793d80c8 [SQL] remove a decimal case branch that has no effect at runtime it generates warnings at compile time marmbrus Author: Xiangrui Meng m...@databricks.com Closes #3192 from mengxr/dtc-decimal and squashes the

spark git commit: [SPARK-4250] [SQL] Fix bug of constant null value mapping to ConstantObjectInspector

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 1ed1c68c0 - ff071e351 [SPARK-4250] [SQL] Fix bug of constant null value mapping to ConstantObjectInspector Author: Cheng Hao hao.ch...@intel.com Closes #3114 from chenghao-intel/constant_null_oi and squashes the following commits:

spark git commit: [SPARK-4149][SQL] ISO 8601 support for json date time strings

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master fa777833b - a1fc059b6 [SPARK-4149][SQL] ISO 8601 support for json date time strings This implement the feature davies mentioned in https://github.com/apache/spark/pull/2901#discussion-diff-19313312 Author: Daoyuan Wang

spark git commit: [SPARK-4149][SQL] ISO 8601 support for json date time strings

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 ff071e351 - f0eb0a79c [SPARK-4149][SQL] ISO 8601 support for json date time strings This implement the feature davies mentioned in https://github.com/apache/spark/pull/2901#discussion-diff-19313312 Author: Daoyuan Wang

spark git commit: [SPARK-3954][Streaming] Optimization to FileInputDStream

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.2 f0eb0a79c - 07ba50f7e [SPARK-3954][Streaming] Optimization to FileInputDStream about convert files to RDDS there are 3 loops with files sequence in spark source. loops files sequence: 1.files.map(...) 2.files.zip(fileRDDs)

spark git commit: [SPARK-3954][Streaming] Optimization to FileInputDStream

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master a1fc059b6 - ce6ed2abd [SPARK-3954][Streaming] Optimization to FileInputDStream about convert files to RDDS there are 3 loops with files sequence in spark source. loops files sequence: 1.files.map(...) 2.files.zip(fileRDDs)

spark git commit: [SPARK-3954][Streaming] Optimization to FileInputDStream

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.1 64945f868 - 3d889dfc1 [SPARK-3954][Streaming] Optimization to FileInputDStream about convert files to RDDS there are 3 loops with files sequence in spark source. loops files sequence: 1.files.map(...) 2.files.zip(fileRDDs)

spark git commit: [SPARK-4274] [SQL] Fix NPE in printing the details of the query plan

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master ce6ed2abd - c764d0ac1 [SPARK-4274] [SQL] Fix NPE in printing the details of the query plan Author: Cheng Hao hao.ch...@intel.com Closes #3139 from chenghao-intel/comparison_test and squashes the following commits: f5d7146 [Cheng Hao]

spark git commit: [SPARK-4274] [SQL] Fix NPE in printing the details of the query plan

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.2 07ba50f7e - 50c02d68a [SPARK-4274] [SQL] Fix NPE in printing the details of the query plan Author: Cheng Hao hao.ch...@intel.com Closes #3139 from chenghao-intel/comparison_test and squashes the following commits: f5d7146 [Cheng

spark git commit: [SPARK-3495][SPARK-3496] Backporting block replication fixes made in master to branch 1.1

2014-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.1 3d889dfc1 - be0cc9952 [SPARK-3495][SPARK-3496] Backporting block replication fixes made in master to branch 1.1 The original PR was #2366 This backport was non-trivial because Spark 1.1 uses ConnectionManager instead of

spark git commit: Update versions for 1.1.1 release

2014-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 be0cc9952 - 01d233e4a Update versions for 1.1.1 release Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/01d233e4 Tree:

spark git commit: [SPARK-3649] Remove GraphX custom serializers

2014-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master c764d0ac1 - 300887bd7 [SPARK-3649] Remove GraphX custom serializers As [reported][1] on the mailing list, GraphX throws ``` java.lang.ClassCastException: java.lang.Long cannot be cast to scala.Tuple2 at

spark git commit: [SPARK-3649] Remove GraphX custom serializers

2014-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 50c02d68a - e725cab66 [SPARK-3649] Remove GraphX custom serializers As [reported][1] on the mailing list, GraphX throws ``` java.lang.ClassCastException: java.lang.Long cannot be cast to scala.Tuple2 at

spark git commit: [SQL] Backport backtick and smallint JDBC fixes to 1.1

2014-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.1 01d233e4a - 8a1d818a5 [SQL] Backport backtick and smallint JDBC fixes to 1.1 Author: Michael Armbrust mich...@databricks.com Author: ravipesala ravindra.pes...@huawei.com Author: scwf wangf...@huawei.com Closes #3199 from

spark git commit: [SPARK-4330][Doc] Link to proper URL for YARN overview

2014-11-10 Thread matei
Repository: spark Updated Branches: refs/heads/master 300887bd7 - 3c07b8f08 [SPARK-4330][Doc] Link to proper URL for YARN overview In running-on-yarn.md, a link to YARN overview is here. But the URL is to YARN alpha's. It should be stable's. Author: Kousuke Saruta saru...@oss.nttdata.co.jp

spark git commit: [SPARK-4330][Doc] Link to proper URL for YARN overview

2014-11-10 Thread matei
Repository: spark Updated Branches: refs/heads/branch-1.2 e725cab66 - 4eeaf3395 [SPARK-4330][Doc] Link to proper URL for YARN overview In running-on-yarn.md, a link to YARN overview is here. But the URL is to YARN alpha's. It should be stable's. Author: Kousuke Saruta

spark git commit: [SPARK-4330][Doc] Link to proper URL for YARN overview

2014-11-10 Thread matei
Repository: spark Updated Branches: refs/heads/branch-1.1 8a1d818a5 - d313be8f9 [SPARK-4330][Doc] Link to proper URL for YARN overview In running-on-yarn.md, a link to YARN overview is here. But the URL is to YARN alpha's. It should be stable's. Author: Kousuke Saruta

spark git commit: [BRANCH-1.1][SPARK-2652] change the default spark.serializer in pyspark back to Kryo

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.1 d313be8f9 - 11798d00f [BRANCH-1.1][SPARK-2652] change the default spark.serializer in pyspark back to Kryo This reverts #2916 . We shouldn't change the default settings in a minor release. JoshRosen davies Author: Xiangrui Meng

spark git commit: [SPARK-4324] [PySpark] [MLlib] support numpy.array for all MLlib API

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/master 3c07b8f08 - 65083e93d [SPARK-4324] [PySpark] [MLlib] support numpy.array for all MLlib API This PR check all of the existing Python MLlib API to make sure that numpy.array is supported as Vector (also RDD of numpy.array). It also improve

spark git commit: [SPARK-4324] [PySpark] [MLlib] support numpy.array for all MLlib API

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 4eeaf3395 - df8242c9b [SPARK-4324] [PySpark] [MLlib] support numpy.array for all MLlib API This PR check all of the existing Python MLlib API to make sure that numpy.array is supported as Vector (also RDD of numpy.array). It also

spark git commit: [branch-1.1][SPARK-3990] add a note on ALS usage

2014-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.1 11798d00f - b2cb357d7 [branch-1.1][SPARK-3990] add a note on ALS usage Because we switched back to Kryo in #3187 , we need to leave a note about the workaround. Author: Xiangrui Meng m...@databricks.com Closes #3190 from

svn commit: r1638035 - in /spark: images/spark-runs-everywhere.png index.md site/images/spark-runs-everywhere.png site/index.html

2014-11-10 Thread rxin
Author: rxin Date: Tue Nov 11 07:11:21 2014 New Revision: 1638035 URL: http://svn.apache.org/r1638035 Log: Update Spark Runs Everywhere section Added: spark/images/spark-runs-everywhere.png (with props) spark/site/images/spark-runs-everywhere.png (with props) Modified:

svn commit: r1638041 [2/2] - in /spark: ./ _layouts/ site/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2014-11-10 Thread rxin
Modified: spark/site/news/two-weeks-to-spark-summit-2014.html URL: http://svn.apache.org/viewvc/spark/site/news/two-weeks-to-spark-summit-2014.html?rev=1638041r1=1638040r2=1638041view=diff == ---

svn commit: r1638041 [1/2] - in /spark: ./ _layouts/ site/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2014-11-10 Thread rxin
Author: rxin Date: Tue Nov 11 07:40:10 2014 New Revision: 1638041 URL: http://svn.apache.org/r1638041 Log: various minor updates to the site Modified: spark/_layouts/global.html spark/community.md spark/index.md spark/site/community.html spark/site/documentation.html

svn commit: r1638042 - /spark/site/community.html

2014-11-10 Thread rxin
Author: rxin Date: Tue Nov 11 07:41:21 2014 New Revision: 1638042 URL: http://svn.apache.org/r1638042 Log: various minor updates to the site Modified: spark/site/community.html Modified: spark/site/community.html URL: