spark git commit: [SPARK-7886] Use FunctionRegistry for built-in expressions in HiveContext.

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 778f3ca81 -> 57c60c5be [SPARK-7886] Use FunctionRegistry for built-in expressions in HiveContext. This builds on #6710 and also uses FunctionRegistry for function lookup in HiveContext. Author: Reynold Xin Closes #6712 from rxin/udf-reg

spark git commit: [SPARK-7886] Added unit test for HAVING aggregate pushdown.

2015-06-10 Thread lian
Repository: spark Updated Branches: refs/heads/master 57c60c5be -> e90035e67 [SPARK-7886] Added unit test for HAVING aggregate pushdown. This is a followup to #6712. Author: Reynold Xin Closes #6739 from rxin/6712-followup and squashes the following commits: fd9acfb [Reynold Xin] [SPARK-78

spark git commit: [SPARK-8215] [SPARK-8212] [SQL] add leaf math expression for e and pi

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master e90035e67 -> c6ba7cca3 [SPARK-8215] [SPARK-8212] [SQL] add leaf math expression for e and pi Author: Daoyuan Wang Closes #6716 from adrian-wang/epi and squashes the following commits: e2e8dbd [Daoyuan Wang] move tests 11b351c [Daoyuan Wa

spark git commit: [SPARK-7996] Deprecate the developer api SparkEnv.actorSystem

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master c6ba7cca3 -> 2b550a521 [SPARK-7996] Deprecate the developer api SparkEnv.actorSystem Changed ```SparkEnv.actorSystem``` to be a function such that we can use the deprecated flag with it and added a deprecated message. Author: Ilya Ganelin

spark git commit: [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 2b550a521 -> 8f7308f9c [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide Author: Cheng Lian Closes #6749 from liancheng/java-sample-fix and squashes the following commits: 5b44585 [Cheng Lian] Fixes a minor Java exa

spark git commit: [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 a0a7f2f92 -> 7b88e6a1e [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide Author: Cheng Lian Closes #6749 from liancheng/java-sample-fix and squashes the following commits: 5b44585 [Cheng Lian] Fixes a minor Java

spark git commit: [SPARK-5479] [YARN] Handle --py-files correctly in YARN.

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 8f7308f9c -> 38112905b [SPARK-5479] [YARN] Handle --py-files correctly in YARN. The bug description is a little misleading: the actual issue is that .py files are not handled correctly when distributed by YARN. They're added to "spark.submi

spark git commit: [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 7b88e6a1e -> 28e8a6ea6 [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable Read number of threads for RBackend from configuration. [SPARK-8282] #comment Linking with JIRA Author: Hossein Closes #6730 from fala

spark git commit: [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 38112905b -> 30ebf1a23 [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable Read number of threads for RBackend from configuration. [SPARK-8282] #comment Linking with JIRA Author: Hossein Closes #6730 from falaki/S

spark git commit: [SPARK-7756] CORE RDDOperationScope fix for IBM Java

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 30ebf1a23 -> 19e30b48f [SPARK-7756] CORE RDDOperationScope fix for IBM Java IBM Java has an extra method when we do getStackTrace(): this is "getStackTraceImpl", a native method. This causes two tests to fail within "DStreamScopeSuite" wh

spark git commit: [SPARK-7756] CORE RDDOperationScope fix for IBM Java

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 28e8a6ea6 -> 568d1d51d [SPARK-7756] CORE RDDOperationScope fix for IBM Java IBM Java has an extra method when we do getStackTrace(): this is "getStackTraceImpl", a native method. This causes two tests to fail within "DStreamScopeSuite

spark git commit: [SPARK-7527] [CORE] Fix createNullValue to return the correct null values and REPL mode detection

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 19e30b48f -> e90c9d92d [SPARK-7527] [CORE] Fix createNullValue to return the correct null values and REPL mode detection The root cause of SPARK-7527 is `createNullValue` returns an incompatible value `Byte(0)` for `char` and `boolean`.

spark git commit: [SPARK-7261] [CORE] Change default log level to WARN in the REPL

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master e90c9d92d -> 80043e9e7 [SPARK-7261] [CORE] Change default log level to WARN in the REPL 1. Add `log4j-defaults-repl.properties` that has log level WARN. 2. When logging is initialized, check whether inside the REPL. If so, use `log4j-defau

spark git commit: [SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 80043e9e7 -> cb871c44c [SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add it back so process launched by spark-

spark git commit: [SPARK-8273] Driver hangs up when yarn shutdown in client mode

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master cb871c44c -> 5014d0ed7 [SPARK-8273] Driver hangs up when yarn shutdown in client mode In client mode, if yarn was shut down with spark application running, the application will hang up after several retries(default: 30) because the except

spark git commit: [SPARK-8273] Driver hangs up when yarn shutdown in client mode

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 568d1d51d -> 2846a357f [SPARK-8273] Driver hangs up when yarn shutdown in client mode In client mode, if yarn was shut down with spark application running, the application will hang up after several retries(default: 30) because the ex

spark git commit: [SPARK-2774] Set preferred locations for reduce tasks

2015-06-10 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master 5014d0ed7 -> 96a7c888d [SPARK-2774] Set preferred locations for reduce tasks Set preferred locations for reduce tasks. The basic design is that we maintain a map from reducerId to a list of (sizes, locations) for each shuffle. We then set

spark git commit: [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm

2015-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 96a7c888d -> b928f5438 [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm Test cases for both StreamingLinearRegression and StreamingLogisticRegression, and code fix. Edit: This contribution is my original work and I li

spark git commit: [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm

2015-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.4 2846a357f -> 59fc3f197 [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm Test cases for both StreamingLinearRegression and StreamingLogisticRegression, and code fix. Edit: This contribution is my original work and

svn commit: r9330 - /dev/spark/spark-1.4.0-rc4/

2015-06-10 Thread pwendell
Author: pwendell Date: Wed Jun 10 23:16:15 2015 New Revision: 9330 Log: Adding Spark 1.4.0 RC4 Added: dev/spark/spark-1.4.0-rc4/ dev/spark/spark-1.4.0-rc4/spark-1.4.0-bin-cdh4.tgz (with props) dev/spark/spark-1.4.0-rc4/spark-1.4.0-bin-cdh4.tgz.asc (with props) dev/spark/spark-

svn commit: r9331 - /dev/spark/spark-1.4.0-rc4/ /release/spark/spark-1.4.0/

2015-06-10 Thread pwendell
Author: pwendell Date: Wed Jun 10 23:18:01 2015 New Revision: 9331 Log: Adding Spark release 1.4.0 Added: release/spark/spark-1.4.0/ - copied from r9330, dev/spark/spark-1.4.0-rc4/ Removed: dev/spark/spark-1.4.0-rc4/ ---

spark git commit: [SPARK-8189] [SQL] use Long for TimestampType in SQL

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master b928f5438 -> 37719e0cd [SPARK-8189] [SQL] use Long for TimestampType in SQL This PR change to use Long as internal type for TimestampType for efficiency, which means it will the precision below 100ns. Author: Davies Liu Closes #6733 fro

spark git commit: [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 59fc3f197 -> 5c05b5c0d [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first case cs CombineSum(expr) => val calcType = expr.dataType expr.dataType match { case DecimalType.Fixe

spark git commit: [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 37719e0cd -> 6a47114bc [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first case cs CombineSum(expr) => val calcType = expr.dataType expr.dataType match { case DecimalType.Fixed(_,

spark git commit: [SPARK-8164] transformExpressions should support nested expression sequence

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6a47114bc -> 4e42842e8 [SPARK-8164] transformExpressions should support nested expression sequence Currently we only support `Seq[Expression]`, we should handle cases like `Seq[Seq[Expression]]` so that we can remove the unnecessary `Group

spark git commit: [SPARK-8248][SQL] string function: length

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4e42842e8 -> 9fe3adcce [SPARK-8248][SQL] string function: length Author: Cheng Hao Closes #6724 from chenghao-intel/length and squashes the following commits: aaa3c31 [Cheng Hao] revert the additional change 97148a9 [Cheng Hao] remove th

spark git commit: [SPARK-8217] [SQL] math function log2

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9fe3adcce -> 2758ff0a9 [SPARK-8217] [SQL] math function log2 Author: Daoyuan Wang This patch had conflicts when merged, resolved by Committer: Reynold Xin Closes #6718 from adrian-wang/udflog2 and squashes the following commits: 3909f4

spark git commit: [HOTFIX] Adding more contributor name bindings

2015-06-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 2758ff0a9 -> a777eb04b [HOTFIX] Adding more contributor name bindings Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a777eb04 Tree: http://git-wip-us.apache.org/repos/

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0 [created] 22596c534 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc2 [deleted] 03fb26a3e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc1 [deleted] 777a08166 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc3 [deleted] dd109a874 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc4 [deleted] 22596c534 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [HOTFIX] Fixing errors in name mappings

2015-06-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master a777eb04b -> e84545fa7 [HOTFIX] Fixing errors in name mappings Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e84545fa Tree: http://git-wip-us.apache.org/repos/asf/spa