spark git commit: [SPARK-7886] Added unit test for HAVING aggregate pushdown.

2015-06-10 Thread lian
Repository: spark Updated Branches: refs/heads/master 57c60c5be - e90035e67 [SPARK-7886] Added unit test for HAVING aggregate pushdown. This is a followup to #6712. Author: Reynold Xin r...@databricks.com Closes #6739 from rxin/6712-followup and squashes the following commits: fd9acfb

spark git commit: [SPARK-7886] Use FunctionRegistry for built-in expressions in HiveContext.

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 778f3ca81 - 57c60c5be [SPARK-7886] Use FunctionRegistry for built-in expressions in HiveContext. This builds on #6710 and also uses FunctionRegistry for function lookup in HiveContext. Author: Reynold Xin r...@databricks.com Closes

spark git commit: [SPARK-8215] [SPARK-8212] [SQL] add leaf math expression for e and pi

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master e90035e67 - c6ba7cca3 [SPARK-8215] [SPARK-8212] [SQL] add leaf math expression for e and pi Author: Daoyuan Wang daoyuan.w...@intel.com Closes #6716 from adrian-wang/epi and squashes the following commits: e2e8dbd [Daoyuan Wang] move

spark git commit: [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 7b88e6a1e - 28e8a6ea6 [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable Read number of threads for RBackend from configuration. [SPARK-8282] #comment Linking with JIRA Author: Hossein hoss...@databricks.com

spark git commit: [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 38112905b - 30ebf1a23 [SPARK-8282] [SPARKR] Make number of threads used in RBackend configurable Read number of threads for RBackend from configuration. [SPARK-8282] #comment Linking with JIRA Author: Hossein hoss...@databricks.com

spark git commit: [SPARK-7756] CORE RDDOperationScope fix for IBM Java

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 30ebf1a23 - 19e30b48f [SPARK-7756] CORE RDDOperationScope fix for IBM Java IBM Java has an extra method when we do getStackTrace(): this is getStackTraceImpl, a native method. This causes two tests to fail within DStreamScopeSuite when

spark git commit: [SPARK-7756] CORE RDDOperationScope fix for IBM Java

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 28e8a6ea6 - 568d1d51d [SPARK-7756] CORE RDDOperationScope fix for IBM Java IBM Java has an extra method when we do getStackTrace(): this is getStackTraceImpl, a native method. This causes two tests to fail within DStreamScopeSuite

spark git commit: [SPARK-7261] [CORE] Change default log level to WARN in the REPL

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master e90c9d92d - 80043e9e7 [SPARK-7261] [CORE] Change default log level to WARN in the REPL 1. Add `log4j-defaults-repl.properties` that has log level WARN. 2. When logging is initialized, check whether inside the REPL. If so, use

spark git commit: [SPARK-8273] Driver hangs up when yarn shutdown in client mode

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master cb871c44c - 5014d0ed7 [SPARK-8273] Driver hangs up when yarn shutdown in client mode In client mode, if yarn was shut down with spark application running, the application will hang up after several retries(default: 30) because the

spark git commit: [SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 80043e9e7 - cb871c44c [SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add it back so process launched by

spark git commit: [SPARK-8273] Driver hangs up when yarn shutdown in client mode

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 568d1d51d - 2846a357f [SPARK-8273] Driver hangs up when yarn shutdown in client mode In client mode, if yarn was shut down with spark application running, the application will hang up after several retries(default: 30) because the

spark git commit: [SPARK-5479] [YARN] Handle --py-files correctly in YARN.

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 8f7308f9c - 38112905b [SPARK-5479] [YARN] Handle --py-files correctly in YARN. The bug description is a little misleading: the actual issue is that .py files are not handled correctly when distributed by YARN. They're added to

spark git commit: [SPARK-7527] [CORE] Fix createNullValue to return the correct null values and REPL mode detection

2015-06-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 19e30b48f - e90c9d92d [SPARK-7527] [CORE] Fix createNullValue to return the correct null values and REPL mode detection The root cause of SPARK-7527 is `createNullValue` returns an incompatible value `Byte(0)` for `char` and `boolean`.

spark git commit: [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 2b550a521 - 8f7308f9c [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide Author: Cheng Lian l...@databricks.com Closes #6749 from liancheng/java-sample-fix and squashes the following commits: 5b44585 [Cheng Lian]

spark git commit: [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 a0a7f2f92 - 7b88e6a1e [SQL] [MINOR] Fixes a minor Java example error in SQL programming guide Author: Cheng Lian l...@databricks.com Closes #6749 from liancheng/java-sample-fix and squashes the following commits: 5b44585 [Cheng Lian]

spark git commit: [SPARK-7996] Deprecate the developer api SparkEnv.actorSystem

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master c6ba7cca3 - 2b550a521 [SPARK-7996] Deprecate the developer api SparkEnv.actorSystem Changed ```SparkEnv.actorSystem``` to be a function such that we can use the deprecated flag with it and added a deprecated message. Author: Ilya Ganelin

spark git commit: [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm

2015-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.4 2846a357f - 59fc3f197 [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm Test cases for both StreamingLinearRegression and StreamingLogisticRegression, and code fix. Edit: This contribution is my original work and

spark git commit: [SPARK-2774] Set preferred locations for reduce tasks

2015-06-10 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master 5014d0ed7 - 96a7c888d [SPARK-2774] Set preferred locations for reduce tasks Set preferred locations for reduce tasks. The basic design is that we maintain a map from reducerId to a list of (sizes, locations) for each shuffle. We then set

spark git commit: [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm

2015-06-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 96a7c888d - b928f5438 [SPARK-8200] [MLLIB] Check for empty RDDs in StreamingLinearAlgorithm Test cases for both StreamingLinearRegression and StreamingLogisticRegression, and code fix. Edit: This contribution is my original work and I

spark git commit: [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 59fc3f197 - 5c05b5c0d [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first case cs CombineSum(expr) = val calcType = expr.dataType expr.dataType match { case

spark git commit: [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 37719e0cd - 6a47114bc [SPARK-8285] [SQL] CombineSum should be calculated as unlimited decimal first case cs CombineSum(expr) = val calcType = expr.dataType expr.dataType match { case DecimalType.Fixed(_,

svn commit: r9331 - /dev/spark/spark-1.4.0-rc4/ /release/spark/spark-1.4.0/

2015-06-10 Thread pwendell
Author: pwendell Date: Wed Jun 10 23:18:01 2015 New Revision: 9331 Log: Adding Spark release 1.4.0 Added: release/spark/spark-1.4.0/ - copied from r9330, dev/spark/spark-1.4.0-rc4/ Removed: dev/spark/spark-1.4.0-rc4/

spark git commit: [SPARK-8164] transformExpressions should support nested expression sequence

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6a47114bc - 4e42842e8 [SPARK-8164] transformExpressions should support nested expression sequence Currently we only support `Seq[Expression]`, we should handle cases like `Seq[Seq[Expression]]` so that we can remove the unnecessary

svn commit: r9330 - /dev/spark/spark-1.4.0-rc4/

2015-06-10 Thread pwendell
Author: pwendell Date: Wed Jun 10 23:16:15 2015 New Revision: 9330 Log: Adding Spark 1.4.0 RC4 Added: dev/spark/spark-1.4.0-rc4/ dev/spark/spark-1.4.0-rc4/spark-1.4.0-bin-cdh4.tgz (with props) dev/spark/spark-1.4.0-rc4/spark-1.4.0-bin-cdh4.tgz.asc (with props)

spark git commit: [SPARK-8189] [SQL] use Long for TimestampType in SQL

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master b928f5438 - 37719e0cd [SPARK-8189] [SQL] use Long for TimestampType in SQL This PR change to use Long as internal type for TimestampType for efficiency, which means it will the precision below 100ns. Author: Davies Liu

spark git commit: [HOTFIX] Fixing errors in name mappings

2015-06-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master a777eb04b - e84545fa7 [HOTFIX] Fixing errors in name mappings Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e84545fa Tree:

spark git commit: [SPARK-8248][SQL] string function: length

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4e42842e8 - 9fe3adcce [SPARK-8248][SQL] string function: length Author: Cheng Hao hao.ch...@intel.com Closes #6724 from chenghao-intel/length and squashes the following commits: aaa3c31 [Cheng Hao] revert the additional change 97148a9

spark git commit: [SPARK-8217] [SQL] math function log2

2015-06-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9fe3adcce - 2758ff0a9 [SPARK-8217] [SQL] math function log2 Author: Daoyuan Wang daoyuan.w...@intel.com This patch had conflicts when merged, resolved by Committer: Reynold Xin r...@databricks.com Closes #6718 from adrian-wang/udflog2

spark git commit: [HOTFIX] Adding more contributor name bindings

2015-06-10 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 2758ff0a9 - a777eb04b [HOTFIX] Adding more contributor name bindings Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a777eb04 Tree:

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc4 [deleted] 22596c534 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0 [created] 22596c534 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc2 [deleted] 03fb26a3e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc1 [deleted] 777a08166 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

Git Push Summary

2015-06-10 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc3 [deleted] dd109a874 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org