spark git commit: [SPARK-10786][SQL] Take the whole statement to generate the CommandProcessor

Repository: spark Updated Branches: refs/heads/master ea4a3e7d0 -> 767522dc4 [SPARK-10786][SQL] Take the whole statement to generate the CommandProcessor In the now implementation of `SparkSQLCLIDriver.scala`: `val proc: CommandProcessor = CommandProcessorFactory.get(Array(tokens(0)),

spark git commit: [SPARK-10997][CORE] Add "client mode" to netty rpc env.

Repository: spark Updated Branches: refs/heads/master a930e624e -> 71d1c907d [SPARK-10997][CORE] Add "client mode" to netty rpc env. "Client mode" means the RPC env will not listen for incoming connections. This allows certain processes in the Spark stack (such as Executors or tha YARN

spark git commit: [SPARK-9817][YARN] Improve the locality calculation of containers by taking pending container requests into consideraion

Repository: spark Updated Branches: refs/heads/master 74ba95228 -> a930e624e [SPARK-9817][YARN] Improve the locality calculation of containers by taking pending container requests into consideraion This is a follow-up PR to further improve the locality calculation by considering the pending

spark git commit: [SPARK-11456][TESTS] Remove deprecated junit.framework in Java tests

Repository: spark Updated Branches: refs/heads/master f92f334ca -> b3aedca6b [SPARK-11456][TESTS] Remove deprecated junit.framework in Java tests Replace use of `junit.framework` with `org.junit`, and touch up tests in question Author: Sean Owen Closes #9411 from

spark git commit: [SPARK-9034][SQL] Reflect field names defined in GenericUDTF

Repository: spark Updated Branches: refs/heads/master 9cf56c96b -> c34c27fe9 [SPARK-9034][SQL] Reflect field names defined in GenericUDTF Hive GenericUDTF#initialize() defines field names in a returned schema though, the current HiveGenericUDTF drops these names. We might need to reflect

spark git commit: [SPARK-11469][SQL] Allow users to define nondeterministic udfs.

Repository: spark Updated Branches: refs/heads/master efaa4721b -> 9cf56c96b [SPARK-11469][SQL] Allow users to define nondeterministic udfs. This is the first task (https://issues.apache.org/jira/browse/SPARK-11469) of https://issues.apache.org/jira/browse/SPARK-11438 Author: Yin Huai

spark git commit: [SPARK-11371] Make "mean" an alias for "avg" operator

Repository: spark Updated Branches: refs/heads/master 33ae7a35d -> db11ee5e5 [SPARK-11371] Make "mean" an alias for "avg" operator >From Reynold in the thread 'Exception when using some aggregate operators' >(http://search-hadoop.com/m/q3RTt0xFr22nXB4/): I don't think these are bugs. The

spark git commit: [SPARK-11271][SPARK-11016][CORE] Use Spark BitSet instead of RoaringBitmap to reduce memory usage

Repository: spark Updated Branches: refs/heads/master e963070c1 -> e209fa271 [SPARK-11271][SPARK-11016][CORE] Use Spark BitSet instead of RoaringBitmap to reduce memory usage JIRA: https://issues.apache.org/jira/browse/SPARK-11271 As reported in the JIRA ticket, when there are too many

spark git commit: [SPARK-11413][BUILD] Bump joda-time version to 2.9 for java 8 and s3

Repository: spark Updated Branches: refs/heads/master e209fa271 -> ea4a3e7d0 [SPARK-11413][BUILD] Bump joda-time version to 2.9 for java 8 and s3 It's a known issue that joda-time before 2.8.1 is incompatible with java 1.8u60 or later, which causes s3 request to fail. This affects Spark when

spark git commit: [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.4 backport)

Repository: spark Updated Branches: refs/heads/branch-1.4 ef42ce613 -> 00efa3ced [SPARK-11424] Guard against double-close() of RecordReaders (branch-1.4 backport) This is a branch-1.4 backport of #9382, a fix for SPARK-11424. Author: Josh Rosen Closes #9388 from

spark git commit: [SPARK-11343][ML] Allow float and double prediction/label columns in RegressionEvaluator

Repository: spark Updated Branches: refs/heads/master ecfb3e73f -> ec03866a7 [SPARK-11343][ML] Allow float and double prediction/label columns in RegressionEvaluator mengxr, felixcheung This pull request just relaxes the type of the prediction/label columns to be float and double.

spark git commit: [SPARK-10592] [ML] [PySpark] Deprecate weights and use coefficients instead in ML models

Repository: spark Updated Branches: refs/heads/master ec03866a7 -> c020f7d9d [SPARK-10592] [ML] [PySpark] Deprecate weights and use coefficients instead in ML models Deprecated in `LogisticRegression` and `LinearRegression` Author: vectorijk Closes #9311 from

spark git commit: [SPARK-10286][ML][PYSPARK][DOCS] Add @since annotation to pyspark.ml.param and pyspark.ml.*

Repository: spark Updated Branches: refs/heads/master 2804674a7 -> ecfb3e73f [SPARK-10286][ML][PYSPARK][DOCS] Add @since annotation to pyspark.ml.param and pyspark.ml.* Author: lihao Closes #9275 from lidinghao/SPARK-10286. Project:

spark git commit: [SPARK-11236] [TEST-MAVEN] [TEST-HADOOP1.0] [CORE] Update Tachyon dependency 0.7.1 -> 0.8.1

Repository: spark Updated Branches: refs/heads/master c020f7d9d -> 476f4348e [SPARK-11236] [TEST-MAVEN] [TEST-HADOOP1.0] [CORE] Update Tachyon dependency 0.7.1 -> 0.8.1 This is a reopening of #9204 which failed hadoop1 sbt tests. With the original PR, a classpath issue would occur due to

spark git commit: [SPARK-11311][SQL] spark cannot describe temporary functions

Repository: spark Updated Branches: refs/heads/master 767522dc4 -> 74ba95228 [SPARK-11311][SQL] spark cannot describe temporary functions When describe temporary function, spark would return 'Unable to find function', this is not right. Author: Daoyuan Wang Closes

spark git commit: [SPARK-11329][SQL] Support star expansion for structs.

Repository: spark Updated Branches: refs/heads/master 2cef1bb0b -> 9cb5c731d [SPARK-11329][SQL] Support star expansion for structs. 1. Supporting expanding structs in Projections. i.e. "SELECT s.*" where s is a struct type. This is fixed by allowing the expand function to handle structs

spark git commit: [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization

Repository: spark Updated Branches: refs/heads/master 9cb5c731d -> efaa4721b [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization Changes the personalized pagerank initialization to be non-uniform. Author: Yves Raimond Closes #9386 from

spark git commit: [MINOR][ML] removed the old `getModelWeights` function

Repository: spark Updated Branches: refs/heads/master 476f4348e -> 21ad84623 [MINOR][ML] removed the old `getModelWeights` function Removed the old `getModelWeights` function which was private and renamed into `getModelCoefficients` Author: DB Tsai Closes #9426 from

spark git commit: [SPARK-5354][SQL] Cached tables should preserve partitioning and ord…

Repository: spark Updated Branches: refs/heads/master 21ad84623 -> 2cef1bb0b [SPARK-5354][SQL] Cached tables should preserve partitioning and ord… …ering. For cached tables, we can just maintain the partitioning and ordering from the source relation. Author: Nong Li