spark git commit: [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext

Repository: spark Updated Branches: refs/heads/branch-2.0 69327667d -> 27f26a39d [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext ## What changes were proposed in this pull request? Override the existing SparkContext is the

spark git commit: [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext

Repository: spark Updated Branches: refs/heads/master b120fba6a -> 01e7b9c85 [SPARK-15345][SQL][PYSPARK] SparkSession's conf doesn't take effect when this already an existing SparkContext ## What changes were proposed in this pull request? Override the existing SparkContext is the provided

spark git commit: [MINOR][CORE][TEST] Update obsolete `takeSample` test case.

Repository: spark Updated Branches: refs/heads/branch-2.0 1bb0aa4b0 -> 2574abea0 [MINOR][CORE][TEST] Update obsolete `takeSample` test case. ## What changes were proposed in this pull request? This PR fixes some obsolete comments and assertion in `takeSample` testcase of `RDDSuite.scala`.

spark git commit: [MINOR][CORE][TEST] Update obsolete `takeSample` test case.

Repository: spark Updated Branches: refs/heads/master 784cc07d1 -> be99a99fe [MINOR][CORE][TEST] Update obsolete `takeSample` test case. ## What changes were proposed in this pull request? This PR fixes some obsolete comments and assertion in `takeSample` testcase of `RDDSuite.scala`. ##

spark git commit: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1

Repository: spark Updated Branches: refs/heads/branch-2.0 988d4dbf4 -> 1bb0aa4b0 [SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1 ## What changes were proposed in this pull request? spark.sql("CREATE FUNCTION myfunc AS 'com.haizhi.bdp.udf.UDFGetGeoCode'") throws

spark git commit: [SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1

Repository: spark Updated Branches: refs/heads/master a313a5ae7 -> 784cc07d1 [SPARK-15388][SQL] Fix spark sql CREATE FUNCTION with hive 1.2.1 ## What changes were proposed in this pull request? spark.sql("CREATE FUNCTION myfunc AS 'com.haizhi.bdp.udf.UDFGetGeoCode'") throws

spark git commit: [SPARK-15397][SQL] fix string udf locate as hive

Repository: spark Updated Branches: refs/heads/branch-2.0 1890f5fdf -> 6adbc0613 [SPARK-15397][SQL] fix string udf locate as hive ## What changes were proposed in this pull request? in hive, `locate("aa", "aaa", 0)` would yield 0, `locate("aa", "aaa", 1)` would yield 1 and `locate("aa",

spark git commit: [SPARK-15397][SQL] fix string udf locate as hive

Repository: spark Updated Branches: refs/heads/master de726b0d5 -> d642b2735 [SPARK-15397][SQL] fix string udf locate as hive ## What changes were proposed in this pull request? in hive, `locate("aa", "aaa", 0)` would yield 0, `locate("aa", "aaa", 1)` would yield 1 and `locate("aa", "aaa",

spark git commit: Revert "[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB"

Repository: spark Updated Branches: refs/heads/branch-2.0 d0bcec157 -> 1890f5fdf Revert "[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB" This reverts commit d0bcec157d2bd2ed4eff848f831841bef4745904. Project:

spark git commit: Revert "[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB"

Repository: spark Updated Branches: refs/heads/master fa244e5a9 -> de726b0d5 Revert "[SPARK-15285][SQL] Generated SpecificSafeProjection.apply method grows beyond 64 KB" This reverts commit fa244e5a90690d6a31be50f2aa203ae1a2e9a1cf. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

Repository: spark Updated Branches: refs/heads/branch-2.0 220b9a08e -> f3162b96d [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code ## What changes were proposed in this pull request? Replace SQLContext and

spark git commit: [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code

Repository: spark Updated Branches: refs/heads/master 5afd927a4 -> a15ca5533 [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code ## What changes were proposed in this pull request? Replace SQLContext and

spark git commit: [SPARK-15311][SQL] Disallow DML on Regular Tables when Using In-Memory Catalog

Repository: spark Updated Branches: refs/heads/branch-2.0 3def56120 -> 220b9a08e [SPARK-15311][SQL] Disallow DML on Regular Tables when Using In-Memory Catalog What changes were proposed in this pull request? So far, when using In-Memory Catalog, we allow DDL operations for the tables.

spark git commit: [SPARK-15311][SQL] Disallow DML on Regular Tables when Using In-Memory Catalog

Repository: spark Updated Branches: refs/heads/master 01659bc50 -> 5afd927a4 [SPARK-15311][SQL] Disallow DML on Regular Tables when Using In-Memory Catalog What changes were proposed in this pull request? So far, when using In-Memory Catalog, we allow DDL operations for the tables.

spark git commit: [SPARK-15279][SQL] Catch conflicting SerDe when creating table

tch: - `ROW FORMAT DELIMITED` is only compatible with `TEXTFILE` - `ROW FORMAT SERDE` is only compatible with `TEXTFILE`, `RCFILE` and `SEQUENCEFILE` ## How was this patch tested? New tests in `DDLCommandSuite`. Author: Andrew Or <and...@databricks.com> Closes #13068 from andrewor14/r

spark git commit: [SPARK-15279][SQL] Catch conflicting SerDe when creating table

tch: - `ROW FORMAT DELIMITED` is only compatible with `TEXTFILE` - `ROW FORMAT SERDE` is only compatible with `TEXTFILE`, `RCFILE` and `SEQUENCEFILE` ## How was this patch tested? New tests in `DDLCommandSuite`. Author: Andrew Or <and...@databricks.com> Closes #13068 from andrewor14/r

spark git commit: [SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present

Repository: spark Updated Branches: refs/heads/branch-2.0 684167862 -> c7e013f18 [SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present ## What changes were proposed in this pull request? When PySpark shell cannot find HiveConf, it will fallback to

spark git commit: [SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present

Repository: spark Updated Branches: refs/heads/master 127bf1bb0 -> 021c19702 [SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present ## What changes were proposed in this pull request? When PySpark shell cannot find HiveConf, it will fallback to create a

spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in examples

LIB` is not recommended to use now, so examples in `MLLIB` are ignored in this PR. `StreamingContext` can not be directly obtained from `SparkSession`, so example in `Streaming` are ignored too. cc andrewor14 ## How was this patch tested? manual tests with spark-submit Author: Zheng RuiFeng <

spark git commit: [SPARK-15031][EXAMPLE] Use SparkSession in examples

LIB` is not recommended to use now, so examples in `MLLIB` are ignored in this PR. `StreamingContext` can not be directly obtained from `SparkSession`, so example in `Streaming` are ignored too. cc andrewor14 ## How was this patch tested? manual tests with spark-submit Author: Zheng RuiFeng <

spark git commit: [SPARK-15417][SQL][PYTHON] PySpark shell always uses in-memory catalog

rew Or <and...@databricks.com> Closes #13203 from andrewor14/fix-pyspark-shell. (cherry picked from commit c32b1b162e7e5ecc5c823f79ba9f23cbd1407dbf) Signed-off-by: Andrew Or <and...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apach

spark git commit: [SPARK-15417][SQL][PYTHON] PySpark shell always uses in-memory catalog

rew Or <and...@databricks.com> Closes #13203 from andrewor14/fix-pyspark-shell. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c32b1b16 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c32b1b16 Diff: http://git-wip-us.apache.

spark git commit: [SPARK-15421][SQL] Validate DDL property values

uld throw exceptions instead. ## How was this patch tested? `DDLCommandSuite` Author: Andrew Or <and...@databricks.com> Closes #13205 from andrewor14/ddl-prop-values. (cherry picked from commit 257375019266ab9e3c320e33026318cc31f58ada) Signed-off-by: Andrew Or <and...@databricks.com>

spark git commit: [SPARK-15421][SQL] Validate DDL property values

row exceptions instead. ## How was this patch tested? `DDLCommandSuite` Author: Andrew Or <and...@databricks.com> Closes #13205 from andrewor14/ddl-prop-values. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2573

spark git commit: [HOTFIX] Add back intended change from SPARK-15392

Repository: spark Updated Branches: refs/heads/branch-2.0 2ef645724 -> 612866473 [HOTFIX] Add back intended change from SPARK-15392 This was accidentally reverted in f8d0177. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [HOTFIX] Test compilation error from 52b967f

Repository: spark Updated Branches: refs/heads/branch-2.0 2126fb0c2 -> 1fc0f95eb [HOTFIX] Test compilation error from 52b967f Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1fc0f95e Tree:

spark git commit: [SPARK-15392][SQL] fix default value of size estimation of logical plan

Repository: spark Updated Branches: refs/heads/master 4e3cb7a5d -> 5ccecc078 [SPARK-15392][SQL] fix default value of size estimation of logical plan ## What changes were proposed in this pull request? We use autoBroadcastJoinThreshold + 1L as the default value of size estimation, that is

spark git commit: [SPARK-15392][SQL] fix default value of size estimation of logical plan

Repository: spark Updated Branches: refs/heads/branch-2.0 62e5158f1 -> d1b5df83d [SPARK-15392][SQL] fix default value of size estimation of logical plan ## What changes were proposed in this pull request? We use autoBroadcastJoinThreshold + 1L as the default value of size estimation, that

spark git commit: [SPARK-15317][CORE] Don't store accumulators for every task in listeners

Repository: spark Updated Branches: refs/heads/master 6ac1c3a04 -> 4e3cb7a5d [SPARK-15317][CORE] Don't store accumulators for every task in listeners ## What changes were proposed in this pull request? In general, the Web UI doesn't need to store the Accumulator/AccumulableInfo for every

spark git commit: [SPARK-15317][CORE] Don't store accumulators for every task in listeners

Repository: spark Updated Branches: refs/heads/branch-2.0 4f8639f9d -> 62e5158f1 [SPARK-15317][CORE] Don't store accumulators for every task in listeners ## What changes were proposed in this pull request? In general, the Web UI doesn't need to store the Accumulator/AccumulableInfo for

spark git commit: [SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output

Repository: spark Updated Branches: refs/heads/master e71cd96bf -> 6ac1c3a04 [SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output ## What changes were proposed in this pull request? This PR is a follow-up of #13079. It replaces `hasUnsupportedFeatures: Boolean` in

spark git commit: [SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output

Repository: spark Updated Branches: refs/heads/branch-2.0 97fd9a09c -> 4f8639f9d [SPARK-14346][SQL] Lists unsupported Hive features in SHOW CREATE TABLE output ## What changes were proposed in this pull request? This PR is a follow-up of #13079. It replaces `hasUnsupportedFeatures: Boolean`

spark git commit: [SPARK-15322][SQL][FOLLOW-UP] Update deprecated accumulator usage into accumulatorV2

Repository: spark Updated Branches: refs/heads/branch-2.0 9c817d027 -> 554e0f30a [SPARK-15322][SQL][FOLLOW-UP] Update deprecated accumulator usage into accumulatorV2 ## What changes were proposed in this pull request? This PR corrects another case that uses deprecated

spark git commit: [SPARK-14603][SQL][FOLLOWUP] Verification of Metadata Operations by Session Catalog

Repository: spark Updated Branches: refs/heads/master 9308bf119 -> ef7a5e0bc [SPARK-14603][SQL][FOLLOWUP] Verification of Metadata Operations by Session Catalog What changes were proposed in this pull request? This follow-up PR is to address the remaining comments in

spark git commit: [SPARK-15387][SQL] SessionCatalog in SimpleAnalyzer does not need to make database directory.

Repository: spark Updated Branches: refs/heads/branch-2.0 96a473a11 -> 9c817d027 [SPARK-15387][SQL] SessionCatalog in SimpleAnalyzer does not need to make database directory. ## What changes were proposed in this pull request? After #12871 is fixed, we are forced to make

spark git commit: [SPARK-15387][SQL] SessionCatalog in SimpleAnalyzer does not need to make database directory.

Repository: spark Updated Branches: refs/heads/master ad182086c -> faafd1e9d [SPARK-15387][SQL] SessionCatalog in SimpleAnalyzer does not need to make database directory. ## What changes were proposed in this pull request? After #12871 is fixed, we are forced to make `/user/hive/warehouse`

spark git commit: [SPARK-15300] Fix writer lock conflict when remove a block

Repository: spark Updated Branches: refs/heads/master ef7a5e0bc -> ad182086c [SPARK-15300] Fix writer lock conflict when remove a block ## What changes were proposed in this pull request? A writer lock could be acquired when 1) create a new block 2) remove a block 3) evict a block to disk.

spark git commit: [SPARK-14603][SQL][FOLLOWUP] Verification of Metadata Operations by Session Catalog

Repository: spark Updated Branches: refs/heads/branch-2.0 2604eadcf -> 496f6d0fc [SPARK-14603][SQL][FOLLOWUP] Verification of Metadata Operations by Session Catalog What changes were proposed in this pull request? This follow-up PR is to address the remaining comments in

spark git commit: [SPARK-14684][SPARK-15277][SQL] Partition Spec Validation in SessionCatalog and Checking Partition Spec Existence Before Dropping

Repository: spark Updated Branches: refs/heads/branch-2.0 68617e1ad -> 9c5c9013d [SPARK-14684][SPARK-15277][SQL] Partition Spec Validation in SessionCatalog and Checking Partition Spec Existence Before Dropping What changes were proposed in this pull request? ~~Currently, multiple

spark git commit: [SPARK-14684][SPARK-15277][SQL] Partition Spec Validation in SessionCatalog and Checking Partition Spec Existence Before Dropping

Repository: spark Updated Branches: refs/heads/master 470de743e -> be617f3d0 [SPARK-14684][SPARK-15277][SQL] Partition Spec Validation in SessionCatalog and Checking Partition Spec Existence Before Dropping What changes were proposed in this pull request? ~~Currently, multiple

spark git commit: [HOTFIX] SQL test compilation error from merge conflict

Repository: spark Updated Branches: refs/heads/branch-2.0 9098b1a17 -> b3f145442 [HOTFIX] SQL test compilation error from merge conflict Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b3f14544 Tree:

spark git commit: [SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in examples

Repository: spark Updated Branches: refs/heads/master ba5487c06 -> 9e266d07a [SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in examples ## What changes were proposed in this pull request? 1, Use `SparkSession` according to

spark git commit: [SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in examples

Repository: spark Updated Branches: refs/heads/branch-2.0 7d187539e -> 86acb5efd [SPARK-15031][SPARK-15134][EXAMPLE][DOC] Use SparkSession and update indent in examples ## What changes were proposed in this pull request? 1, Use `SparkSession` according to

spark git commit: [SPARK-15072][SQL][PYSPARK] FollowUp: Remove SparkSession.withHiveSupport in PySpark

Repository: spark Updated Branches: refs/heads/branch-2.0 f8804bb10 -> 114be703d [SPARK-15072][SQL][PYSPARK] FollowUp: Remove SparkSession.withHiveSupport in PySpark ## What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/12851 Remove

spark git commit: [SPARK-15264][SPARK-15274][SQL] CSV Reader Error on Blank Column Names

Repository: spark Updated Branches: refs/heads/master f14c4ba00 -> 603f4453a [SPARK-15264][SPARK-15274][SQL] CSV Reader Error on Blank Column Names ## What changes were proposed in this pull request? When a CSV begins with: - `,,` OR - `"","",` meaning that the first column names are either

spark git commit: [SPARK-15276][SQL] CREATE TABLE with LOCATION should imply EXTERNAL

kes the `EXTERNAL` field optional. This is related to #13032. ## How was this patch tested? New test in `DDLCommandSuite`. Author: Andrew Or <and...@databricks.com> Closes #13060 from andrewor14/location-implies-external. (cherry picked from commit f14c4ba001fbdbcc9faa46896f1f9d08a7d06609) S

spark git commit: [SPARK-15276][SQL] CREATE TABLE with LOCATION should imply EXTERNAL

kes the `EXTERNAL` field optional. This is related to #13032. ## How was this patch tested? New test in `DDLCommandSuite`. Author: Andrew Or <and...@databricks.com> Closes #13060 from andrewor14/location-implies-external. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http:

[3/3] spark git commit: [SPARK-13522][CORE] Fix the exit log place for heartbeat

[SPARK-13522][CORE] Fix the exit log place for heartbeat ## What changes were proposed in this pull request? Just fixed the log place introduced by #11401 ## How was this patch tested? unit tests. Author: Shixiong Zhu Closes #11432 from

[1/3] spark git commit: [SPARK-13519][CORE] Driver should tell Executor to stop itself when cleaning executor's state

Repository: spark Updated Branches: refs/heads/branch-1.6 d1654864a -> ced71d353 [SPARK-13519][CORE] Driver should tell Executor to stop itself when cleaning executor's state ## What changes were proposed in this pull request? When the driver removes an executor's state, the connection

[2/3] spark git commit: [SPARK-13522][CORE] Executor should kill itself when it's unable to heartbeat to driver more than N times

[SPARK-13522][CORE] Executor should kill itself when it's unable to heartbeat to driver more than N times ## What changes were proposed in this pull request? Sometimes, network disconnection event won't be triggered for other potential race conditions that we may not have thought of, then the

spark git commit: [SPARK-15249][SQL] Use FunctionResource instead of (String, String) in CreateFunction and CatalogFunction for resource

Repository: spark Updated Branches: refs/heads/master 9533f5390 -> da02d006b [SPARK-15249][SQL] Use FunctionResource instead of (String, String) in CreateFunction and CatalogFunction for resource Use FunctionResource instead of (String, String) in CreateFunction and CatalogFunction for

spark git commit: [SPARK-15037][HOTFIX] Don't create 2 SparkSessions in constructor

get the `SparkContext` from it. This ends up creating 2 `SparkSession`s from one call, which is definitely not what we want. ## How was this patch tested? Jenkins. Author: Andrew Or <and...@databricks.com> Closes #13031 from andrewor14/sql-test. Project: http://git-wip-us.apache.org/

spark git commit: [SPARK-15037][HOTFIX] Replace `sqlContext` and `sparkSession` with `spark`.

Repository: spark Updated Branches: refs/heads/master cddb9da07 -> db3b4a201 [SPARK-15037][HOTFIX] Replace `sqlContext` and `sparkSession` with `spark`. This replaces `sparkSession` with `spark` in CatalogSuite.scala. Pass the Jenkins tests. Author: Dongjoon Hyun

spark git commit: [SPARK-15037][HOTFIX] Replace `sqlContext` and `sparkSession` with `spark`.

Repository: spark Updated Branches: refs/heads/branch-2.0 42db140c5 -> bd7fd14c9 [SPARK-15037][HOTFIX] Replace `sqlContext` and `sparkSession` with `spark`. This replaces `sparkSession` with `spark` in CatalogSuite.scala. Pass the Jenkins tests. Author: Dongjoon Hyun

spark git commit: [HOTFIX] SQL test compilation error from merge conflict

Repository: spark Updated Branches: refs/heads/master 5c6b08557 -> cddb9da07 [HOTFIX] SQL test compilation error from merge conflict Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/cddb9da0 Tree:

spark git commit: [SPARK-14603][SQL] Verification of Metadata Operations by Session Catalog

Repository: spark Updated Branches: refs/heads/branch-2.0 5bf74b44d -> 42db140c5 [SPARK-14603][SQL] Verification of Metadata Operations by Session Catalog Since we cannot really trust if the underlying external catalog can throw exceptions when there is an invalid metadata operation, let's

spark git commit: [SPARK-14603][SQL] Verification of Metadata Operations by Session Catalog

Repository: spark Updated Branches: refs/heads/master ed0b4070f -> 5c6b08557 [SPARK-14603][SQL] Verification of Metadata Operations by Session Catalog Since we cannot really trust if the underlying external catalog can throw exceptions when there is an invalid metadata operation, let's do it

[07/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/mllib/src/test/scala/org/apache/spark/ml/feature/StopWordsRemoverSuite.scala -- diff --git a/mllib/src/test/scala/org/apache/spark/ml/feature/StopWordsRemoverSuite.scala

[08/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/mllib/src/test/java/org/apache/spark/mllib/regression/JavaIsotonicRegressionSuite.java -- diff --git

[02/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/sql/core/src/test/scala/org/apache/spark/sql/sources/PartitionedWriteSuite.scala -- diff --git

[02/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/sql/core/src/test/scala/org/apache/spark/sql/sources/PartitionedWriteSuite.scala -- diff --git

[05/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala -- diff --git a/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala

[10/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

[SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites ## What changes were proposed in this pull request? Use SparkSession instead of SQLContext in Scala/Java TestSuites as this PR already very big working Python TestSuites in a diff PR. ## How was this patch

[06/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/sql/core/src/test/java/test/org/apache/spark/sql/sources/JavaDatasetAggregatorSuiteBase.java -- diff --git

[04/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala -- diff --git

[07/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/mllib/src/test/scala/org/apache/spark/ml/feature/StopWordsRemoverSuite.scala -- diff --git a/mllib/src/test/scala/org/apache/spark/ml/feature/StopWordsRemoverSuite.scala

[01/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

Repository: spark Updated Branches: refs/heads/branch-2.0 19a9c23c2 -> 5bf74b44d http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala

[09/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/5bf74b44/mllib/src/test/java/org/apache/spark/ml/regression/JavaLinearRegressionSuite.java -- diff --git

[04/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala -- diff --git

[08/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/mllib/src/test/java/org/apache/spark/mllib/regression/JavaIsotonicRegressionSuite.java -- diff --git

[03/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala -- diff --git

[06/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/sql/core/src/test/java/test/org/apache/spark/sql/sources/JavaDatasetAggregatorSuiteBase.java -- diff --git

[01/10] spark git commit: [SPARK-15037][SQL][MLLIB] Use SparkSession instead of SQLContext in Scala/Java TestSuites

Repository: spark Updated Branches: refs/heads/master bcfee153b -> ed0b4070f http://git-wip-us.apache.org/repos/asf/spark/blob/ed0b4070/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/AggregationQuerySuite.scala --

spark git commit: [SPARK-12837][CORE] reduce network IO for accumulators

Repository: spark Updated Branches: refs/heads/branch-2.0 af12b0a50 -> 19a9c23c2 [SPARK-12837][CORE] reduce network IO for accumulators Sending un-updated accumulators back to driver makes no sense, as merging a zero value accumulator is a no-op. We should only send back updated

spark git commit: [SPARK-12837][CORE] reduce network IO for accumulators

Repository: spark Updated Branches: refs/heads/master 0b9cae424 -> bcfee153b [SPARK-12837][CORE] reduce network IO for accumulators Sending un-updated accumulators back to driver makes no sense, as merging a zero value accumulator is a no-op. We should only send back updated accumulators,

spark git commit: [SPARK-10653][CORE] Remove unnecessary things from SparkEnv

Repository: spark Updated Branches: refs/heads/branch-2.0 e3f000a36 -> 40d24686a [SPARK-10653][CORE] Remove unnecessary things from SparkEnv ## What changes were proposed in this pull request? Removed blockTransferService and sparkFilesDir from SparkEnv since they're rarely used and don't

spark git commit: [SPARK-15166][SQL] Move some hive-specific code from SparkSession

sts. Author: Andrew Or <and...@databricks.com> Closes #12941 from andrewor14/move-code. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7bf9b120 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7bf9b120 Diff: http:

spark git commit: [SPARK-15210][SQL] Add missing @DeveloperApi annotation in sql.types

Repository: spark Updated Branches: refs/heads/master f8aca5b4a -> dfdcab00c [SPARK-15210][SQL] Add missing @DeveloperApi annotation in sql.types add DeveloperApi annotation for `AbstractDataType` `MapType` `UserDefinedType` local build Author: Zheng RuiFeng Closes

spark git commit: [SPARK-15210][SQL] Add missing @DeveloperApi annotation in sql.types

Repository: spark Updated Branches: refs/heads/branch-2.0 c6d23b660 -> f81d25139 [SPARK-15210][SQL] Add missing @DeveloperApi annotation in sql.types add DeveloperApi annotation for `AbstractDataType` `MapType` `UserDefinedType` local build Author: Zheng RuiFeng

spark git commit: [SAPRK-15220][UI] add hyperlink to running application and completed application

Repository: spark Updated Branches: refs/heads/master ee6a8d7ea -> f8aca5b4a [SAPRK-15220][UI] add hyperlink to running application and completed application ## What changes were proposed in this pull request? Add hyperlink to "running application" and "completed application", so user can

spark git commit: [SAPRK-15220][UI] add hyperlink to running application and completed application

Repository: spark Updated Branches: refs/heads/branch-2.0 1d5615857 -> c6d23b660 [SAPRK-15220][UI] add hyperlink to running application and completed application ## What changes were proposed in this pull request? Add hyperlink to "running application" and "completed application", so user can

spark git commit: [MINOR][SQL] Enhance the exception message if checkpointLocation is not set

Repository: spark Updated Branches: refs/heads/master 6747171eb -> ee6a8d7ea [MINOR][SQL] Enhance the exception message if checkpointLocation is not set Enhance the exception message when `checkpointLocation` is not set, previously the message is: ``` java.util.NoSuchElementException:

spark git commit: [MINOR][SQL] Enhance the exception message if checkpointLocation is not set

Repository: spark Updated Branches: refs/heads/branch-2.0 3c6f686f9 -> 1d5615857 [MINOR][SQL] Enhance the exception message if checkpointLocation is not set Enhance the exception message when `checkpointLocation` is not set, previously the message is: ``` java.util.NoSuchElementException:

spark git commit: [SPARK-15067][YARN] YARN executors are launched with fixed perm gen size

Repository: spark Updated Branches: refs/heads/master e083db2e9 -> 6747171eb [SPARK-15067][YARN] YARN executors are launched with fixed perm gen size ## What changes were proposed in this pull request? Look for MaxPermSize arguments anywhere in an arg, to account for quoted args. See JIRA

spark git commit: [SPARK-15067][YARN] YARN executors are launched with fixed perm gen size

Repository: spark Updated Branches: refs/heads/branch-2.0 8f0ed2891 -> 3c6f686f9 [SPARK-15067][YARN] YARN executors are launched with fixed perm gen size ## What changes were proposed in this pull request? Look for MaxPermSize arguments anywhere in an arg, to account for quoted args. See

spark git commit: [SPARK-15225][SQL] Replace SQLContext with SparkSession in Encoder documentation

Repository: spark Updated Branches: refs/heads/master 65b4ab281 -> e083db2e9 [SPARK-15225][SQL] Replace SQLContext with SparkSession in Encoder documentation `Encoder`'s doc mentions `sqlContext.implicits._`. We should use `sparkSession.implicits._` instead now. Only doc update. Author:

spark git commit: [SPARK-15225][SQL] Replace SQLContext with SparkSession in Encoder documentation

Repository: spark Updated Branches: refs/heads/branch-2.0 1b4e99ff1 -> 8f0ed2891 [SPARK-15225][SQL] Replace SQLContext with SparkSession in Encoder documentation `Encoder`'s doc mentions `sqlContext.implicits._`. We should use `sparkSession.implicits._` instead now. Only doc update.

spark git commit: [SPARK-15223][DOCS] fix wrongly named config reference

Repository: spark Updated Branches: refs/heads/branch-2.0 6371197c6 -> 1b4e99ff1 [SPARK-15223][DOCS] fix wrongly named config reference ## What changes were proposed in this pull request? The configuration setting `spark.executor.logs.rolling.size.maxBytes` was changed to

spark git commit: [SPARK-15223][DOCS] fix wrongly named config reference

Repository: spark Updated Branches: refs/heads/master 2992a215c -> 65b4ab281 [SPARK-15223][DOCS] fix wrongly named config reference ## What changes were proposed in this pull request? The configuration setting `spark.executor.logs.rolling.size.maxBytes` was changed to

spark git commit: [MINOR][DOCS] Remove remaining sqlContext in documentation at examples

Repository: spark Updated Branches: refs/heads/branch-2.0 de6afc887 -> 6371197c6 [MINOR][DOCS] Remove remaining sqlContext in documentation at examples This PR removes `sqlContext` in examples. Actual usage was all replaced in https://github.com/apache/spark/pull/12809 but there are some in

spark git commit: [MINOR][DOCS] Remove remaining sqlContext in documentation at examples

Repository: spark Updated Branches: refs/heads/master 671b382a8 -> 2992a215c [MINOR][DOCS] Remove remaining sqlContext in documentation at examples This PR removes `sqlContext` in examples. Actual usage was all replaced in https://github.com/apache/spark/pull/12809 but there are some in

spark git commit: [SPARK-15199][SQL] Disallow Dropping Build-in Functions

Repository: spark Updated Branches: refs/heads/branch-2.0 5cdb7bea5 -> 29bc8d2ec [SPARK-15199][SQL] Disallow Dropping Build-in Functions What changes were proposed in this pull request? As Hive and the major RDBMS behave, the built-in functions are not allowed to drop. In the current

spark git commit: [SPARK-15199][SQL] Disallow Dropping Build-in Functions

Repository: spark Updated Branches: refs/heads/master beb16ec55 -> b1e01fd51 [SPARK-15199][SQL] Disallow Dropping Build-in Functions What changes were proposed in this pull request? As Hive and the major RDBMS behave, the built-in functions are not allowed to drop. In the current

spark git commit: [SPARK-15093][SQL] create/delete/rename directory for InMemoryCatalog operations if needed

Repository: spark Updated Branches: refs/heads/master ee3b17156 -> beb16ec55 [SPARK-15093][SQL] create/delete/rename directory for InMemoryCatalog operations if needed ## What changes were proposed in this pull request? following operations have file system operation now: 1. CREATE

spark git commit: [SPARK-15093][SQL] create/delete/rename directory for InMemoryCatalog operations if needed

Repository: spark Updated Branches: refs/heads/branch-2.0 fb73663db -> 5cdb7bea5 [SPARK-15093][SQL] create/delete/rename directory for InMemoryCatalog operations if needed ## What changes were proposed in this pull request? following operations have file system operation now: 1. CREATE

spark git commit: [SPARK-13566][CORE] Avoid deadlock between BlockManager and Executor Thread

Repository: spark Updated Branches: refs/heads/branch-1.6 a3aa22a59 -> ab006523b [SPARK-13566][CORE] Avoid deadlock between BlockManager and Executor Thread Temp patch for branch 1.6， avoid deadlock between BlockManager and Executor Thread. Author: cenyuhai

spark git commit: [HOTFIX] Fix MLUtils compile

Repository: spark Updated Branches: refs/heads/branch-2.0 a1887f213 -> 7dc3fb6ae [HOTFIX] Fix MLUtils compile Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7dc3fb6a Tree:

spark git commit: [HOTFIX] Fix MLUtils compile

Repository: spark Updated Branches: refs/heads/master bbb777343 -> 7f5922aa4 [HOTFIX] Fix MLUtils compile Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7f5922aa Tree:

spark git commit: [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements

Repository: spark Updated Branches: refs/heads/branch-2.0 1064a3303 -> a1887f213 [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements ## What changes were proposed in this pull request? Minor doc and code style fixes ## How was this patch tested? local build Author: Jacek

spark git commit: [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements

Repository: spark Updated Branches: refs/heads/master 02c07e899 -> bbb777343 [SPARK-15152][DOC][MINOR] Scaladoc and Code style Improvements ## What changes were proposed in this pull request? Minor doc and code style fixes ## How was this patch tested? local build Author: Jacek Laskowski

<    1   2   3   4   5   6   7   8   9   10   >