[46/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/attach.html -- diff --git a/site/docs/2.1.0/api/R/attach.html b/site/docs/2.1.0/api/R/attach.html new file mode 100644 index

[12/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/java/org/apache/spark/Accumulable.html -- diff --git a/site/docs/2.1.0/api/java/org/apache/spark/Accumulable.html

[03/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/java/org/apache/spark/SparkContext.html -- diff --git a/site/docs/2.1.0/api/java/org/apache/spark/SparkContext.html

[08/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/java/org/apache/spark/ExecutorRemoved.html -- diff --git a/site/docs/2.1.0/api/java/org/apache/spark/ExecutorRemoved.html

[36/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/join.html -- diff --git a/site/docs/2.1.0/api/R/join.html b/site/docs/2.1.0/api/R/join.html new file mode 100644 index 000..b01ef1c ---

[09/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/java/org/apache/spark/ComplexFutureAction.html -- diff --git a/site/docs/2.1.0/api/java/org/apache/spark/ComplexFutureAction.html

[26/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/spark.gaussianMixture.html -- diff --git a/site/docs/2.1.0/api/R/spark.gaussianMixture.html

[27/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/show.html -- diff --git a/site/docs/2.1.0/api/R/show.html b/site/docs/2.1.0/api/R/show.html new file mode 100644 index 000..5adbc9a ---

[29/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/rpad.html -- diff --git a/site/docs/2.1.0/api/R/rpad.html b/site/docs/2.1.0/api/R/rpad.html new file mode 100644 index 000..6f6f15f ---

[28/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/select.html -- diff --git a/site/docs/2.1.0/api/R/select.html b/site/docs/2.1.0/api/R/select.html new file mode 100644 index

[32/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/persist.html -- diff --git a/site/docs/2.1.0/api/R/persist.html b/site/docs/2.1.0/api/R/persist.html new file mode 100644 index

[33/51] [partial] spark-website git commit: Add docs for 2.1.0

http://git-wip-us.apache.org/repos/asf/spark-website/blob/066dfa27/site/docs/2.1.0/api/R/nanvl.html -- diff --git a/site/docs/2.1.0/api/R/nanvl.html b/site/docs/2.1.0/api/R/nanvl.html new file mode 100644 index 000..3bceb0c

svn commit: r17546 - /dev/spark/spark-2.1.0/ /release/spark/spark-2.1.0/

Author: rxin Date: Thu Dec 22 22:03:22 2016 New Revision: 17546 Log: Spark 2.1.0 Added: release/spark/spark-2.1.0/ - copied from r17545, dev/spark/spark-2.1.0/ Removed: dev/spark/spark-2.1.0/ - To unsubscribe, e

svn commit: r17545 - /dev/spark/spark-2.1.0/

Author: rxin Date: Thu Dec 22 21:58:11 2016 New Revision: 17545 Log: add spark 2.1.0 Added: dev/spark/spark-2.1.0/ dev/spark/spark-2.1.0/spark-2.1.0-bin-hadoop2.3.tgz (with props) dev/spark/spark-2.1.0/spark-2.1.0-bin-hadoop2.3.tgz.asc dev/spark/spark-2.1.0/spark-2.1.0-bin

spark git commit: [SPARK-18975][CORE] Add an API to remove SparkListener

Repository: spark Updated Branches: refs/heads/master 261510005 -> 31da755c8 [SPARK-18975][CORE] Add an API to remove SparkListener ## What changes were proposed in this pull request? In current Spark we could add customized SparkListener through `SparkContext#addListener` API, but there's

spark git commit: [SPARK-16975][SQL][FOLLOWUP] Do not duplicately check file paths in data sources implementing FileFormat

Repository: spark Updated Branches: refs/heads/master 4186aba63 -> 76622c661 [SPARK-16975][SQL][FOLLOWUP] Do not duplicately check file paths in data sources implementing FileFormat ## What changes were proposed in this pull request? This PR cleans up duplicated checking for file paths in

spark git commit: [DOC] bucketing is applicable to all file-based data sources

xes some function docs that haven't yet been updated to reflect that. ## How was this patch tested? N/A Author: Reynold Xin <r...@databricks.com> Closes #16349 from rxin/ds-doc. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spa

spark git commit: [DOC] bucketing is applicable to all file-based data sources

tch fixes some function docs that haven't yet been updated to reflect that. ## How was this patch tested? N/A Author: Reynold Xin <r...@databricks.com> Closes #16349 from rxin/ds-doc. (cherry picked from commit 2e861df96eacd821edbbd9883121bff67611074f) Signed-off-by: Reynold Xin <r...@da

spark git commit: [SQL] Minor readability improvement for partition handling code

und some naming / implicit type inference not as intuitive. This patch should have no semantic change and the changes should be covered by existing test cases. Author: Reynold Xin <r...@databricks.com> Closes #16378 from rxin/minor-fix. (cherry picked fr

spark git commit: [SPARK-18949][SQL][BACKPORT-2.1] Add recoverPartitions API to Catalog

Repository: spark Updated Branches: refs/heads/branch-2.1 318483421 -> 0e51bb085 [SPARK-18949][SQL][BACKPORT-2.1] Add recoverPartitions API to Catalog ### What changes were proposed in this pull request? This PR is to backport https://github.com/apache/spark/pull/16356 to Spark 2.1.1

spark git commit: [SPARK-18949][SQL] Add recoverPartitions API to Catalog

Repository: spark Updated Branches: refs/heads/master b2dd8ec6b -> 24c0c9412 [SPARK-18949][SQL] Add recoverPartitions API to Catalog ### What changes were proposed in this pull request? Currently, we only have a SQL interface for recovering all the partitions in the directory of a table and

spark git commit: [SPARK-18576][PYTHON] Add basic TaskContext information to PySpark

Repository: spark Updated Branches: refs/heads/master caed89321 -> 047a9d92c [SPARK-18576][PYTHON] Add basic TaskContext information to PySpark ## What changes were proposed in this pull request? Adds basic TaskContext information to PySpark. ## How was this patch tested? New unit tests to

spark git commit: Tiny style improvement.

Repository: spark Updated Branches: refs/heads/master f923c849e -> 150d26cad Tiny style improvement. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/150d26ca Tree:

spark git commit: Fix test case for SubquerySuite.

Repository: spark Updated Branches: refs/heads/branch-2.0 b41668349 -> 2a5ab1490 Fix test case for SubquerySuite. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2a5ab149 Tree:

spark git commit: [SPARK-18892][SQL] Alias percentile_approx approx_percentile

nge, but it seemed too trivial to me. Author: Reynold Xin <r...@databricks.com> Closes #16300 from rxin/SPARK-18892. (cherry picked from commit 172a52f5d31337d90155feb7072381e8d5712288) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/sp

spark git commit: [SPARK-18892][SQL] Alias percentile_approx approx_percentile

nge, but it seemed too trivial to me. Author: Reynold Xin <r...@databricks.com> Closes #16300 from rxin/SPARK-18892. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/172a52f5 Tree: http://git-wip-us.apache.org/repos/asf/s

spark git commit: [MINOR] Only rename SparkR tar.gz if names mismatch

Repository: spark Updated Branches: refs/heads/branch-2.1 08e427287 -> ae853e8f3 [MINOR] Only rename SparkR tar.gz if names mismatch ## What changes were proposed in this pull request? For release builds the R_PACKAGE_VERSION and VERSION are the same (e.g., 2.1.0). Thus `cp` throws an error

spark git commit: [MINOR] Only rename SparkR tar.gz if names mismatch

Repository: spark Updated Branches: refs/heads/master 9c7f83b02 -> 9634018c4 [MINOR] Only rename SparkR tar.gz if names mismatch ## What changes were proposed in this pull request? For release builds the R_PACKAGE_VERSION and VERSION are the same (e.g., 2.1.0). Thus `cp` throws an error

spark git commit: [SPARK-8425][SCHEDULER][HOTFIX] fix scala 2.10 compile error

Repository: spark Updated Branches: refs/heads/master 0917c8ee0 -> 32ff96452 [SPARK-8425][SCHEDULER][HOTFIX] fix scala 2.10 compile error ## What changes were proposed in this pull request? https://github.com/apache/spark/commit/93cdb8a7d0f124b4db069fd8242207c82e263c52 Introduced a compile

spark git commit: [SPARK-17910][SQL] Allow users to update the comment of a column

Repository: spark Updated Branches: refs/heads/master 93cdb8a7d -> 01e14bf30 [SPARK-17910][SQL] Allow users to update the comment of a column ## What changes were proposed in this pull request? Right now, once a user set the comment of a column with create table command, he/she cannot

spark git commit: [SPARK-18869][SQL] Add TreeNode.p that returns BaseType

old Xin <r...@databricks.com> Closes #16288 from rxin/SPARK-18869. (cherry picked from commit 5d510c693aca8c3fd3364b4453160bc8585ffc8e) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/

spark git commit: [SPARK-18869][SQL] Add TreeNode.p that returns BaseType

old Xin <r...@databricks.com> Closes #16288 from rxin/SPARK-18869. (cherry picked from commit 5d510c693aca8c3fd3364b4453160bc8585ffc8e) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/

spark git commit: [SPARK-18869][SQL] Add TreeNode.p that returns BaseType

Xin <r...@databricks.com> Closes #16288 from rxin/SPARK-18869. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5d510c69 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5d510c69 Diff: http://git-wip-us.apache.org/

spark git commit: [SPARK-18856][SQL] non-empty partitioned table should not report zero size

Repository: spark Updated Branches: refs/heads/branch-2.1 0d94201e0 -> cb2c8428d [SPARK-18856][SQL] non-empty partitioned table should not report zero size ## What changes were proposed in this pull request? In `DataSource`, if the table is not analyzed, we will use 0 as the default value

spark git commit: [SPARK-18856][SQL] non-empty partitioned table should not report zero size

Repository: spark Updated Branches: refs/heads/master 8db4d95c0 -> d6f11a12a [SPARK-18856][SQL] non-empty partitioned table should not report zero size ## What changes were proposed in this pull request? In `DataSource`, if the table is not analyzed, we will use 0 as the default value for

[1/2] spark git commit: Revert "Revert "[SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries""

Repository: spark Updated Branches: refs/heads/branch-2.0 a5c178bc0 -> a32317845 Revert "Revert "[SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries"" This reverts commit a5c178bc07092b698ee17894a439deb47699db0f. Project:

[2/2] spark git commit: Fix compilation error

Fix compilation error Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a3231784 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a3231784 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a3231784 Branch:

spark git commit: Revert "[SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries"

Repository: spark Updated Branches: refs/heads/branch-2.0 1ff738afc -> a5c178bc0 Revert "[SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries" This reverts commit 1ff738afc1b11eacb11ac4f37324334a6b6fe41b. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries

sal of numbering as well as making sure the two methods are consistent. Author: Reynold Xin <r...@databricks.com> Closes #16277 from rxin/SPARK-18854. (cherry picked from commit ffdd1fcd1e8f4f6453d5b0517c0ce82766b8e75f) Signed-off-by: Reynold Xin <r...@databricks.com> Project:

spark git commit: [SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries

ted? Added a test case in SubquerySuite.scala to test both the depth-first traversal of numbering as well as making sure the two methods are consistent. Author: Reynold Xin <r...@databricks.com> Closes #16277 from rxin/SPARK-18854. (cherry picked from commit ffdd1fcd1e8f4f6453d5b0517c0ce82

spark git commit: [SPARK-18854][SQL] numberedTreeString and apply(i) inconsistent for subqueries

ted? Added a test case in SubquerySuite.scala to test both the depth-first traversal of numbering as well as making sure the two methods are consistent. Author: Reynold Xin <r...@databricks.com> Closes #16277 from rxin/SPARK-18854. Project: http://git-wip-us.apache.org/repos/asf/spark/re

spark git commit: [SPARK-18730] Post Jenkins test report page instead of the full console output page to GitHub

Repository: spark Updated Branches: refs/heads/master ac013ea58 -> ba4aab9b8 [SPARK-18730] Post Jenkins test report page instead of the full console output page to GitHub ## What changes were proposed in this pull request? Currently, the full console output page of a Spark Jenkins PR build

spark git commit: [SPARK-18730] Post Jenkins test report page instead of the full console output page to GitHub

Repository: spark Updated Branches: refs/heads/branch-2.1 f999312e7 -> 16d4bd4a2 [SPARK-18730] Post Jenkins test report page instead of the full console output page to GitHub ## What changes were proposed in this pull request? Currently, the full console output page of a Spark Jenkins PR

spark git commit: [SPARK-18588][TESTS] Ignore KafkaSourceStressForDontFailOnDataLossSuite

Repository: spark Updated Branches: refs/heads/branch-2.1 5693ac8e5 -> 019d1fa3d [SPARK-18588][TESTS] Ignore KafkaSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? Disable KafkaSourceStressForDontFailOnDataLossSuite for now. ## How was this patch

spark git commit: [SPARK-18588][TESTS] Ignore KafkaSourceStressForDontFailOnDataLossSuite

Repository: spark Updated Branches: refs/heads/master 3ae63b808 -> e104e55c1 [SPARK-18588][TESTS] Ignore KafkaSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? Disable KafkaSourceStressForDontFailOnDataLossSuite for now. ## How was this patch tested?

spark git commit: [SPARK-18752][SQL] Follow-up: add scaladoc explaining isSrcLocal arg.

Repository: spark Updated Branches: refs/heads/master ae5b2d3e4 -> 3ae63b808 [SPARK-18752][SQL] Follow-up: add scaladoc explaining isSrcLocal arg. Author: Marcelo Vanzin Closes #16257 from vanzin/SPARK-18752.2. Project:

spark git commit: [SPARK-18815][SQL] Fix NPE when collecting column stats for string/binary column having only null values

Repository: spark Updated Branches: refs/heads/master e094d0115 -> a29ee55aa [SPARK-18815][SQL] Fix NPE when collecting column stats for string/binary column having only null values ## What changes were proposed in this pull request? During column stats collection, average and max length

spark git commit: [SPARK-18815][SQL] Fix NPE when collecting column stats for string/binary column having only null values

Repository: spark Updated Branches: refs/heads/branch-2.1 5151dafaa -> de21ca46e [SPARK-18815][SQL] Fix NPE when collecting column stats for string/binary column having only null values ## What changes were proposed in this pull request? During column stats collection, average and max

spark git commit: [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled (branch 2.1)

Repository: spark Updated Branches: refs/heads/branch-2.1 fcd22e538 -> 1cafc76ea [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled (branch 2.1) ## What changes were proposed in this pull request? Backport #16203 to branch 2.1. ## How was this patch

spark git commit: [SPARK-18760][SQL] Consistent format specification for FileFormats

ext formats are the only two that are different from the rest): Before: ``` scala> spark.read.text("test.text").explain() == Physical Plan == *FileScan text [value#15] Batched: false, Format: org.apache.spark.sql.execution.datasources.text.TextFileFormatxyz, Location: InMemoryFileIndex[fi

spark git commit: [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled

Repository: spark Updated Branches: refs/heads/master 330fda8aa -> b47b892e4 [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled ## What changes were proposed in this pull request? When `ignoreCorruptFiles` is enabled, it's better to also ignore

spark git commit: Close stale pull requests.

Repository: spark Updated Branches: refs/heads/master 97255497d -> 330fda8aa Close stale pull requests. Closes #15689 Closes #14640 Closes #15917 Closes #16188 Closes #16206 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-18654][SQL] Remove unreachable patterns in makeRootConverter

Repository: spark Updated Branches: refs/heads/master 70b2bf717 -> bec0a9217 [SPARK-18654][SQL] Remove unreachable patterns in makeRootConverter ## What changes were proposed in this pull request? `makeRootConverter` is only called with a `StructType` value. By making this method less

spark git commit: Closes stale & invalid pull requests.

Repository: spark Updated Branches: refs/heads/master 01c7c6b88 -> 08d644127 Closes stale & invalid pull requests. Closes #14537. Closes #16181. Closes #8318. Closes #6848. Closes #7265. Closes #9543. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: Update Spark documentation to provide information on how to create External Table

Repository: spark Updated Branches: refs/heads/master 539bb3cf9 -> 01c7c6b88 Update Spark documentation to provide information on how to create External Table ## What changes were proposed in this pull request? Although, currently, the saveAsTable does not provide an API to save the table

spark git commit: [SPARK-18555][SQL] DataFrameNaFunctions.fill miss up original values in long integers

Repository: spark Updated Branches: refs/heads/master 2398fde45 -> 508de38c9 [SPARK-18555][SQL] DataFrameNaFunctions.fill miss up original values in long integers ## What changes were proposed in this pull request? DataSet.na.fill(0) used on a DataSet which has a long value column, it

spark git commit: [SPARK-18702][SQL] input_file_block_start and input_file_block_length

ted? Updated existing test cases in ColumnExpressionSuite that covered input_file_name to also cover the two new functions. Author: Reynold Xin <r...@databricks.com> Closes #16133 from rxin/SPARK-18702. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.

spark git commit: [MINOR][README] Correct Markdown link inside readme

Repository: spark Updated Branches: refs/heads/master 34777184c -> edb0ad9d4 [MINOR][README] Correct Markdown link inside readme ## What changes were proposed in this pull request? "Useful Developer Tools" link inside

spark git commit: [SPARK-18362][SQL] Use TextFileFormat in implementation of CSVFileFormat

Repository: spark Updated Branches: refs/heads/master c7c726595 -> 7c33b0fd0 [SPARK-18362][SQL] Use TextFileFormat in implementation of CSVFileFormat ## What changes were proposed in this pull request? This patch significantly improves the IO / file listing performance of schema inference

spark git commit: [SPARK-18695] Bump master branch version to 2.2.0-SNAPSHOT

hor: Reynold Xin <r...@databricks.com> Closes #16126 from rxin/SPARK-18695. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c7c72659 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c7c72659 Diff: http://git-wip-us.a

spark git commit: [SPARK-18690][PYTHON][SQL] Backward compatibility of unbounded frames

Repository: spark Updated Branches: refs/heads/branch-2.1 839d4e9ca -> cf3dbec68 [SPARK-18690][PYTHON][SQL] Backward compatibility of unbounded frames ## What changes were proposed in this pull request? Makes `Window.unboundedPreceding` and `Window.unboundedFollowing` backward compatible.

spark git commit: [SPARK-18690][PYTHON][SQL] Backward compatibility of unbounded frames

Repository: spark Updated Branches: refs/heads/master 2dc0d7efe -> a9cbfc4f6 [SPARK-18690][PYTHON][SQL] Backward compatibility of unbounded frames ## What changes were proposed in this pull request? Makes `Window.unboundedPreceding` and `Window.unboundedFollowing` backward compatible. ##

spark git commit: [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686

Repository: spark Updated Branches: refs/heads/branch-2.1 0f0903d17 -> a7f8ebb86 [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686 This PR targets to both master and branch-2.1. ## What changes were proposed in this pull request? Due to

spark git commit: [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686

Repository: spark Updated Branches: refs/heads/master c82f16c15 -> ca6391637 [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686 This PR targets to both master and branch-2.1. ## What changes were proposed in this pull request? Due to

spark git commit: [SPARK-18639] Build only a single pip package

we prepare releases. Author: Reynold Xin <r...@databricks.com> Closes #16072 from rxin/SPARK-18639. (cherry picked from commit 37e52f8793bff306a7ae5a9aecc16f28333b70e3) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-18639] Build only a single pip package

we prepare releases. Author: Reynold Xin <r...@databricks.com> Closes #16072 from rxin/SPARK-18639. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/37e52f87 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/37e52f87 D

spark git commit: [SPARK-18617][BACKPORT] Follow up PR to Close "kryo auto pick" feature for Spark Streaming

to branch-2.0 with incremental update in #16091 ## How was this patch tested? new unit test cc zsxwing rxin Author: uncleGen <husty...@gmail.com> Closes #16096 from uncleGen/branch-2.0-SPARK-18617. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/rep

spark git commit: [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled

Repository: spark Updated Branches: refs/heads/master 88f559f20 -> dbf842b7a [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled ## What changes were proposed in this pull request? `spark.sql.unsafe.enabled` is deprecated since 1.6. There still are

spark git commit: [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled

Repository: spark Updated Branches: refs/heads/branch-2.1 9dc3ef6e1 -> 8579ab5d7 [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled ## What changes were proposed in this pull request? `spark.sql.unsafe.enabled` is deprecated since 1.6. There still are

spark git commit: [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly

Repository: spark Updated Branches: refs/heads/branch-2.1 6e2e987bd -> 7d4596734 [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly ## What changes were proposed in this pull request? Fixed the potential SparkContext leak in

spark git commit: [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly

Repository: spark Updated Branches: refs/heads/master c4979f6ea -> 0a811210f [SPARK-18617][SPARK-18560][TEST] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly ## What changes were proposed in this pull request? Fixed the potential SparkContext leak in

spark git commit: [SPARK-18220][SQL] read Hive orc table with varchar column should not fail

Repository: spark Updated Branches: refs/heads/branch-2.1 6e044ab9a -> 3de93fb48 [SPARK-18220][SQL] read Hive orc table with varchar column should not fail ## What changes were proposed in this pull request? Spark SQL only has `StringType`, when reading hive table with varchar column, we

spark git commit: [SPARK-18220][SQL] read Hive orc table with varchar column should not fail

Repository: spark Updated Branches: refs/heads/master c24076dcf -> 3f03c90a8 [SPARK-18220][SQL] read Hive orc table with varchar column should not fail ## What changes were proposed in this pull request? Spark SQL only has `StringType`, when reading hive table with varchar column, we will

spark git commit: [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" feature for Spark Streaming

Repository: spark Updated Branches: refs/heads/master 879ba7111 -> 56c82edab [SPARK-18617][CORE][STREAMING] Close "kryo auto pick" feature for Spark Streaming ## What changes were proposed in this pull request? #15992 provided a solution to fix the bug, i.e. **receiver data can not be

spark git commit: [SPARK-18145] Update documentation for hive partition management in 2.1

ing tables. ## How was this patch tested? Built docs locally. rxin Author: Eric Liang <e...@databricks.com> Closes #16074 from ericl/spark-18145. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/489845f3 Tree: http:

spark git commit: [SPARK-18632][SQL] AggregateFunction should not implement ImplicitCastInputTypes

Repository: spark Updated Branches: refs/heads/master 9b670bcae -> af9789a4f [SPARK-18632][SQL] AggregateFunction should not implement ImplicitCastInputTypes ## What changes were proposed in this pull request? `AggregateFunction` currently implements `ImplicitCastInputTypes` (which enables

spark git commit: [SPARK-18429][SQL] implement a new Aggregate for CountMinSketch

Repository: spark Updated Branches: refs/heads/master f643fe47f -> d57a594b8 [SPARK-18429][SQL] implement a new Aggregate for CountMinSketch ## What changes were proposed in this pull request? This PR implements a new Aggregate to generate count min sketch, which is a wrapper of

spark git commit: [SPARK-18544][SQL] Append with df.saveAsTable writes data to wrong location

Repository: spark Updated Branches: refs/heads/branch-2.1 1759cf69a -> 27a1a5c99 [SPARK-18544][SQL] Append with df.saveAsTable writes data to wrong location ## What changes were proposed in this pull request? We failed to properly propagate table metadata for existing tables for the

spark git commit: [SPARK-18544][SQL] Append with df.saveAsTable writes data to wrong location

Repository: spark Updated Branches: refs/heads/master d449988b8 -> e2318ede0 [SPARK-18544][SQL] Append with df.saveAsTable writes data to wrong location ## What changes were proposed in this pull request? We failed to properly propagate table metadata for existing tables for the saveAsTable

spark git commit: [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...) instead equality on asNullable datatypes

Repository: spark Updated Branches: refs/heads/branch-2.1 c4cbdc864 -> 1759cf69a [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...) instead equality on asNullable datatypes ## What changes were proposed in this pull request? This is absolutely minor. PR

spark git commit: [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...) instead equality on asNullable datatypes

Repository: spark Updated Branches: refs/heads/master 8b325b17e -> d449988b8 [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...) instead equality on asNullable datatypes ## What changes were proposed in this pull request? This is absolutely minor. PR

spark git commit: [SPARK-18523][PYSPARK] Make SparkContext.stop more reliable

Repository: spark Updated Branches: refs/heads/branch-2.1 cdf315ba1 -> c46928ff9 [SPARK-18523][PYSPARK] Make SparkContext.stop more reliable ## What changes were proposed in this pull request? This PR fixes SparkContext broken state in which it may fall if spark driver get crashed or killed

spark git commit: [SQL][MINOR] DESC should use 'Catalog' as partition provider

Repository: spark Updated Branches: refs/heads/master eba727757 -> 185642846 [SQL][MINOR] DESC should use 'Catalog' as partition provider ## What changes were proposed in this pull request? `CatalogTable` has a parameter named `tracksPartitionsInCatalog`, and in `CatalogTable.toString` we

spark git commit: [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize.

Repository: spark Updated Branches: refs/heads/master fc2c13bdf -> 87141622e [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize. ## What changes were proposed in this pull request? Janino can optimize `true ? a : b` into `a` or `false ? a : b`

spark git commit: [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize.

Repository: spark Updated Branches: refs/heads/branch-2.1 6b77889e8 -> 886f880df [SPARK-18585][SQL] Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize. ## What changes were proposed in this pull request? Janino can optimize `true ? a : b` into `a` or `false ? a :

spark git commit: [SPARK-18482][SQL] make sure Spark can access the table metadata created by older version of spark

Repository: spark Updated Branches: refs/heads/master 07f32c228 -> fc2c13bdf [SPARK-18482][SQL] make sure Spark can access the table metadata created by older version of spark ## What changes were proposed in this pull request? In Spark 2.1, we did a lot of refactor for

spark git commit: [SPARK-18482][SQL] make sure Spark can access the table metadata created by older version of spark

Repository: spark Updated Branches: refs/heads/branch-2.1 1e8fbefa3 -> 6b77889e8 [SPARK-18482][SQL] make sure Spark can access the table metadata created by older version of spark ## What changes were proposed in this pull request? In Spark 2.1, we did a lot of refactor for

spark git commit: [SPARK-18583][SQL] Fix nullability of InputFileName.

Repository: spark Updated Branches: refs/heads/master e2fb9fd36 -> a88329d45 [SPARK-18583][SQL] Fix nullability of InputFileName. ## What changes were proposed in this pull request? The nullability of `InputFileName` should be `false`. ## How was this patch tested? Existing tests. Author:

spark git commit: [SPARK-18583][SQL] Fix nullability of InputFileName.

Repository: spark Updated Branches: refs/heads/branch-2.1 906d82c4c -> da66b9742 [SPARK-18583][SQL] Fix nullability of InputFileName. ## What changes were proposed in this pull request? The nullability of `InputFileName` should be `false`. ## How was this patch tested? Existing tests.

spark git commit: [SPARK-18413][SQL][FOLLOW-UP] Use `numPartitions` instead of `maxConnections`

Repository: spark Updated Branches: refs/heads/master 445d4d9e1 -> fb07bbe57 [SPARK-18413][SQL][FOLLOW-UP] Use `numPartitions` instead of `maxConnections` ## What changes were proposed in this pull request? This is a follow-up PR of #15868 to merge `maxConnections` option into

spark git commit: [SPARK-18179][SQL] Throws analysis exception with a proper message for unsupported argument types in reflect/java_method function

Repository: spark Updated Branches: refs/heads/branch-2.1 fc5fee83e -> fabb5aeaf [SPARK-18179][SQL] Throws analysis exception with a proper message for unsupported argument types in reflect/java_method function ## What changes were proposed in this pull request? This PR proposes throwing an

spark git commit: [SPARK-18179][SQL] Throws analysis exception with a proper message for unsupported argument types in reflect/java_method function

Repository: spark Updated Branches: refs/heads/master 982b82e32 -> 2559fb4b4 [SPARK-18179][SQL] Throws analysis exception with a proper message for unsupported argument types in reflect/java_method function ## What changes were proposed in this pull request? This PR proposes throwing an

spark git commit: [SPARK-17765][SQL] Support for writing out user-defined type in ORC datasource

Repository: spark Updated Branches: refs/heads/master ddd02f50b -> a2d464770 [SPARK-17765][SQL] Support for writing out user-defined type in ORC datasource ## What changes were proposed in this pull request? This PR adds the support for `UserDefinedType` when writing out instead of throwing

spark git commit: [SPARK-17765][SQL] Support for writing out user-defined type in ORC datasource

Repository: spark Updated Branches: refs/heads/branch-2.1 406f33987 -> 2afc18be2 [SPARK-17765][SQL] Support for writing out user-defined type in ORC datasource ## What changes were proposed in this pull request? This PR adds the support for `UserDefinedType` when writing out instead of

spark git commit: [HOTFIX][SQL] Fix DDLSuite failure.

Repository: spark Updated Branches: refs/heads/branch-2.1 cffaf5035 -> f8662db72 [HOTFIX][SQL] Fix DDLSuite failure. (cherry picked from commit b625a36ebc59cbacc223fc03005bc0f6d296b6e7) Signed-off-by: Reynold Xin Project:

spark git commit: [HOTFIX][SQL] Fix DDLSuite failure.

Repository: spark Updated Branches: refs/heads/master 6659ae555 -> b625a36eb [HOTFIX][SQL] Fix DDLSuite failure. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b625a36e Tree:

spark git commit: Fix Mesos build break for Scala 2.10.

Repository: spark Updated Branches: refs/heads/master c528812ce -> 6659ae555 Fix Mesos build break for Scala 2.10. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6659ae55 Tree:

spark git commit: [SPARK-18508][SQL] Fix documentation error for DateDiff

nly change. Author: Reynold Xin <r...@databricks.com> Closes #15937 from rxin/datediff-doc. (cherry picked from commit bce9a03677f931d52491e7768aba9e4a19a7e696) Signed-off-by: Reynold Xin <r...@databricks.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-18508][SQL] Fix documentation error for DateDiff

nly change. Author: Reynold Xin <r...@databricks.com> Closes #15937 from rxin/datediff-doc. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bce9a036 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bce9a036 Diff: http:

spark git commit: [SPARK-18458][CORE] Fix signed integer overflow problem at an expression in RadixSort.java

Repository: spark Updated Branches: refs/heads/branch-2.1 15eb86c29 -> b0b2f1081 [SPARK-18458][CORE] Fix signed integer overflow problem at an expression in RadixSort.java ## What changes were proposed in this pull request? This PR avoids that a result of an expression is negative due to

<    1   2   3   4   5   6   7   8   9   10   >