spark git commit: [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled

Repository: spark Updated Branches: refs/heads/master 330fda8aa -> b47b892e4 [SPARK-18774][CORE][SQL] Ignore non-existing files when ignoreCorruptFiles is enabled ## What changes were proposed in this pull request? When `ignoreCorruptFiles` is enabled, it's better to also ignore

spark git commit: Close stale pull requests.

Repository: spark Updated Branches: refs/heads/master 97255497d -> 330fda8aa Close stale pull requests. Closes #15689 Closes #14640 Closes #15917 Closes #16188 Closes #16206 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

[1/2] spark git commit: Preparing Spark release v2.1.0-rc2

Repository: spark Updated Branches: refs/heads/branch-2.1 1c3f1da82 -> 48aa6775d Preparing Spark release v2.1.0-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/08071749 Tree:

[2/2] spark git commit: Preparing development version 2.1.1-SNAPSHOT

Preparing development version 2.1.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/48aa6775 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/48aa6775 Diff:

[spark] Git Push Summary

Repository: spark Updated Tags: refs/tags/v2.1.0-rc2 [created] 080717497 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API for 2.1

Repository: spark Updated Branches: refs/heads/master 82253617f -> 97255497d [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API for 2.1 ## What changes were proposed in this pull request? Reviewing SparkR ML wrappers API for 2.1 release, mainly two issues: * Remove ```probabilityCol```

spark git commit: [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API for 2.1

Repository: spark Updated Branches: refs/heads/branch-2.1 ab865cfd9 -> 1c3f1da82 [SPARK-18326][SPARKR][ML] Review SparkR ML wrappers API for 2.1 ## What changes were proposed in this pull request? Reviewing SparkR ML wrappers API for 2.1 release, mainly two issues: * Remove

spark git commit: [SPARK-18705][ML][DOC] Update user guide to reflect one pass solver for L1 and elastic-net

Repository: spark Updated Branches: refs/heads/branch-2.1 617ce3ba7 -> ab865cfd9 [SPARK-18705][ML][DOC] Update user guide to reflect one pass solver for L1 and elastic-net ## What changes were proposed in this pull request? WeightedLeastSquares now supports L1 and elastic net penalties and

spark git commit: [SPARK-18705][ML][DOC] Update user guide to reflect one pass solver for L1 and elastic-net

Repository: spark Updated Branches: refs/heads/master 9ab725eab -> 82253617f [SPARK-18705][ML][DOC] Update user guide to reflect one pass solver for L1 and elastic-net ## What changes were proposed in this pull request? WeightedLeastSquares now supports L1 and elastic net penalties and has

spark git commit: [SPARK-18758][SS] StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

Repository: spark Updated Branches: refs/heads/branch-2.1 839c2eb97 -> 617ce3ba7 [SPARK-18758][SS] StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query ## What changes were proposed in this pull request? Listeners added

spark git commit: [SPARK-18758][SS] StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query

Repository: spark Updated Branches: refs/heads/master aad11209e -> 9ab725eab [SPARK-18758][SS] StreamingQueryListener events from a StreamingQuery should be sent only to the listeners in the same session as the query ## What changes were proposed in this pull request? Listeners added with

spark git commit: [SPARK-18633][ML][EXAMPLE] Add multiclass logistic regression summary python example and document

Repository: spark Updated Branches: refs/heads/branch-2.1 1c6419718 -> 839c2eb97 [SPARK-18633][ML][EXAMPLE] Add multiclass logistic regression summary python example and document ## What changes were proposed in this pull request? Logistic Regression summary is added in Python API. We need

spark git commit: [SPARK-18633][ML][EXAMPLE] Add multiclass logistic regression summary python example and document

Repository: spark Updated Branches: refs/heads/master bec0a9217 -> aad11209e [SPARK-18633][ML][EXAMPLE] Add multiclass logistic regression summary python example and document ## What changes were proposed in this pull request? Logistic Regression summary is added in Python API. We need to

spark git commit: [SPARK-18654][SQL] Remove unreachable patterns in makeRootConverter

Repository: spark Updated Branches: refs/heads/master 70b2bf717 -> bec0a9217 [SPARK-18654][SQL] Remove unreachable patterns in makeRootConverter ## What changes were proposed in this pull request? `makeRootConverter` is only called with a `StructType` value. By making this method less

spark git commit: [SPARK-18754][SS] Rename recentProgresses to recentProgress

Repository: spark Updated Branches: refs/heads/branch-2.1 e9b3afac9 -> 1c6419718 [SPARK-18754][SS] Rename recentProgresses to recentProgress Based on an informal survey, users find this option easier to understand / remember. Author: Michael Armbrust Closes #16182

spark git commit: [SPARK-18754][SS] Rename recentProgresses to recentProgress

Repository: spark Updated Branches: refs/heads/master edc87e189 -> 70b2bf717 [SPARK-18754][SS] Rename recentProgresses to recentProgress Based on an informal survey, users find this option easier to understand / remember. Author: Michael Armbrust Closes #16182 from

spark git commit: [SPARK-18588][TESTS] Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite

Repository: spark Updated Branches: refs/heads/master bb94f61a7 -> edc87e189 [SPARK-18588][TESTS] Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? Fixed the following failures: ```

spark git commit: [SPARK-18588][TESTS] Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite

Repository: spark Updated Branches: refs/heads/branch-2.1 76e1f1651 -> e9b3afac9 [SPARK-18588][TESTS] Fix flaky test: KafkaSourceStressForDontFailOnDataLossSuite ## What changes were proposed in this pull request? Fixed the following failures: ```

spark git commit: [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040

Repository: spark Updated Branches: refs/heads/branch-2.0 7fbb07372 -> 44df6d2ce [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040 ## What changes were proposed in this pull request? When SSL is enabled, the Spark shell shows: ``` Spark context Web UI available at

spark git commit: [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040

Repository: spark Updated Branches: refs/heads/branch-2.1 acb6ac5da -> 76e1f1651 [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040 ## What changes were proposed in this pull request? When SSL is enabled, the Spark shell shows: ``` Spark context Web UI available at

spark git commit: [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040

Repository: spark Updated Branches: refs/heads/master dbf3e298a -> bb94f61a7 [SPARK-18762][WEBUI] Web UI should be http:4040 instead of https:4040 ## What changes were proposed in this pull request? When SSL is enabled, the Spark shell shows: ``` Spark context Web UI available at

spark git commit: [SPARK-18764][CORE] Add a warning log when skipping a corrupted file

Repository: spark Updated Branches: refs/heads/branch-2.1 5dbcd4fcf -> acb6ac5da [SPARK-18764][CORE] Add a warning log when skipping a corrupted file ## What changes were proposed in this pull request? It's better to add a warning log when skipping a corrupted file. It will be helpful when

spark git commit: [SPARK-18764][CORE] Add a warning log when skipping a corrupted file

Repository: spark Updated Branches: refs/heads/master f1fca81b1 -> dbf3e298a [SPARK-18764][CORE] Add a warning log when skipping a corrupted file ## What changes were proposed in this pull request? It's better to add a warning log when skipping a corrupted file. It will be helpful when we

spark git commit: [SPARK-17760][SQL][BACKPORT] AnalysisException with dataframe pivot when groupBy column is not attribute

Repository: spark Updated Branches: refs/heads/branch-2.0 e05ad8830 -> 7fbb07372 [SPARK-17760][SQL][BACKPORT] AnalysisException with dataframe pivot when groupBy column is not attribute ## What changes were proposed in this pull request? Backport of #16177 to branch-2.0 ## How was this

spark git commit: [SPARK-17760][SQL] AnalysisException with dataframe pivot when groupBy column is not attribute

Repository: spark Updated Branches: refs/heads/branch-2.1 4432a2a83 -> 5dbcd4fcf [SPARK-17760][SQL] AnalysisException with dataframe pivot when groupBy column is not attribute ## What changes were proposed in this pull request? Fixes AnalysisException for pivot queries that have group by

spark git commit: [SPARK-17760][SQL] AnalysisException with dataframe pivot when groupBy column is not attribute

Repository: spark Updated Branches: refs/heads/master c496d03b5 -> f1fca81b1 [SPARK-17760][SQL] AnalysisException with dataframe pivot when groupBy column is not attribute ## What changes were proposed in this pull request? Fixes AnalysisException for pivot queries that have group by

spark git commit: [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap

Repository: spark Updated Branches: refs/heads/branch-2.0 f5c5a07bd -> e05ad8830 [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap ## What changes were proposed in this pull request? BytesToBytesMap currently does not release the in-memory storage (the

spark git commit: [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap

Repository: spark Updated Branches: refs/heads/branch-2.1 51754d6df -> 4432a2a83 [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap ## What changes were proposed in this pull request? BytesToBytesMap currently does not release the in-memory storage (the

spark git commit: [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap

Repository: spark Updated Branches: refs/heads/master 79f5f281b -> c496d03b5 [SPARK-18208][SHUFFLE] Executor OOM due to a growing LongArray in BytesToBytesMap ## What changes were proposed in this pull request? BytesToBytesMap currently does not release the in-memory storage (the longArray

spark git commit: [SPARK-18678][ML] Skewed reservoir sampling in SamplingUtils

Repository: spark Updated Branches: refs/heads/branch-2.1 99c293eea -> 51754d6df [SPARK-18678][ML] Skewed reservoir sampling in SamplingUtils ## What changes were proposed in this pull request? Fix reservoir sampling bias for small k. An off-by-one error meant that the probability of

spark git commit: [SPARK-18678][ML] Skewed reservoir sampling in SamplingUtils

Repository: spark Updated Branches: refs/heads/master b82802713 -> 79f5f281b [SPARK-18678][ML] Skewed reservoir sampling in SamplingUtils ## What changes were proposed in this pull request? Fix reservoir sampling bias for small k. An off-by-one error meant that the probability of

spark git commit: [SPARK-18701][ML] Fix Poisson GLM failure due to wrong initialization

Repository: spark Updated Branches: refs/heads/master 90b59d1bf -> b82802713 [SPARK-18701][ML] Fix Poisson GLM failure due to wrong initialization Poisson GLM fails for many standard data sets (see example in test or JIRA). The issue is incorrect initialization leading to almost zero

spark git commit: [SPARK-18701][ML] Fix Poisson GLM failure due to wrong initialization

Repository: spark Updated Branches: refs/heads/branch-2.1 340e9aea4 -> 99c293eea [SPARK-18701][ML] Fix Poisson GLM failure due to wrong initialization Poisson GLM fails for many standard data sets (see example in test or JIRA). The issue is incorrect initialization leading to almost zero

spark git commit: [SPARK-18686][SPARKR][ML] Several cleanup and improvements for spark.logit.

Repository: spark Updated Branches: refs/heads/branch-2.1 3750c6e9b -> 340e9aea4 [SPARK-18686][SPARKR][ML] Several cleanup and improvements for spark.logit. ## What changes were proposed in this pull request? Several cleanup and improvements for ```spark.logit```: * ```summary``` should

spark git commit: [SPARK-18686][SPARKR][ML] Several cleanup and improvements for spark.logit.

Repository: spark Updated Branches: refs/heads/master 5c6bcdbda -> 90b59d1bf [SPARK-18686][SPARKR][ML] Several cleanup and improvements for spark.logit. ## What changes were proposed in this pull request? Several cleanup and improvements for ```spark.logit```: * ```summary``` should return